OptaPlanner: Solver.isEveryProblemFactChangeProcessed() return true before the last fact change is processed

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

OptaPlanner: Solver.isEveryProblemFactChangeProcessed() return true before the last fact change is processed

Hagai
Using OptaPlanner 6.0.1.Final and following the documentation for real-time planning:
"Alternatively, you can subscribe to the BestSolutionChangedEvent.
A BestSolutionChangedEvent doesn't guarantee that every ProblemFactChange has been
processed already, so check Solver.isEveryProblemFactChangeProcessed() and ignore any
BestSolutionChangedEvent fired while that method returns false."

However, Solver.isEveryProblemFactChangeProcessed() return true before the last fact change is processed.
This is documented in the code DefaultSolver.java: // TODO bug: the last ProblemFactChange might already been polled, but not processed yet

I believe this can be fixed using the following code in DefaultSolver.checkProblemFactChanges:

            ProblemFactChange problemFactChange = problemFactChangeQueue.peek();
            while (problemFactChange != null) {
                score = doProblemFactChange(problemFactChange);
                problemFactChangeQueue.poll();
                count++;
                problemFactChange = problemFactChangeQueue.peek();
            }

This way the queue will not be empty until the fact change is processed.
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] OptaPlanner: Solver.isEveryProblemFactChangeProcessed() return true before the last fact change is processed

ge0ffrey
Administrator

On 30-04-14 12:17, Hagai wrote:

> Using OptaPlanner 6.0.1.Final and following the documentation for real-time
> planning:
> "Alternatively, you can subscribe to the BestSolutionChangedEvent.
> A BestSolutionChangedEvent doesn't guarantee that every ProblemFactChange
> has been
> processed already, so check Solver.isEveryProblemFactChangeProcessed() and
> ignore any
> BestSolutionChangedEvent fired while that method returns false."
>
> However, Solver.isEveryProblemFactChangeProcessed() return true before the
> last fact change is processed.
> This is documented in the code DefaultSolver.java: *// TODO bug: the last
> ProblemFactChange might already been polled, but not processed yet*
>
> I believe this can be fixed using the following code in
> DefaultSolver.checkProblemFactChanges:
>
>              ProblemFactChange problemFactChange =
> problemFactChangeQueue*.peek()*;
>              while (problemFactChange != null) {
>                  score = doProblemFactChange(problemFactChange);
>                  *problemFactChangeQueue.poll();*
>                  count++;
>                  problemFactChange = problemFactChangeQueue*.peek()*;
>              }
>
> This way the queue will not be empty until the fact change is processed.

Thanks for reporting.
This is definitely an issue indeed, if
isEveryProblemFactChangeProcessed() is called from a different thread
than then solver thread.
The javadoc clearly promises that isEveryProblemFactChangeProcessed() is
thread-safe.
I 'll take a look into fixing it with your suggestion.

Note: In practice, you 'll usually call
isEveryProblemFactChangeProcessed() in the bestSolutionChanged() event,
which is called in the solver thread,
so it's not an issue in that particular case.
For 6.1.0.Beta4, I 've cleaned up the docs to describe that usage
pattern better:
https://github.com/droolsjbpm/optaplanner/commit/256fa1fe285392f8ad81b1a40816db641a768bb9


>
>
>
> --
> View this message in context: http://drools.46999.n3.nabble.com/OptaPlanner-Solver-isEveryProblemFactChangeProcessed-return-true-before-the-last-fact-change-is-procd-tp4029389.html
> Sent from the Drools: User forum mailing list archive at Nabble.com.
> _______________________________________________
> rules-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/rules-users
>


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] OptaPlanner: Solver.isEveryProblemFactChangeProcessed() return true before the last fact change is processed

Hagai
I do use it from other threads in the following 2 scenarios:

1. In a thread that monitors a stream of messages with fact changes. Just before the thread add a new problem fact change to the solver, the code checks if the message is relevant by examining the current solution facts. But if there is a pending problem fact change in the queue, this evaluation cannot be done and a new problem fact change must be added (even if it will not do any change when processed). The idea is to minimize the situations where a message does not result in any change to facts and the solver restart itself to process the a fact change that does not change anything.

2. The solver produce many new solutions when starting and after fact changes. After some time, less new solutions are found. But when a new solution is found, some additional improvements are usually found shortly after. To reduce the amount of new best solutions produced by the application, a thread is scheduled to read/save/send the best solution after a short idle time (no new best solution for X ms). To make sure the best solution is valid, the code checks if every problem facts change processed. If there are still fact changes to process, the solver will produce another best solution shortly with the updated facts.

I hope this fix is simple to do so I can remove my workaround.

BTW, I'm looking forward to use the new demon mode (instead of a similar implementation outside of the solver).
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] OptaPlanner: Solver.isEveryProblemFactChangeProcessed() return true before the last fact change is processed

ge0ffrey
Administrator

On 30-04-14 15:07, Hagai wrote:

> I do use it from other threads in the following 2 scenarios:
>
> 1. In a thread that monitors a stream of messages with fact changes. Just
> before the thread add a new problem fact change to the solver, the code
> checks if the message is relevant by examining the current solution facts.
> But if there is a pending problem fact change in the queue, this evaluation
> cannot be done and a new problem fact change must be added (even if it will
> not do any change when processed). The idea is to minimize the situations
> where a message does not result in any change to facts and the solver
> restart itself to process the a fact change that does not change anything.
>
> 2. The solver produce many new solutions when starting and after fact
> changes. After some time, less new solutions are found. But when a new
> solution is found, some additional improvements are usually found shortly
> after. To reduce the amount of new best solutions produced by the
> application, a thread is scheduled to read/save/send the best solution after
> a short idle time (no new best solution for X ms). To make sure the best
> solution is valid, the code checks if every problem facts change processed.
> If there are still fact changes to process, the solver will produce another
> best solution shortly with the updated facts.

Thanks for the feedback: it's very helpful for me to know how it's used
better.
Out of interest: What kind of planning problem are you solving?

>
> I hope this fix is simple to do so I can remove my workaround.

Yes, I'll fix the problem (I might go for an alternative on the
peek()'s) on Friday (holiday tomorrow with plans).

>
> BTW, I'm looking forward to use the new demon mode (instead of a similar
> implementation outside of the solver).

Grab 6.1.0-SNAPSHOT if you can't wait. Or read the SNAPSHOT docs.
Feedback/concerns welcome.

>
>
>
> --
> View this message in context: http://drools.46999.n3.nabble.com/OptaPlanner-Solver-isEveryProblemFactChangeProcessed-return-true-before-the-last-fact-change-is-procd-tp4029389p4029392.html
> Sent from the Drools: User forum mailing list archive at Nabble.com.
> _______________________________________________
> rules-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/rules-users
>


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] OptaPlanner: Solver.isEveryProblemFactChangeProcessed() return true before the last fact change is processed

Hagai
I'm working on a scheduling solution for traveling engineers. Similar to vehicle routing but with daily routes and more complex constrains (like dependencies between visits).


Thanks,
Hagai Shatz

On 30 Apr 2014, at 20:31, Geoffrey De Smet <[hidden email]> wrote:


> On 30-04-14 15:07, Hagai wrote:
> I do use it from other threads in the following 2 scenarios:
>
> 1. In a thread that monitors a stream of messages with fact changes. Just
> before the thread add a new problem fact change to the solver, the code
> checks if the message is relevant by examining the current solution facts.
> But if there is a pending problem fact change in the queue, this evaluation
> cannot be done and a new problem fact change must be added (even if it will
> not do any change when processed). The idea is to minimize the situations
> where a message does not result in any change to facts and the solver
> restart itself to process the a fact change that does not change anything.
>
> 2. The solver produce many new solutions when starting and after fact
> changes. After some time, less new solutions are found. But when a new
> solution is found, some additional improvements are usually found shortly
> after. To reduce the amount of new best solutions produced by the
> application, a thread is scheduled to read/save/send the best solution after
> a short idle time (no new best solution for X ms). To make sure the best
> solution is valid, the code checks if every problem facts change processed.
> If there are still fact changes to process, the solver will produce another
> best solution shortly with the updated facts.

Thanks for the feedback: it's very helpful for me to know how it's used
better.
Out of interest: What kind of planning problem are you solving?

>
> I hope this fix is simple to do so I can remove my workaround.

Yes, I'll fix the problem (I might go for an alternative on the
peek()'s) on Friday (holiday tomorrow with plans).

>
> BTW, I'm looking forward to use the new demon mode (instead of a similar
> implementation outside of the solver).

Grab 6.1.0-SNAPSHOT if you can't wait. Or read the SNAPSHOT docs.
Feedback/concerns welcome.

>
>
>
> --
> View this message in context: http://drools.46999.n3.nabble.com/OptaPlanner-Solver-isEveryProblemFactChangeProcessed-return-true-before-the-last-fact-change-is-procd-tp4029389p4029392.html
> Sent from the Drools: User forum mailing list archive at Nabble.com.
> _______________________________________________
> rules-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/rules-users


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] OptaPlanner: Solver.isEveryProblemFactChangeProcessed() return true before the last fact change is processed

ge0ffrey
Administrator
Bug fixed:
https://github.com/droolsjbpm/optaplanner/commit/4389730069327837fb375f3b1702911a1ee7e294
   I went for a slightly different solution, to confine the logic as
much as possible in the BasicPlumbingTermination class.

Thanks for reporting :)

On 01-05-14 10:36, Hagai Shatz wrote:

> I'm working on a scheduling solution for traveling engineers. Similar to vehicle routing but with daily routes and more complex constrains (like dependencies between visits).
>
>
> Thanks,
> Hagai Shatz
>
> On 30 Apr 2014, at 20:31, Geoffrey De Smet <[hidden email]> wrote:
>
>
>> On 30-04-14 15:07, Hagai wrote:
>> I do use it from other threads in the following 2 scenarios:
>>
>> 1. In a thread that monitors a stream of messages with fact changes. Just
>> before the thread add a new problem fact change to the solver, the code
>> checks if the message is relevant by examining the current solution facts.
>> But if there is a pending problem fact change in the queue, this evaluation
>> cannot be done and a new problem fact change must be added (even if it will
>> not do any change when processed). The idea is to minimize the situations
>> where a message does not result in any change to facts and the solver
>> restart itself to process the a fact change that does not change anything.
>>
>> 2. The solver produce many new solutions when starting and after fact
>> changes. After some time, less new solutions are found. But when a new
>> solution is found, some additional improvements are usually found shortly
>> after. To reduce the amount of new best solutions produced by the
>> application, a thread is scheduled to read/save/send the best solution after
>> a short idle time (no new best solution for X ms). To make sure the best
>> solution is valid, the code checks if every problem facts change processed.
>> If there are still fact changes to process, the solver will produce another
>> best solution shortly with the updated facts.
> Thanks for the feedback: it's very helpful for me to know how it's used
> better.
> Out of interest: What kind of planning problem are you solving?
>
>> I hope this fix is simple to do so I can remove my workaround.
> Yes, I'll fix the problem (I might go for an alternative on the
> peek()'s) on Friday (holiday tomorrow with plans).
>
>> BTW, I'm looking forward to use the new demon mode (instead of a similar
>> implementation outside of the solver).
> Grab 6.1.0-SNAPSHOT if you can't wait. Or read the SNAPSHOT docs.
> Feedback/concerns welcome.
>
>>
>>
>> --
>> View this message in context: http://drools.46999.n3.nabble.com/OptaPlanner-Solver-isEveryProblemFactChangeProcessed-return-true-before-the-last-fact-change-is-procd-tp4029389p4029392.html
>> Sent from the Drools: User forum mailing list archive at Nabble.com.
>> _______________________________________________
>> rules-users mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/rules-users
>
> _______________________________________________
> rules-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/rules-users
>
> _______________________________________________
> rules-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/rules-users
>


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users