[rules-users] EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[rules-users] EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule

kanderson
The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT:

(Stream mode)

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end


I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session.  While the event is waiting, I see a FactCount of 1 in the session.  After the rule fires, the fact count goes to 0.  However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects.

Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact?



_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule

kanderson
Correction:  The original post did not include another rule that exists in the stream.  The memory leak does not appear unless both rules are active in the stream.

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end

/* When CLEAR, and buffered, clear them both out */
rule "forget it ever happened"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
$raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId)
then
System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
delete($clear);
delete($raise);
end


On Jul 10, 2014, at 2:50 PM, Kent Anderson <[hidden email]> wrote:

The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT:

(Stream mode)

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end


I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session.  While the event is waiting, I see a FactCount of 1 in the session.  After the rule fires, the fact count goes to 0.  However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects.

Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact?

<PastedGraphic-1.png>

<PastedGraphic-2.png>
_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule

kanderson
We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): 

The rule “forget it ever happened” (seen below) causes the problem.  Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak.  Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS.  It’s ugly, but it resolved the issue.

query existsRaise($id)
$raise : MyEvent( eventState == EventState.RAISE, eventId == $id )
end

rule "process clear"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
then
QueryResults results = kcontext.getKieRuntime().getQueryResults( "existsRaise", $clearId );
if (results.size() == 0) { 
System.out.println( "Forwarding CLEAR(" + $clearId + ")" ); 
} else {
        System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
        for (QueryResultsRow row : results){
        MyEvent raise = (MyEvent) row.get ("$raise");
        delete(raise);
}
}
delete($clear);
end

This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498.



On Jul 10, 2014, at 3:54 PM, Kent Anderson <[hidden email]> wrote:

Correction:  The original post did not include another rule that exists in the stream.  The memory leak does not appear unless both rules are active in the stream.

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end

/* When CLEAR, and buffered, clear them both out */
rule "forget it ever happened"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
$raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId)
then
System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
delete($clear);
delete($raise);
end


On Jul 10, 2014, at 2:50 PM, Kent Anderson <[hidden email]> wrote:

The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT:

(Stream mode)

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end


I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session.  While the event is waiting, I see a FactCount of 1 in the session.  After the rule fires, the fact count goes to 0.  However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects.

Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact?

<PastedGraphic-1.png>

<PastedGraphic-2.png>
_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule

Davide Sottara
Thanks for reporting, we'll investigate both this and 498
Davide

On 07/11/2014 08:38 PM, Kent Anderson wrote:
We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): 

The rule “forget it ever happened” (seen below) causes the problem.  Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak.  Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS.  It’s ugly, but it resolved the issue.

query existsRaise($id)
$raise : MyEvent( eventState == EventState.RAISE, eventId == $id )
end

rule "process clear"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
then
QueryResults results = kcontext.getKieRuntime().getQueryResults( "existsRaise", $clearId );
if (results.size() == 0) { 
System.out.println( "Forwarding CLEAR(" + $clearId + ")" ); 
} else {
        System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
        for (QueryResultsRow row : results){
        MyEvent raise = (MyEvent) row.get ("$raise");
        delete(raise);
}
}
delete($clear);
end

This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498.



On Jul 10, 2014, at 3:54 PM, Kent Anderson <[hidden email]> wrote:

Correction:  The original post did not include another rule that exists in the stream.  The memory leak does not appear unless both rules are active in the stream.

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end

/* When CLEAR, and buffered, clear them both out */
rule "forget it ever happened"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
$raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId)
then
System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
delete($clear);
delete($raise);
end


On Jul 10, 2014, at 2:50 PM, Kent Anderson <[hidden email]> wrote:

The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT:

(Stream mode)

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end


I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session.  While the event is waiting, I see a FactCount of 1 in the session.  After the rule fires, the fact count goes to 0.  However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects.

Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact?

<PastedGraphic-1.png>

<PastedGraphic-2.png>
_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users



_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule

Mark Proctor
In reply to this post by kanderson
Could you submit a unit test as a pull request?

Add it to here, and follow existing conventions:

Mark
On 11 Jul 2014, at 20:38, Kent Anderson <[hidden email]> wrote:

We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): 

The rule “forget it ever happened” (seen below) causes the problem.  Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak.  Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS.  It’s ugly, but it resolved the issue.

query existsRaise($id)
$raise : MyEvent( eventState == EventState.RAISE, eventId == $id )
end

rule "process clear"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
then
QueryResults results = kcontext.getKieRuntime().getQueryResults( "existsRaise", $clearId );
if (results.size() == 0) { 
System.out.println( "Forwarding CLEAR(" + $clearId + ")" ); 
} else {
        System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
        for (QueryResultsRow row : results){
        MyEvent raise = (MyEvent) row.get ("$raise");
        delete(raise);
}
}
delete($clear);
end

This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498.



On Jul 10, 2014, at 3:54 PM, Kent Anderson <[hidden email]> wrote:

Correction:  The original post did not include another rule that exists in the stream.  The memory leak does not appear unless both rules are active in the stream.

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end

/* When CLEAR, and buffered, clear them both out */
rule "forget it ever happened"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
$raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId)
then
System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
delete($clear);
delete($raise);
end


On Jul 10, 2014, at 2:50 PM, Kent Anderson <[hidden email]> wrote:

The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT:

(Stream mode)

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end


I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session.  While the event is waiting, I see a FactCount of 1 in the session.  After the rule fires, the fact count goes to 0.  However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects.

Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact?

<PastedGraphic-1.png>

<PastedGraphic-2.png>
_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule

kanderson
Is there a reliable way to find an EventFactHandle instance buried within the TupleEntryQueue’s and the PhreakPropagationContext’s?

The multiple containers of interface/impl’s makes it nearly impossible to write a “good” test that can peak under the hood where this memory leak is being held.  If there is a recommended way to do it, I’d be happy to do so.




On Jul 13, 2014, at 7:31 PM, Mark Proctor <[hidden email]> wrote:

Could you submit a unit test as a pull request?

Add it to here, and follow existing conventions:

Mark
On 11 Jul 2014, at 20:38, Kent Anderson <[hidden email]> wrote:

We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): 

The rule “forget it ever happened” (seen below) causes the problem.  Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak.  Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS.  It’s ugly, but it resolved the issue.

query existsRaise($id)
$raise : MyEvent( eventState == EventState.RAISE, eventId == $id )
end

rule "process clear"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
then
QueryResults results = kcontext.getKieRuntime().getQueryResults( "existsRaise", $clearId );
if (results.size() == 0) { 
System.out.println( "Forwarding CLEAR(" + $clearId + ")" ); 
} else {
        System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
        for (QueryResultsRow row : results){
        MyEvent raise = (MyEvent) row.get ("$raise");
        delete(raise);
}
}
delete($clear);
end

This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498.



On Jul 10, 2014, at 3:54 PM, Kent Anderson <[hidden email]> wrote:

Correction:  The original post did not include another rule that exists in the stream.  The memory leak does not appear unless both rules are active in the stream.

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end

/* When CLEAR, and buffered, clear them both out */
rule "forget it ever happened"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
$raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId)
then
System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
delete($clear);
delete($raise);
end


On Jul 10, 2014, at 2:50 PM, Kent Anderson <[hidden email]> wrote:

The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT:

(Stream mode)

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end


I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session.  While the event is waiting, I see a FactCount of 1 in the session.  After the rule fires, the fact count goes to 0.  However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects.

Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact?

<PastedGraphic-1.png>

<PastedGraphic-2.png>
_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule

Davide Sottara
I'd need to check, but if you can put together the basics of the test case - the combination of rules and facts that causes the leak,
we'll add the checks to ensure it's solved. Mark should have given you enough pointers to the style we use for test cases.
We really appreciate your collaboration, and since this is a critical bug we'll fix it asap
Thanks!!
Davide

On 07/14/2014 08:33 PM, Kent Anderson wrote:
Is there a reliable way to find an EventFactHandle instance buried within the TupleEntryQueue’s and the PhreakPropagationContext’s?

The multiple containers of interface/impl’s makes it nearly impossible to write a “good” test that can peak under the hood where this memory leak is being held.  If there is a recommended way to do it, I’d be happy to do so.




On Jul 13, 2014, at 7:31 PM, Mark Proctor <[hidden email]> wrote:

Could you submit a unit test as a pull request?

Add it to here, and follow existing conventions:

Mark
On 11 Jul 2014, at 20:38, Kent Anderson <[hidden email]> wrote:

We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): 

The rule “forget it ever happened” (seen below) causes the problem.  Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak.  Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS.  It’s ugly, but it resolved the issue.

query existsRaise($id)
$raise : MyEvent( eventState == EventState.RAISE, eventId == $id )
end

rule "process clear"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
then
QueryResults results = kcontext.getKieRuntime().getQueryResults( "existsRaise", $clearId );
if (results.size() == 0) { 
System.out.println( "Forwarding CLEAR(" + $clearId + ")" ); 
} else {
        System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
        for (QueryResultsRow row : results){
        MyEvent raise = (MyEvent) row.get ("$raise");
        delete(raise);
}
}
delete($clear);
end

This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498.



On Jul 10, 2014, at 3:54 PM, Kent Anderson <[hidden email]> wrote:

Correction:  The original post did not include another rule that exists in the stream.  The memory leak does not appear unless both rules are active in the stream.

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end

/* When CLEAR, and buffered, clear them both out */
rule "forget it ever happened"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
$raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId)
then
System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
delete($clear);
delete($raise);
end


On Jul 10, 2014, at 2:50 PM, Kent Anderson <[hidden email]> wrote:

The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT:

(Stream mode)

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end


I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session.  While the event is waiting, I see a FactCount of 1 in the session.  After the rule fires, the fact count goes to 0.  However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects.

Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact?

<PastedGraphic-1.png>

<PastedGraphic-2.png>
_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users



_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule

kanderson
https://github.com/droolsjbpm/drools/pull/358

Per your advice, this pull request sets up the memory leak, but it cannot verify that it exists without diving into internal drools structures. 

Please let me know what else we can do to facilitate fixing this issue.


On Jul 14, 2014, at 2:09 PM, Davide Sottara <[hidden email]> wrote:

I'd need to check, but if you can put together the basics of the test case - the combination of rules and facts that causes the leak,
we'll add the checks to ensure it's solved. Mark should have given you enough pointers to the style we use for test cases.
We really appreciate your collaboration, and since this is a critical bug we'll fix it asap
Thanks!!
Davide

On 07/14/2014 08:33 PM, Kent Anderson wrote:
Is there a reliable way to find an EventFactHandle instance buried within the TupleEntryQueue’s and the PhreakPropagationContext’s?

The multiple containers of interface/impl’s makes it nearly impossible to write a “good” test that can peak under the hood where this memory leak is being held.  If there is a recommended way to do it, I’d be happy to do so.


<Mail Attachment.png>


On Jul 13, 2014, at 7:31 PM, Mark Proctor <[hidden email]> wrote:

Could you submit a unit test as a pull request?

Add it to here, and follow existing conventions:

Mark
On 11 Jul 2014, at 20:38, Kent Anderson <[hidden email]> wrote:

We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): 

The rule “forget it ever happened” (seen below) causes the problem.  Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak.  Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS.  It’s ugly, but it resolved the issue.

query existsRaise($id)
$raise : MyEvent( eventState == EventState.RAISE, eventId == $id )
end

rule "process clear"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
then
QueryResults results = kcontext.getKieRuntime().getQueryResults( "existsRaise", $clearId );
if (results.size() == 0) { 
System.out.println( "Forwarding CLEAR(" + $clearId + ")" ); 
} else {
         System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
        for (QueryResultsRow row : results){
         MyEvent raise = (MyEvent) row.get ("$raise");
         delete(raise);
}
}
delete($clear);
end

This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498.



On Jul 10, 2014, at 3:54 PM, Kent Anderson <[hidden email]> wrote:

Correction:  The original post did not include another rule that exists in the stream.  The memory leak does not appear unless both rules are active in the stream.

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end

/* When CLEAR, and buffered, clear them both out */
rule "forget it ever happened"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
$raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId)
then
System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
delete($clear);
delete($raise);
end


On Jul 10, 2014, at 2:50 PM, Kent Anderson <[hidden email]> wrote:

The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT:

(Stream mode)

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end


I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session.                                          While the event is waiting, I see a FactCount of 1 in the session.  After the rule fires, the fact count goes to 0.  However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects.

Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact?

<PastedGraphic-1.png>

<PastedGraphic-2.png>
_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users



_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule

Davide Sottara
Thanks, we'll look at it tomorrow
Davide

On 07/15/2014 02:09 AM, Kent Anderson wrote:
https://github.com/droolsjbpm/drools/pull/358

Per your advice, this pull request sets up the memory leak, but it cannot verify that it exists without diving into internal drools structures. 

Please let me know what else we can do to facilitate fixing this issue.


On Jul 14, 2014, at 2:09 PM, Davide Sottara [hidden email] wrote:

I'd need to check, but if you can put together the basics of the test case - the combination of rules and facts that causes the leak,
we'll add the checks to ensure it's solved. Mark should have given you enough pointers to the style we use for test cases.
We really appreciate your collaboration, and since this is a critical bug we'll fix it asap
Thanks!!
Davide

On 07/14/2014 08:33 PM, Kent Anderson wrote:
Is there a reliable way to find an EventFactHandle instance buried within the TupleEntryQueue’s and the PhreakPropagationContext’s?

The multiple containers of interface/impl’s makes it nearly impossible to write a “good” test that can peak under the hood where this memory leak is being held.  If there is a recommended way to do it, I’d be happy to do so.


<Mail Attachment.png>


On Jul 13, 2014, at 7:31 PM, Mark Proctor <[hidden email]> wrote:

Could you submit a unit test as a pull request?

Add it to here, and follow existing conventions:

Mark
On 11 Jul 2014, at 20:38, Kent Anderson <[hidden email]> wrote:

We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): 

The rule “forget it ever happened” (seen below) causes the problem.  Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak.  Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS.  It’s ugly, but it resolved the issue.

query existsRaise($id)
$raise : MyEvent( eventState == EventState.RAISE, eventId == $id )
end

rule "process clear"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
then
QueryResults results = kcontext.getKieRuntime().getQueryResults( "existsRaise", $clearId );
if (results.size() == 0) { 
System.out.println( "Forwarding CLEAR(" + $clearId + ")" ); 
} else {
         System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
        for (QueryResultsRow row : results){
         MyEvent raise = (MyEvent) row.get ("$raise");
         delete(raise);
}
}
delete($clear);
end

This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498.



On Jul 10, 2014, at 3:54 PM, Kent Anderson <[hidden email]> wrote:

Correction:  The original post did not include another rule that exists in the stream.  The memory leak does not appear unless both rules are active in the stream.

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end

/* When CLEAR, and buffered, clear them both out */
rule "forget it ever happened"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
$raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId)
then
System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
delete($clear);
delete($raise);
end


On Jul 10, 2014, at 2:50 PM, Kent Anderson <[hidden email]> wrote:

The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT:

(Stream mode)

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end


I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session.                                          While the event is waiting, I see a FactCount of 1 in the session.  After the rule fires, the fact count goes to 0.  However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects.

Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact?

<PastedGraphic-1.png>

<PastedGraphic-2.png>
_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users



_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users



_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule

kanderson
This issue has been resolved by https://issues.jboss.org/browse/DROOLS-516 (a discussion in the comments in https://issues.jboss.org/browse/DROOLS-498 is relevant, as this issue was originally reported as related to DROOLS-498).


On Jul 14, 2014, at 6:11 PM, Davide Sottara <[hidden email]> wrote:

Thanks, we'll look at it tomorrow
Davide

On 07/15/2014 02:09 AM, Kent Anderson wrote:
https://github.com/droolsjbpm/drools/pull/358

Per your advice, this pull request sets up the memory leak, but it cannot verify that it exists without diving into internal drools structures. 

Please let me know what else we can do to facilitate fixing this issue.


On Jul 14, 2014, at 2:09 PM, Davide Sottara [hidden email] wrote:

I'd need to check, but if you can put together the basics of the test case - the combination of rules and facts that causes the leak,
we'll add the checks to ensure it's solved. Mark should have given you enough pointers to the style we use for test cases.
We really appreciate your collaboration, and since this is a critical bug we'll fix it asap
Thanks!!
Davide

On 07/14/2014 08:33 PM, Kent Anderson wrote:
Is there a reliable way to find an EventFactHandle instance buried within the TupleEntryQueue’s and the PhreakPropagationContext’s?

The multiple containers of interface/impl’s makes it nearly impossible to write a “good” test that can peak under the hood where this memory leak is being held.  If there is a recommended way to do it, I’d be happy to do so.


<Mail Attachment.png>


On Jul 13, 2014, at 7:31 PM, Mark Proctor <[hidden email]> wrote:

Could you submit a unit test as a pull request?

Add it to here, and follow existing conventions:

Mark
On 11 Jul 2014, at 20:38, Kent Anderson <[hidden email]> wrote:

We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): 

The rule “forget it ever happened” (seen below) causes the problem.  Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak.  Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS.  It’s ugly, but it resolved the issue.

query existsRaise($id)
$raise : MyEvent( eventState == EventState.RAISE, eventId == $id )
end

rule "process clear"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
then
QueryResults results = kcontext.getKieRuntime().getQueryResults( "existsRaise", $clearId );
if (results.size() == 0) { 
System.out.println( "Forwarding CLEAR(" + $clearId + ")" ); 
} else {
         System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
        for (QueryResultsRow row : results){
         MyEvent raise = (MyEvent) row.get ("$raise");
         delete(raise);
}
}
delete($clear);
end

This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498.



On Jul 10, 2014, at 3:54 PM, Kent Anderson <[hidden email]> wrote:

Correction:  The original post did not include another rule that exists in the stream.  The memory leak does not appear unless both rules are active in the stream.

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end

/* When CLEAR, and buffered, clear them both out */
rule "forget it ever happened"
no-loop
when
$clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId)
$raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId)
then
System.out.println("Forgetting RAISE/CLEAR(" + $clearId + ")");
delete($clear);
delete($raise);
end


On Jul 10, 2014, at 2:50 PM, Kent Anderson <[hidden email]> wrote:

The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT:

(Stream mode)

declare MyEvent 
  @role(event) 
  @timestamp(timestamp) 
end 

/* If a RAISE is buffered for N seconds, send it out */
rule "forward raise"
no-loop
duration (3s)
when
$raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId)
then
System.out.println("Forwarding RAISE(" + $raiseId + ")");
delete($raise);
end


I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session.                                          While the event is waiting, I see a FactCount of 1 in the session.  After the rule fires, the fact count goes to 0.  However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects.

Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact?

<PastedGraphic-1.png>

<PastedGraphic-2.png>
_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users



_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users



_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users

_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users