Prevent re-evaluation of events in stream mode

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Prevent re-evaluation of events in stream mode

sumantp
Hi,

I've written a simple rule to illustrate the issue I'm running into. Transaction events representing amounts deposited into a bank account are inserted sequentially. The goal is to detect a transaction that is higher than the average from a rolling window. What I'm expecting is that only the most recent insertion is compared with the average from the window. However, I'm observing that all previous events in the window are re-evaluated each time a new event is inserted. So events that previously were below the average (at that point in time) may now be above the average since subsequent events brought the average down. I don't want those transactions to be re-evaluated.

Here's my rule:
        declare Transaction
                @role(event)
        end
       
        rule "Higher than average amount"
        when
                $transaction : Transaction($account : getAccount(), $amount : getAmount())
                Number($avg : doubleValue, $amount > $avg) from accumulate(Transaction(getAccount() == $account, $amt : getAmount()) over window:length(100), average($amt))
        then
                System.out.println("\t***ALERT***: Higher than average transaction amount (Average = " + $avg + "): " + $transaction.toString());
        end


Here's a snippet from my Java class:
        public static final void main(String[] args) {
                try {
                        // Load the knowledge base
                        KieServices ks = KieServices.Factory.get();
                        KieContainer kContainer = ks.getKieClasspathContainer();
                        KieSession kSession = kContainer.newKieSession("ksession-rules");

                        // Insert events
                        List<Transaction> list = getEvents();
                        for (Transaction event : list) {
                                printEvent(event);
                                kSession.insert(event);
                                kSession.fireAllRules();
                        }
                } catch (Throwable t) {
                        t.printStackTrace();
                }
        }

        private static List<Transaction> getEvents() {
                List<Transaction> list = new Vector<Transaction>();

                list.add(new Transaction(1, 600));
                list.add(new Transaction(1, 600));
                list.add(new Transaction(1, 800)); // This should trigger an alert
                list.add(new Transaction(1, 100)); // This should NOT re-evaluate previous transactions

                return list;
        }

This is my kmodule.xml:
        <?xml version="1.0" encoding="UTF-8"?>
        <kmodule xmlns="http://jboss.org/kie/6.0.0/kmodule">
    <kbase name="rules" eventProcessingMode="stream" packages="rules">
        <ksession name="ksession-rules"/>
    </kbase>
        </kmodule>

I'm using 6.0.0.Final.

Any help is appreciated!
Reply | Threaded
Open this post in threaded view
|

Re: Prevent re-evaluation of events in stream mode

sumantp
I forgot to include some output from my test. It shows that the two $600 transactions do not initially trigger alerts (as expected), the $800 one triggers an alert (as expected), but the $100 one results in alerts for the first 3 transaction (not expected).

Inserting Account = 1, Amount = 600.0, current average = 600.0
Inserting Account = 1, Amount = 600.0, current average = 600.0
Inserting Account = 1, Amount = 800.0, current average = 666.6666666666666
        ***ALERT***: Higher than average transaction amount (Average = 666.6666666666666): Account = 1, Amount = 800.0
Inserting Account = 1, Amount = 100.0, current average = 525.0
        ***ALERT***: Higher than average transaction amount (Average = 525.0): Account = 1, Amount = 800.0
        ***ALERT***: Higher than average transaction amount (Average = 525.0): Account = 1, Amount = 600.0
        ***ALERT***: Higher than average transaction amount (Average = 525.0): Account = 1, Amount = 600.0
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Prevent re-evaluation of events in stream mode

laune
accumulate (irrespective of its form and particular coding) is
absolutely fickle and finicky in a changing fact data base. The basic
caveat is that each change to the domain of the accumulat CE starts a
new game.

Forget accumulate, roll your own!

Set up a fact type Avg that's capable of holding the running average
for an account (and other data that may be useful). One rule inserts
an Avg if it isn't there for a new Transactions and initializes the
average. Another rule should just extend the average while the count
is below 100. And for the 101st Transaction to arrive,  the third rule
checks the amount and modifies the average, too. Obviously, you'll
have to keep 100 amounts in Avg as well in order to remove the
contribution of the 1st Transaction, or you just continue to extend
the average.

Note that this permits you to drop Transactions from the system
earlier than with accumulate, which should more than compensate for
the additional memory requirements due to Avg for each observed
account.

Average may be a poor threshold for raising an alarm, but I suspect
you haven't been showing all of your cards ;-)

-W


On 09/12/2013, sumantp <[hidden email]> wrote:

> I forgot to include some output from my test. It shows that the two $600
> transactions do not initially trigger alerts (as expected), the $800 one
> triggers an alert (as expected), but the $100 one results in alerts for the
> first 3 transaction (not expected).
>
> Inserting Account = 1, Amount = 600.0, current average = 600.0
> Inserting Account = 1, Amount = 600.0, current average = 600.0
> Inserting Account = 1, Amount = 800.0, current average = 666.6666666666666
> ***ALERT***: Higher than average transaction amount (Average =
> 666.6666666666666): Account = 1, Amount = 800.0
> Inserting Account = 1, Amount = 100.0, current average = 525.0
> ***ALERT***: Higher than average transaction amount (Average = 525.0):
> Account = 1, Amount = 800.0
> ***ALERT***: Higher than average transaction amount (Average = 525.0):
> Account = 1, Amount = 600.0
> ***ALERT***: Higher than average transaction amount (Average = 525.0):
> Account = 1, Amount = 600.0
>
>
>
>
> --
> View this message in context:
> http://drools.46999.n3.nabble.com/Prevent-re-evaluation-of-events-in-stream-mode-tp4027171p4027172.html
> Sent from the Drools: User forum mailing list archive at Nabble.com.
> _______________________________________________
> rules-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/rules-users
>
_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Prevent re-evaluation of events in stream mode

Mark Proctor
In reply to this post by sumantp
Logically insert the average, in a separate rule. This will ensure there is an “average” fact that is maintained n the TMS. As it’s logically inserted, you don’t need to delete it.

After any Transaction has been evaluated, you need to set a flag on it, so it’s not processed again.

Mark


On 9 Dec 2013, at 17:59, sumantp <[hidden email]> wrote:

> Hi,
>
> I've written a simple rule to illustrate the issue I'm running into.
> Transaction events representing amounts deposited into a bank account are
> inserted sequentially. The goal is to detect a transaction that is higher
> than the average from a rolling window. What I'm expecting is that only the
> most recent insertion is compared with the average from the window. However,
> I'm observing that all previous events in the window are re-evaluated each
> time a new event is inserted. So events that previously were below the
> average (at that point in time) may now be above the average since
> subsequent events brought the average down. I don't want those transactions
> to be re-evaluated.
>
> Here's my rule:
> declare Transaction
> @role(event)
> end
>
> rule "Higher than average amount"
> when
> $transaction : Transaction($account : getAccount(), $amount : getAmount())
> Number($avg : doubleValue, $amount > $avg) from
> accumulate(Transaction(getAccount() == $account, $amt : getAmount()) over
> window:length(100), average($amt))
> then
> System.out.println("\t***ALERT***: Higher than average transaction amount
> (Average = " + $avg + "): " + $transaction.toString());
> end
>
>
> Here's a snippet from my Java class:
> public static final void main(String[] args) {
> try {
> // Load the knowledge base
> KieServices ks = KieServices.Factory.get();
> KieContainer kContainer = ks.getKieClasspathContainer();
> KieSession kSession = kContainer.newKieSession("ksession-rules");
>
> // Insert events
> List<Transaction> list = getEvents();
> for (Transaction event : list) {
> printEvent(event);
> kSession.insert(event);
> kSession.fireAllRules();
> }
> } catch (Throwable t) {
> t.printStackTrace();
> }
> }
>
> private static List<Transaction> getEvents() {
> List<Transaction> list = new Vector<Transaction>();
>
> list.add(new Transaction(1, 600));
> list.add(new Transaction(1, 600));
> list.add(new Transaction(1, 800)); // This should trigger an alert
> list.add(new Transaction(1, 100)); // This should NOT re-evaluate previous
> transactions
>
> return list;
> }
>
> This is my kmodule.xml:
> <?xml version="1.0" encoding="UTF-8"?>
> <kmodule xmlns="http://jboss.org/kie/6.0.0/kmodule">
>     <kbase name="rules" eventProcessingMode="stream" packages="rules">
>         <ksession name="ksession-rules"/>
>     </kbase>
> </kmodule>
>
> I'm using 6.0.0.Final.
>
> Any help is appreciated!
>
>
>
> --
> View this message in context: http://drools.46999.n3.nabble.com/Prevent-re-evaluation-of-events-in-stream-mode-tp4027171.html
> Sent from the Drools: User forum mailing list archive at Nabble.com.
> _______________________________________________
> rules-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/rules-users


_______________________________________________
rules-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/rules-users
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Prevent re-evaluation of events in stream mode

sumantp
In reply to this post by laune
Thanks Wolfgang! Interesting alternative. I'll try that.

Average is indeed a poor choice for the alarm and bank transactions are not really what I'm evaluating in my application ;-) I intend to use standard deviation with at least some number of data points. But it was easier to illustrate the event re-evaluation issue I was running into using a simple example.
Reply | Threaded
Open this post in threaded view
|

Re: [rules-users] Prevent re-evaluation of events in stream mode

sumantp
In reply to this post by Mark Proctor
Thanks Mark!

It seems like there are two alternatives - either roll my own average accumulation or set a flag in the event to prevent re-evaluation.