Welcome to the IDIOM Decision Products Knowledgebase.


Recent Comments

    Extending the Metaphor: Carole-Ann Matignon - blog post and Idiom response

    Mark Norton  7 August 2010 11:00:08 AM
    Carole-Ann's Post:

    Have you ever wondered why you should be using one business rules engine or another? Granted each product has its own specificities but it is amazing how many people care about the algorithm that the product implements.

    It may be a feminine trait, but I do not care to understand what a V6 or V8 engine is.  I don’t really care what engine is in my car to tell you the truth as long as it gets me from point A to point B in a safe and clean way (as clean as it can be unfortunately).

    I was a lot more enthusiastic about the RETE algorithm when I got to uncover the miracles it does many years ago.  It was quite useful in fact to start my Business Rules journey as a tools specialist.  As a user, it avoided me a few pitfalls but I do not believe that it is mandatory to learn the “How” of the various engines.  Let the tools manufacturers worry about it so that you don’t!

    Why you should not care

    To be brutally honest, you should only care that your engine does what you want and does it sufficiently well — regardless of whether your criteria are speed, functionality or anything else.

    People that worry too much about the low-level stuff sometimes forget the big picture.  So many false “best practices” have been going around because of a micro-benchmark that was unknowingly biased.  Knowing the algorithm may help you get out of those traps but, yet again, it may make you worry too much about over-analyzing your performance.  A strong design for your rules — often driven by the techies — could prevent you from achieving the flexibility you needed; the flexibility that ironically drove you to purchase a rules engine in the first place!

    Here’s one guideline you may not want to push to your business users…  RETE starts with a discrimination tree for each class being referenced in your business rules.  So all the conditions on Customers are linked together.  If you start with the most frequent conditions first — let’s say “State = CA” – and then least frequent — “Age is between 25 and 26″ –, you will be able to reuse that very node more often.  Your resulting RETE network will be smaller in number of nodes.  This is something you could do as part of a test for your own benchmarks but asking your business users to worry about condition frequency may be overkill.  Especially not knowing what your business rules will be tomorrow.  What if you get a bunch of rules for those 25-26 year-old customers?  Will you ask your business users to rewrite those rules?

    The second reason is that the possible performance gain is typically tiny.  Your rules may execute a tiny bit faster but you are very unlikely to make a difference that way.  Your time will be much better spent working on improving the I/Os of your system.  Never mind the fact that your rules engine might include some optimizations on top of the original RETE algorithm that take care of those internal network optimizations for you behind the scene.

    What matters then?

    I have been talking about the RETE algorithm as it is the most popular engine today — originally created by Charles Forgy.  Other algorithms have been used in the past by non-inference rules vendors as well as a complement to the inference engines in the market.  In some cases, one algorithm performs better than another one (sequential versus inference).  It would highly depend on the number of rules, the number of objects you process in your transactions and the nature of those rules (whether or not they “refer to each other”).

    The reality is that people pick one engine and they stick with it.  As a result of a tech support escalation, they may end up trying another one.  Back to my car analogy, even if you were given the opportunity to change your engine whenever you wanted, would you do it?  Would you have the economical engine for going to the store and switch to the high performance one for your longer trips on the highway?  Well, I know I would not.  The hybrid engine is perfect because it does the switch automatically for us.  If it was manual, I doubt that all drivers would use it appropriately for every traffic condition.  I am sure that a few would love to do that but the masses can’t be given that responsibility.

    What matter most is what you need and how you will use it.  Wenger introduced this ridiculously feature-rich swiss army knife.  87 tools!  I suspect that a couple of tools would likely suffice most of the time.  Although the giant swiss army knife might look cool in display, it may not be the most usable / convenient thing in the world.  

    Think of your rules engine the same way.  You need to find one that is good for your most likely needs in a way that will be convenient to use.  Be realistic about what features are worth your time.

    Idiom's response: Extend the Metaphor!

    Carole-Ann, you suggest that looking under the hood for engine size may not be useful for customers, provided that function and performance is adequate. Perhaps this could be extended to suggesting that you do not need to even buy a car if there is a better and more effective solution – move beyond the big picture to the even bigger picture, one in which the algorithm is neither visible nor relevant. It is always about the requirements. No point in discussing engine size if a car is not what you want.  The fastest way from London to Heathrow is not by car, so discussing engine power may not be relevant.

    Maybe the question is not about the ‘algorithm’ at all. Not so long ago an industry analyst told me that all Rete based rule engine vendors now offer sequential processing algorithms, but that none of the sequential algorithm vendors have introduced Rete. Further, no new Rete based rules engines have appeared in recent years (I confess that I have not verified these assertions personally). So I do wonder whether Rete is the most popular algorithm today as you assert. Similarly, I would be intrigued to understand what pitfalls you avoided by understanding the ‘miracles’ of Rete; more importantly, would these pitfalls be of any interest to a business person who is automating their decision making?

    Maybe not just the specifics of the algorithm, but the whole concept of the algorithm has passed its use-by date. In most systems, the cost of executing a process switch to a discrete rules server and back is likely to exceed the cost of the rules execution itself; or, if the rules are tightly coupled, business agility is compromised. Either way, the algorithm is not the issue.    

    Much more important are the issues of direct business oversight and control of decision automation; linkage between business strategy and operational decision making; auditability and rule transparency; and overall business agility. The concept of Rete is irrelevant to these issues.

    I think the bottom line is that the ease with which the business domain expert can create and prove business rules independently of IT technical assistance or implementation consideration (particularly algorithm but also including language, execution platform, OS, performance, etc ) is the key, and that any industrial strength implementation that achieves utterly reliable results within acceptable performance parameters is OK for the runtime – why would the business care otherwise?