As the concepts of decision modeling and management increasingly move from theory to practice, it has been interesting to note how it is perceived and approached by both business and IT. Perhaps not surprisingly, my experience has shown this to be a smoother transition for business. This is something we should expect as it presents a more natural representation of the business for the business. The IT side has certainly had a variety of tools available in numerous platforms to support this approach and the emergence of the Decision Model and Notation (DMN) standard now brings a cohesive view across industries. However, it does seem to present a few challenges to IT in this early stage of evolution.
The two challenges that seem to be most prevalent are the structure of rules within a decision and testing decisions/rules. I will only briefly discuss the former here as it is worthy of its own discussion in far more detail. The core issue is the reliance on decision tables as the sole representation for rules within a decision. Those who have worked with business rules over the years know that nearly all BRMS platforms provide a variety of metaphors for rule expression – IF/THEN rules (both technical and in business friendly language), decision tables, decision trees, workflows, etc. The guiding principles in choosing a rule representation have usually focused on performance and providing an appropriate and understandable business expression. I (and, I believe, many others) have always strongly believed there is no “one size fits all” methodology. Trying to force a square peg into a round hole by using only one form of rules can result in unnatural representations, an explosion in the number of rules, and occasional performance issues. To date, the projects on which we’ve been involved have used the following guiding principle: try for decision tables first but do not hesitate to move to other expressions where it makes sense. We’ve generated more precise guidelines for making this decision in a consistent fashion. These will be the topic of a future paper.
That brings us to testing. IT often tends to dive into the details and work from the bottom up. For decision testing, this implies an initial focus on individual rules. Most platforms have a number of tools available to facilitate this task (e.g., tools that stop you from generating an ill-formed or incomplete rule). Unit testing of rules often considers two types of testing:
- Positive Testing
- A rule will fire when all the conditions on the “IF” side are satisfied
- Negative Testing
- A rule will not fire when the conditions on the “IF” side are not satisfied.
Negative testing occurs implicitly as testing that an individual condition being false would cause the rule to not execute would be testing that the rule engine itself works as specified, and not the actual rule itself. There are a number of well-known best practices here and this process should be relatively straightforward.
Business, of course, doesn’t really care that an individual rule works. They want to know whether or not the decision rendered is correct. “Scenario” testing in some context has been an integral part of testing a rule system and, again, most BRMS platforms have supported it in some fashion. At one point it was often rolled up into a very large, general category of “system” testing. The emphasis on decisions now casts this in a more significant light.
Scenario testing validates that the rules as a complete rule set (or decision) provide the proper results when executed. This approach is necessarily a joint effort between IT and business. As the rule set constitutes a decision point or service, the interface signature can be accessed in a variety of ways to send input and receive output data. However, I believe it is absolutely critical for business to provide the input data and expected results. IT may suggest the scenarios (decisions, sub-decisions) that need to be tested and even provide input on the data variations needed to exercise various paths through the decisions. This last point is where I’ve seen the major disconnect happen on projects. Business often has an attitude of “it’s your code, you test it”. While IT should do the actual testing here, they should not be the ones determining what the valid business scenarios might be for a particular decision(s). Working as a team is the only way to ensure a quality decision implementation.
There are a number of very basic guidelines to keep in mind when doing decision/scenario testing.
- Scenario testing verifies the behavior of a decision as a component, rather than an individual rule.
- Scenario testing executes scenarios with a full set of input data (as opposed to what might only be minimally necessary to execute a single rule).
- Scenario testing must use realistic data similar to what will actually be processed when rendering a decision. This may often be historical data from similar systems that are currently in place.
- Decision testing data should not be prepared by the IT development team. It is ideally prepared by the business owners. This is to ensure that the data is truly representative and that the expected results are what the business expects (and not just what the development team has inferred is correct).
- Actual testing results should be compared against the expected results. The expected results should be persisted as part of a regression test suite that can be used later to validate correct functioning after any changes that might be made.
I fully expect this list to continue evolving as we become more experienced as a decision management community.
