http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Hey, I totally agree that this would be sweet if it works, and I also don't want to spend my life in linear condition evaluation mode if it can be helped. I'm only not quite as sure that the automatic merging of several rules is so easily manageable. If the user has to give hints, visualization tools are helpful but it still has to be done. This requires quite some abstraction and discipline. If you ever build something like that for an actual system, tell us how it works out! http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment17http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment17Fri, 05 Sep 2008 14:42:33 GMTChecked the stats, almost 8% of the visitors are using Chrome. I don't have a UI in mind for that. That is not required. But at any rate, if we had UI, it would be, at most, something like a syntax highlighted text box and a view of the relevant sections of related scenarios http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment16http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment16Fri, 05 Sep 2008 14:13:01 GMT(Totally Off Topic - I am using Google Chrome to write this comment and it is totally sweet...) i guess i am getting hung up over the fact that i (as a simple developer) don't know how to ask the right questions (if i did then why would i need a DSL, right?) perhaps that is what i was missing... or maybe i still am :) i was looking at this as something that the business user will use or create (in the spirit of crossing the great divide) in other words... i was thinking that a business user would sit down and create a file called "preferred_customer.boo" am i correct in this thinking or are you coming from the perspective that the business user will be using a UI to create these scenarios? if we are talking about a file named "preferred_customer.boo" that is generated or created from a UI then i get it... 100% a UI could totally handle the presentation and management of scenarios in such a way where the business user gets the "50,000 foot view" an entire view of the scenario and events is not important to the system only to the business user if we are talking about a business user generating this file then i think that my earlier post and Stefan's points hold more value since again the entire view is more important to the business user http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment15http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment15Fri, 05 Sep 2008 13:53:25 GMTThe option I dismissed was to keep thinking in conditional on the events. I don't _want_ that. Being able to see exactly what will happen in the case of an event, including everything that goes on across all variations is essential. That is understood and agreed. BUT, that isn't how I want to build and think about the system. Also note that I am not actually saying anything about how I am going to implement it, only that the way that I think about it should be scenario based, not conditional on event based. The problem of conditional on event is that it fractures the logic, make maintenance harder and force you to think in unnatural ways. If the only thing that is an issue with the scenario based approach is that you want to see everything that goes on for an event, that is very easily done. How to test? You test each scenario in isolation. Then you go to the business and ask them about more complex scenarios, what happen when a check bounce for a preferred member for the 10th time? Something else that I think you should consider is that the output of this isn't the actual execution. It is the instruction to execute. And we can build a set of rules that manages that. For example calling the cops and extending credits are invalid together. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment14http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment14Fri, 05 Sep 2008 11:12:06 GMTAyende, how you acquire the knowledge you need to build those rules is one matter. But what do you do with this information? You go for the executable specification. The way it flows out of interviewed users mouths is how it's written down, and then you feed it to a machine that can execute it. This does have many advantages, granted. The option that you already dismissed is to take the information you get from your users (grouped by condition) and manually transform it into another structure (by event, with all conditions manually assembled for each event). The question is only, how much thought has to go into this assembling? Can it just be merged automatically? Does the user have to give hints (priority, ignore)? What is more confusing? Keeping the the specs and the implementation separate (with all those well-understood problems), or keeping it together and having to do the merging in your head (or visualized by a tool) each time you need to understand how a specific case would be processed? Which is easier to test? How can I know which FIT tests to write without seeing how many different code paths a single event might follow? Which combinations of input data will give me full coverage? I'm not saying you're wrong. It might work. But I think your polymorphism analogy breaks here. In C#, a refactoring that replaces if loops (ha!) with polymorphism is a good thing and almost always brings more advantages than disadvantages. What I'm saying is that from this you cannot conclude that the same is true for what you're proposing. It's not automatically better by inductive reasoning. It's nothing that we can know in advance, which makes it quite interesting. Experience will tell. What you've written in a) I completely did not get. Sure there are side effects in real live. That's why I think that just executing all possible rules, in any order, will not work. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment13http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment13Fri, 05 Sep 2008 10:52:32 GMTBunter, Amazing, I did something about that two days ago, and wasn't sure if I wanted to post that or nto. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment12http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment12Fri, 05 Sep 2008 10:20:58 GMTA bit off topic but then again, not if I think of it: I don't know if you address this issue in your book, but when I look at those rules, there's one thing what I find difficult - in large rule based systems you will eventually need the meta level i.e. a possibility to have some sort of abstract form to describe the rules so you can analyse and dissect the rules not just process them. The reasons for this are different, from conflict detection to usage analysis and visualisation. While DSL can be most expressive, the expressiveness of the language is in inverse proportion to its "dissectability". Introducing new concepts using objects makes this analysis a lot easier (in addition that people tend to wrap their heads around objects more easily) . So eventually, you will reinvent just a good OO design in your DSL... Hmm, was there a point in my comment, figure out I must. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment11http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment11Fri, 05 Sep 2008 10:15:48 GMTStefan, a) in general, you don't want to see the conflicts. This is a mater of system behavior. The rules are not side effect free, by design (calling the cops is a side effect :-) ). However, they can be set not to immediately execute, so you get an idea about system behavior before it is actually executed. And can make certain checks there. b) providing good visualization about what is going to happen at each action for all the rules isn't hard, and can be done as part of editing the scripts in the UI. c) All conditions for one rule are in a single place is a problem. Not because it is not good software engineering, but because you are now asking the business user to think about things in a way that is most unnatural. Let me try to explain, go ask a business user something like: "What is a preferred customer?" vs. "What should we do for large orders?" In the first case, they usually have a very clear and concise answer, backed by the business strategic vision, contracts and marketing information. For the second, they have to dredge up all the possible conditions. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment10http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment10Fri, 05 Sep 2008 10:08:55 GMTSure, but when a rule is pseudo-polymorphically distributed among several files, and you essentially have to search _all_ files for this rule or use a tool, it's hard to see the conflicts, or to know how to assign priorities or ignore-directives. (ignore does not solve conflicts of more than two rules that can occur in any combination, btw.) On the other hand, if all conditions for one rule are in a single block, it's easy to tell what happens, and to modify it. I agree that this is not a good software engineering practice, but this is not exactly the same game. Executing all rules might work in _some_ cases, but even then it presents the problem of ordering (unless your rules are side-effect free) The only way to tell if this works is to try it, though. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment9http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment9Fri, 05 Sep 2008 09:56:25 GMTStefan, x-ref tool is easy to do, at any rate. And having several scripts responding to the same event is actually simpler to implement than selecting the appropriate one. If you really want that, you can do something like: when customer.TotalPurchases > 10000 upon bounced_check: ignore_other_rules #do something else Which would be very easy to resolve. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment8http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment8Fri, 05 Sep 2008 09:42:55 GMTOh, I just noted that you already went through this with meisinger. You seem to think that this is a rare case. My guts say that this won't be generally true, but it's hard to argue with that if there is no data to look at. I'd say go for it, try it in a real project. When a set of rules is large enough to break down with linear conditions, and still conflicting rules do not present that much of a problem, you win. While it'd be hard to generalize such an experience, it would still be quite interesting. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment7http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment7Fri, 05 Sep 2008 08:46:36 GMTHaving polymorphism instead of conditionals seems like a nice goal. But polymorphism is only simple when using single inheritance. In your example, you're replacing a linear inheritance hierarchy with arbitrary conditions. What if several rules apply to the same transaction, and what if they "override" the same rules? From a software developer's perspective, I'd go out of my way to keep the modularity and think hard about ways to explicitly deal with conflicts. (In fact, we implemented that with our mixin library, where several mixins can overload the same method, and this gets resolved by just chaining base-calls up some non-arbitrary order. Works fine, but it does require some abstract thinking.) I'm not sure if your DSL is expressive enough to let me do this. Also, given enough complexity, you're going to need some kind of x-ref tool that tells you what gets actually executed, and in which order. If you want business analysts to deal with this DSL, I'm quite sure they are more comfortable with conditions in rules. At least that makes the execution of any rule clear to the reader. It makes conflicts obvious, and they can be avoided easily. Not beautiful, true, but how far do you want to take both a single-purpuse language and its users? http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment6http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment6Fri, 05 Sep 2008 07:56:19 GMTMeisinger, Yes, I would have a separate file for that. I would try to build it in such a way that ordering doesn't matter. Because quite often, while this an edge case that would bother me as a developer it is just not a real scenario Can you be both preferred customer who is often asking for credit extensions, maybe... but then I would have a special treatment for this scenario. Another option would be to setup priority for everything. So preferred customer is at priority 10, and if it run, then nothing else does, but if it dosn't, lower priority stuff get to run. I really don't like the idea of conditions, I don't see how it is any different than the problematic scenario that we had before, which we want to escape. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment5http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment5Fri, 05 Sep 2008 00:45:00 GMTWhat you're talking about in terms of "scenario" or "business scenario", I talk about as "context" in Context/Specification styled testing. The goal and the result is similar: stop talking about implementation of the software, and start talking about the experience - either of the business or the software. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment4http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment4Thu, 04 Sep 2008 20:32:15 GMTinteresting... so would you then also have another file for those customers who have asked for too many credit extensions as well? "over_extended_customer.boo" when customer.CreditExtensions > 20 # risk upon bounced_check or refused_credit : call_the account_rep the problem that i see is that now you have to either rank the scenario or setup some sort of condition catching... classify #preferred when customer.TotalPurchases > 10000 classify #risk when customer.CreditExtensions > 20 upon bounced_check or refused_credit : #preferred ask_for_more_credit unless #risk #risk call_the cops unless #preferred otherwise call_the account_rep and i personally don't like that route either. as you can tell you will end up with a conditional nightmare. perhaps something like: classify #preferred when customer.TotalPurchases > 10000 classify #risk when #preferred when customer.CreditExtensions > 20 upon bounced_check or refused_credit : #preferred ask_for_more_credit #risk call_the account_rep otherwise call_the cops i hope i making sense... http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment3http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment3Thu, 04 Sep 2008 20:30:42 GMTSo now we have inheritance in DSL. That's nice, I like the approach but there should be a rule preview and perhaps something like the import or reject statement so we can easily choose inherit from a default set of templates or not. Also a rules review panel so we know exactly whether we missed something or accidentally override the other. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment2http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment2Thu, 04 Sep 2008 20:02:34 GMTWell this approach also has a very good visualization. You can display a table with contexts as rows and events as columns and specify corresponding actions in the cells. So row "Is Preferred", column "New Order" will contain "Apply Discount 5%". This graphical representation will also give you an instant hint if you miss some important rules. http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment1http://ayende.com/3583/answer-dont-stop-with-the-first-dsl-abstraction#comment1Thu, 04 Sep 2008 17:38:17 GMT