Ayende @ Rahien

It's a girl

Limit your abstractions: Analyzing a DDD application

Abstractions have a cost. You should limit them. That seems like an obvious statement, but in a recent discussion I had, I realized that I didn’t articulate things in quite the proper way before.

Let me see if I can explain better now. One of the problems in typical applications is that we don’t really think before we introduce abstractions. For the purpose of this discussion, an abstraction in an interface. Let us take a look at a sample DDD application.

Nitpicker corner: No, I am not saying avoid using interfaces.

I took all the interesting interfaces out of the application, you can see them here:

image

Take a look at those interfaces. They bother me. They bother me because each of them represent an abstraction that is specific for a particular problem. In other words, it represent an non-abstracted abstraction, if that make any sense.

As I said, this is taken a DDD sample application. It isn’t a big one (and no, I didn’t review the actual code to see if it is a good one), but the interfaces that it has reveal a common problem, namely, interface explosion, or over abstraction. I removed any infrastructure / persistence stuff that was in the app, so you are looking just at the business interfaces, mind.

The problem is that the way this application is structured, it is highly procedural and had to maintain.

Huh?! I can hear you say, procedural? This is a DDD app, just look at the names, we have services and facades and events and… those are all good things. This design is pretty much text book. How can you say that this design is hard to maintain?

I’ll answer this question (and propose answers) in this series of posts. In the meantime, feel free to look at the code (it is Java, in its origin, and I simply modified it to C# for easier working) and make your own conclusions.

Comments

Demis Bellot
02/02/2012 08:01 AM by
Demis Bellot

"Abstractions have a cost. You should limit them."

Truer words haven't been spoken. We need to rid .NET of its abstraction fetish.

Shozan Ando
02/02/2012 08:02 AM by
Shozan Ando

As you already know, one reason might be that Java and C# are quite inflexible when it comes to these things, and to be able to test and mock and whatever with some kind of ease, you need to make an explicit interface out of just about everything.

Derek
02/02/2012 08:36 AM by
Derek

There's a .net version of his. http://code.google.com/p/ndddsample/

Artur
02/02/2012 08:37 AM by
Artur

There is a C# version written by my friend, having the same nice name :) http://code.google.com/p/ndddsample/

cbp
02/02/2012 08:50 AM by
cbp

You keep talking about getting rid of abstractions but as far as I know your only answer for how to get rid of abstractions whilst still writing comprehensive unit tests, is to use an in-memory database.

Now that's fine, but the ability to run a decent in-memory database is a recent luxury. The task of setting up a fully-functioning in-memory database has been fraught with difficulties, and those difficulties increase exponentially with the amount of legacy code you have to work with. The fact is, the boom in TDD preceded the ability to run a good in-memory database.

So yes, it would be great if we could get rid of some of these layers of abstraction as time goes forward, but there is good reason that a lot of it is there, and that reason is not because people don't know what they are doing - it's because the support tools were simply not up to scratch until recently (and by recently I mean the last year or two).

Jim
02/02/2012 09:17 AM by
Jim

Yeh I would say a lot of the time those seemingly pointless interfaces are just there to facilitate testing and are a conscious decision to that effect. Any comment on that?

Phillip
02/02/2012 09:23 AM by
Phillip

@Artur - seems there's no way to download it other than SVN... :(

Tim Murphy
02/02/2012 09:49 AM by
Tim Murphy

You might also want to look at http://dddsamplenet.codeplex.com/. It has a RavenDB implementation.

Dalibor Carapic
02/02/2012 10:02 AM by
Dalibor Carapic

I am waiting for Ayende to pull the rabbit out of the hat for this one. To be honest I do not see anything conceptually wrong with the sample, I might have written similar one and probably be proud of myself.

But still, this is why I follow this blog :)

Ayende Rahien
02/02/2012 10:17 AM by
Ayende Rahien

Shozan, There are other ways of testing things, assuming that you don't structure your code like that. I'll cover that in a future post

Ayende Rahien
02/02/2012 10:18 AM by
Ayende Rahien

Derek and Arthur, I am not covering the app itself, to be truthful, I am covering its architecture, and that is much less tied to the actual code base. That is why I am focusing on the abstractions.

Rafal
02/02/2012 10:24 AM by
Rafal

All the 'DDD' projects listed here suffer from code bloat. Most of the .cs files are just empty shells, dumb wrappers for something else, interfaces or various forms of boilerplate so it's really hard to get to application's guts. Or, there are no guts at all? Pure abstraction, Ayende's right. This is not about DDD imho, this is about a long tradition of creating overcomplicated and overabstracted web frameworks in Java and .Net.

Ayende Rahien
02/02/2012 10:45 AM by
Ayende Rahien

cbp, a) It is perfectly fine to run tests against a real (non in mem) database. b) in memory databases are not a new thing. SQLite had them since 2003, for example. And there have been other before that HSQL, for example.

Giedrius
02/02/2012 10:50 AM by
Giedrius

I agree with Shozan, similar interfaces are usually introduced for simpler mocking and if you have an idea how to avoid this without introducing additional abstraction, I'm waiting to see it.

Jonty
02/02/2012 11:00 AM by
Jonty

Ayende, you could argue that these specific interfaces are the wrong abstractions, but the post seems to suggest you are against mocking? Isn't it necessary to create an interface for a class in order to mock it?

Jason Meckley
02/02/2012 11:45 AM by
Jason Meckley

@jonty method only needs to be virtual to mock. However classes without empty ctors are more cumbersome to mock. Therefore most devs create an interface simply for easy mocks. I also think its an evolution of testing. Since most .net devs are "behind the curve" they are adopting practices from a few years ago rather than looking at what's happening today.

cocowalla
02/02/2012 11:54 AM by
cocowalla

@Jason

...and what's happening today?

Jason Meckley
02/02/2012 12:10 PM by
Jason Meckley

a need not to mock every class just to write a test. The idea that interfaces are not required to test segments of code. bascially, just because you have Foo doesn't mean you need IFoo.

John
02/02/2012 12:20 PM by
John

Yeah this kills systems. Worse still when you have tests specific to each abstraction testing no actually system behaviour just your implementation of it. Thousands of tests which people dont really know what they do and are terrified to touch with all kinds of mocking adventures. Keep it simple and test behaviour not implementation I am all for.

Curtis Schlak
02/02/2012 01:49 PM by
Curtis Schlak

@John It seems to me that you've described "integration" testing by your comment "system behavior." I do not disagree that systems testing should occur.

The interface explosion stems from "unit testing in isolation," testing a methods output and/or side effects or use of other objects. I hope that you don't advocate against that ideal....

Tom Dietrich
02/02/2012 02:31 PM by
Tom Dietrich

@Ayende

"a) It is perfectly fine to run tests against a real (non in mem) database"

Sure... as long as you want to set up and maintain a development database with a known set of data and make sure that no one ever changes any of the data in it without changing the associated tests. And then write scripts to return it to the known state after each and every test that mutates it, or just run everything in a single transaction than roll it back at the end, I suppose.

I agree with the desire to remove unnecessary abstraction, but your example doesn't really show unnecessary abstraction. If you want to unit test your business logic, you don't want it to touch a database. A single layer of interfaces to permit easy mocking of the query or command is necessary to facilitate this.

lontivero
02/02/2012 02:33 PM by
lontivero

Ayende, I have seen how you test some of your projects and I love it but... how can a programmer do TDD without those abstractions?. I think many of us need to know about abstractions and unit testing, please write about that.

Mix
02/02/2012 02:37 PM by
Mix

No one is arguing about your point of the code being procedural, contrary to my expectation. Based on my experience, you're dead right - it's totally procedural code (unless there's some interesting surprise inside).

Ayende Rahien
02/02/2012 02:40 PM by
Ayende Rahien

lontivero, Wait for it, this is a series of posts, and I actually answer this question later on.

Matthew Shapiro
02/02/2012 03:28 PM by
Matthew Shapiro

@Tom: Yes you do want to test your business logic against a real database. It is EXTREMELY easy to have tests that pass against an in memory collection but fail on a real database system, and without testing against a real database you won't catch these until Q/A or it gets pushed to production, unless you repeate the tasks against a real database in which case the in memory tests are wasted.

You don't need a complicated setup, all you need is a SqlExpress or SqlCompact database setup locally on each dev machine and have your unit test clear the data out before each test. This is how all my MSTests are setup right now.

Karep
02/02/2012 04:01 PM by
Karep

@Matthew: Maybe integration testing using real database is not that hard. But still it is like 10 times harder than unit tests. Only setup for one test may be X times bigger then the test itself. But I do integration testing. Only I don't agree that it is easy. It is not.

davh
02/02/2012 04:29 PM by
davh

I can remember a hard week of removing interfaces in our code. Think I killed 800 interfaces (that was put into the code before I came), that was not really being used for anything except as an extra file.

David Fauber
02/02/2012 04:30 PM by
David Fauber

"Abstractions have a cost. You should limit them." "In other words, it represent an non-abstracted abstraction, if that make any sense."

Yes, it makes a lot of sense, and is the number one area where I disagree with the design philosophy of colleagues who I otherwise see eye-to-eye on...for some reason people just love one-off abstractions (which is pretty much an oxymoron imo) and if backed into a corner trying to defend them will cycle through hypothetical future use and testability as their justify-anything excuses.

shah
02/02/2012 04:38 PM by
shah

Yeah, I see this problem in majority of projects these days…such concrete abstractions leading to procedural code and a whole host of problems that come with it.

Even though there is usually tons of interfaces and explosion of classes in the project, there is generally no real internal state within the object that the methods operate on, no cohesion amongst methods, little to no encapsulation. Most methods in the classes just take in parameters operate on it (perhaps using services that were passed in constructor and held internally) and then optionally returns something. They more resemble C language code with procedures just encapsulated in container class than true OOP.

So instead of nicely encapsulated code such as:

var authUser = new AuthUser(…); autoUser.RegisterDownloadActivity(…); var pc = autoUser.GetPercentBandwithUsed();

I end up seeing code like:

var authUser = authService.GetUserById(…); authService.RegisterActivity(authUser, ActivityType.Download, …); var bandwidthUsed = authService.GetBandwidthUsed(authUser); var totalBandwidth = authUser.TotalBandwidth; var pc = bandwidthUsed / totalBandwidth;

I use interfaces/abstractions at boundaries between layers and only when it makes sense. For instance, I have abstractions for things like repository, cryptography, emailer, which tend to have generic, API like functions anyways and you would want to abstract them for various reasons, but nothing else.

TDD can sometime lead to more procedural code initially but I refactor it later to more OO code without affecting the tests or near 100% code coverage. Then again, I don’ t insist on each unit test testing every single thing in total isolation by mocking everything else, nor do I feel it necessary.

Matt
02/02/2012 04:55 PM by
Matt

I tend to add interfaces as I find I need them, and that usually happens during testing. So most of my interfaces exist purely for unit testing. I'm another person who would be interested in how to get around this.

Arthur
02/02/2012 04:55 PM by
Arthur

It feels like most of posts in your blog encourage to get rid of abstractions but none of them cover a really complex business logic application. I agree that having a crud application would identify right away that any abstraction is redundant and I agree that every application can be made crud. But in case the architecture of the application is such that huge amount of business logic must be processed, it obviously must be processed somewhere, and messing with abstractions in this case is dangerous - one must introduce abstractions and must properly design them.

Jeff Sternal
02/02/2012 05:13 PM by
Jeff Sternal

"it represent an non-abstracted abstraction, if that make any sense."

That makes perfect sense because these aren't abstractions at all - interfaces are not, on their own, abstractions.

An abstraction makes working with a problem easier by eliminating details that aren't germane to that problem.

For example, "Stream" is an abstraction that reduces any number of complex implementations into the only thing you care about (when you want to use a stream): sequential processing that brings one item into focus at a time.

BookingService.BookNewCargo is an abstraction. Going from that to IBookingService.BookNewCargo is not.

Abstractions rock, but pointless interfaces do not.

Tom Dietrich
02/02/2012 05:58 PM by
Tom Dietrich

@Matthew

If you are testing your business logic against a database then you are not testing your business logic, you are testing your business logic AND your database. That is an integration test, not a unit test. The business logic tests shouldn't fail to pass if the database schema or content changes, or even if the implementation of the query changes (without a change to the schema of the output or the parameters). Business logic tests should fail only when there is a problem with the business logic.

My approach is to have unit tests for the Business logic against mocked query classes that return testable and pre-known data. If the business logic needs to run a command, it runs it against a mocked command class that can report to the test framework that it was invoked as expected and passed the parameters as expected.

I then have a second set of tests in which queries are invoked against a database of a known state and the result of these queries can be inspected for deviation from spec.

So yes, tests are run against a real database. Just not business logic tests.

Jeff Sternal
02/02/2012 06:07 PM by
Jeff Sternal

@Matt, if you want to get around the problem of creating interfaces solely for the purposes of unit testing, you should aim to separate your business logic from data access (and other external dependencies).

For advice about how to do this on the small scale, see http://simpleprogrammer.com/2011/01/26/back-to-basics-mock-eliminating-patterns/. For a broader architectural approach, see http://jeffreypalermo.com/blog/the-onion-architecture-part-1/.

Lars erik roald
02/02/2012 06:08 PM by
Lars erik roald

We don't need interfaces with only one implementation. Just make all methods/properties virtual in order to unit test.

Hpcd
02/02/2012 06:20 PM by
Hpcd

Abstractions are not just about abstracting away the database. Sometimes you have to interact with the file system, configuration files, other external systems, etc, etc.

Sometimes, it is necessary to have abstractions. I don't want to speak for Ayende, but I don't think he's saying to eliminate all abstractions, only use them when needed- not automatically create IFoo because you have a Foo class.

If you app need to interact with say, some exteral CRM system, then create an abstraction for that system.

And I don't believe everyone should just change to using in-memory databases in their unit tests. I believe unit tests should be unit tests, and be complementary to integration tests. We have unit testing and we have automated integration testing against VMs that are snapshot for specific states.

Jimmy Zimms
02/02/2012 06:39 PM by
Jimmy Zimms

Tom,

you write-> "Sure... as long as you want to set up and maintain a development database with a known set of data and make sure that no one ever changes any of the data in it without changing the associated tests." easy enough with a VS DB project to execute the DDL/DML automatically on build.

"And then write scripts to return it to the known state after each and every test that mutates it, or just run everything in a single transaction than roll it back at the end, I suppose." Transactions are easy to use (and there's plenty of NUnit, xUnit extensions to automatically use them and mbUnit has a Rollback attribute) so your argument is moot.

I am not making any comment on using or not using a DB in your tests, I am simply making the point that known good datastores are something EASY to handle in your tests.

Karep
02/02/2012 07:35 PM by
Karep

@Lars: Sometimes I make one method in a class virtual just to be able to override it in unit test. I don't know why but to many this looks ugly. I ask why? In java all methods are virtual even though no one ever overrides them.

Karhgath
02/02/2012 07:36 PM by
Karhgath

I don't know where Ayende wants to go, but I agree with the general idea.

Also, people are talking about repository, in-memory database, etc. Talking to a database is Integration Test, not Unit Test, If you have a small internal/local/embedded database then it is a Unit Test on the DB itself. It's integration as soon as you have a connection string to maintain. You can automate it but it comes with different concerns and usually requires a deployed version.

Databases returns you data, or persists it, thats it. Stop testing the DAL and test the actual business. Stop having: "function Business { client = dal.LoadClient(); var businessvalue = client.y + client.x / client.z; doSomething(businessvalue);}" Just save the damn businessvalue in the DB (if it's complex) or calculate it in a query and get on with life with simply doSomething(dal.GetBusinessValue()); from your app and test doSomething(fakeData) in you Unit Tests.

Also if you build a service on top of the DDD aggregate... why? Will you call the business remotely? Just build a real service(soa/rest), or use messaging/rpc, etc and do Integration Tests. Having a logical IService inside the code is... pretty useless in itself. So you abstract something to help you test, but you also have to test the abstraction, and probably need to abstract the abstraction to help you test....

Karep
02/02/2012 07:38 PM by
Karep

@Jimmy: Then how do you test transactions in your code if there is one transaction open because of test setup? How those nested transactions work? And if you use EF ObjectContext and there is Transaction opened (I assume TransactionScope), then you never commit to the database until you do TransactionScope.Complete which you never do because transaction is rolled back at Testcleanup.

Karep
02/02/2012 07:44 PM by
Karep

@Karhgath: I'm not sure what you mean by calculating in a query. I guess you mean SUM(), MIN(), AVG() because nothing complicated can be calculated in a query. Unless you create 3 screens long stored procedures which is worst I can imagine.

Liang Wu
02/02/2012 07:59 PM by
Liang Wu

Ayende, I got your point. Actually I feel same way. Code structured like this is "procedural" code, not OO. Have to be honest, I wrote this kind of code most time, trying to fit the good practices like "dependency injection" or "testability". But more and more, I think it is very bad code. It is not "over abstraction", instead, there is NO abstraction from business logic perspective.

Steve Sheldon
02/02/2012 08:05 PM by
Steve Sheldon

What @Lars said... A class has an interface, the question is whether it can be easily overridden.

Hpcd
02/02/2012 08:19 PM by
Hpcd

Ayende, it seems that all the interfaces are intended to be used as services- according to http://dddsample.sourceforge.net/architecture.html.

In this case, it is probably unavoidable. What I've noticed coming out from SOA revolution is that devs tend to want to make everything a service.

Karhgath
02/02/2012 08:40 PM by
Karhgath

@karep: yes, if you require more than simple add, sum, averages, etc. you need business rules (ie sales taxes) and thus should probably save the result for later. Whats even worse, and what I see more and more is when people are calculating business on the fly directly through the ORM when they query it, this is especially true in EF with linq where it can results in 35 queries to recalculate taxes for a list of embedded orders. Things should be simple.

var order = orm.load(x); order.applytaxes(); // calculates and sets fields. orm.save(order);

then your UT is: var order = new Order { .... }; order.applytaxes(); assert(128.43 == order.taxedtotal);

No need for any abstractions there.

Wayne M
02/02/2012 09:53 PM by
Wayne M

I would rather see concrete examples/tutorials of how to do things right than to see more and more examples of what is "wrong", especially when there's so much information out there espousing the "wrong" things.

Nick
02/02/2012 10:17 PM by
Nick

I tried to run my unit tests with an in-memory raven db server and it is way too slow to be doing every couple of minutes.....but perfect for my data access / integration tests.

For immediate feedback I mock data access and just test business rules (please don't kill me Ayende, I promise I'll buy your RavenDb series on Tekpub).

I don't see a way around this that gives me immediate feedbak from ultra-fast tests. I'd be keen to hear opinions on this though - some example code of what you do that is so much better would be very welcome @ anyone

Cheers

Juan
02/03/2012 03:12 AM by
Juan

So... Are we saying skip the "I" in SOLID?

Kyle Szklenski
02/03/2012 03:57 AM by
Kyle Szklenski

@Wayne, there's not always a "right" way to do stuff. Or, put differently, there's often thousands of "right" ways of doing things. I know this is a non-answer, but it's the truth. There are many, many more ways of doing things incorrectly, obviously. Anyway, the point is that it's pretty clear that Ayende intends to show a right way (in his opinion) to do it eventually, but prefers to lay out what he feel is wrong first, so that everyone is on the same page when he starts, and I think that makes some good sense.

@Juan - quite the opposite. Ayende is saying that we should have abstractions only where appropriate, I think, and not having superfluous ones that make it so that your client is having to implement a hundred extra interfaces just to use your stupid engine/code/whatever. They should be able to plug into your system with minimal effort, and the Interface Segregation Principle says: Clients should not have to implement a ton of crap that they don't even care about in order to interface with your code, i.e. a few appropriate abstractions are better than one, massive non-abstraction.

@Ayende, stop me if I'm wrong on any of this. :P

Good series. I can't wait to read more of it.

ThomasW
02/03/2012 06:31 AM by
ThomasW

Isn't it odd that we increase the complexity of software (by adding pointless interfaces) to make it easier to test?

Matjaz Muhic
02/03/2012 07:08 AM by
Matjaz Muhic

I think that his application looks/is over abstracted because it is just a sample application. You would not complicate real application like this if it was that simple I guess...

Karep
02/03/2012 04:15 PM by
Karep

@Juan. In order to have I you need interfaces. So, no.

Scott
02/03/2012 05:58 PM by
Scott

@Juan, exactly what I came here to say.

Granted I haven't looked at this code base to see exactly how these interface "abstractions" are used, but I almost always consider my interfaces strictly contracts.

For example, when I need a piece of functionality in a particular class, I define an interface that exposes a method to complete that functionality. Then, code against the interface and let the application determine which implementation is injected. In my opinion, this is not needless abstraction and is foundation for dynamic, supportable code.

If the code has "too many interfaces" it can be refactored and in good teams, will be in future iterations. Code is never completely finished until it is deemed legacy and no longer relevant.

@Ayende I think you should do a series on a code base and exam how dependencies and architecture change through out the life cycle of the application.

Juan
02/03/2012 06:06 PM by
Juan

I guess, like others, would like to understand "when" the abstraction should be put in place.

Ashok Guduru
02/07/2012 09:50 AM by
Ashok Guduru

I don't like to introduce the abstractions for the sake of only testing or testability. I use the abstractions for extendability and some times SoC.

Comments have been closed on this topic.