+972 52-548-6969
Let us burn all those pesky Util & Common libraries
This is a post that is riling against things like Rhino Commons, MyCompany.Util, YourCompany.Shared.
The reason for that, and the reason that I am not longer making direct use of Rhino Commons in my new projects is quite simple.
In computer programming, cohesion is a measure of how strongly-related and focused the various responsibilities of a software module are. Cohesion is an ordinal type of measurementand is usually expressed as "high cohesion" or "low cohesion" when being discussed. Modules with high cohesion tend to be preferable because high cohesion is associated with several desirable traits of software including robustness, reliability, reusability, and understandability whereas low cohesion is associated with undesirable traits such as being difficult to maintain, difficult to test, difficult to reuse, and even difficult to understand.
I am going to rip into Rhino Commons for a while, let us look at how many things it can do:
- Create SQL CE databases dynamically
- Keep track of the performance of ASP.Net requests
- Log to a database in an async manner using bulk insert
- Log to a collection of strings
- Log to an embedded database – with strict size limits
- Same thing for SQLite
- Same thing for SqlCE
- Keep track of desirable properties of the http request for the log
- Add more configuration options to Windsor
- Provide a static gateway to the Windsor Container
- Plus some util methods
- Allow to execute Boo code as part of an msbuild script
- Allow the execute a set of SQL scripts as part of an msbuild script
- Provide cancelable thread pool
- Provide an implementation of a thread safe queue
- Provide an implementation of a countdown latch (threading primitive)
- Expose a SqlCommandSet that is internal in the BCL so we can use it
- Allow to easily record log messages executes in a given piece of code
- Allow to get the time a piece of code executed with high degree of accuracy
- Allow to bulk delete a lot of rows from the database efficiently
- Provide an easy way to read XML based on an XPath
- Provide a way to update XML based on an XPath
- Provide a configuration DSL for Windsor
- Provide local data store that works both in standard code and in ASP.Net scenarios
- Provide collection util methods
- Provide date time util methods
- Provide disposable actions semantics
- Provide generic event args class
- Allow 32 bit process to access 64 bit registry
- Give nice syntax for indexed properties
- Give a nice syntax for static reflection
- Provide guard methods for validating arguments
- Provide guard methods for validating arguments – No, that is not a mistake, there are actually two different and similar implementations of that there
- Provide a very simple data access layer based on IDbConnection.
- Provide a way to query NHibernate by for a many to one using its id with a nicer syntax
- Provide a named expression query for NHibernate (I am not sure what we are doing that for)
- Provide unit of work semantics for NHibernate
- Provide transaction semantics for auto transaction management using the previously mentioned unit of work
- Provide a way to map an interface to an entity and tie it to the Repository implementation
- Provide a fairly complex in memory test base class for testing Container and database code
- Provide a way to warn you when SELECT N+1 occur in your code via http module
- Provide nicer semantics for using MultiCriteria with NHibernate
- Provide Future queries support externally to NHibernate
- Provide an IsA expression for NHibernate
- Provide a way to execute an In statement using XML selection (efficient for _large_ number of queries)
- Provide a pretty comprehensive generic Repository implementation, including a static gateway
- Provide an easy way to correctly implement caching
- Provide a way to easily implement auto transaction management without proxies
- Do much the same for Active Record as well as NHibernate
Well, I think that you get the drift by now. Rhino Commons has been the garbage bin for anything that I came up with for a long time.
It is easy to get to that point, but just not paying attention. In fact, we are pretty much doctrined into doing just that, with “reuse, reuse, reuse” bang into our head so often.
The problem with that?
Well, most of this code is only applicable for just one problem, in one context, in one project. Bulk Deleter is a good example, I needed it for one project, 3 years ago, and never since. The repository & unit of work stuff has been used across many projects, but what the hell do they have to do with a configuration dsl? Or with static reflection?
As a matter of fact, that Rhino Commons has two (different) ways to do parameter validation is a pretty good indication of a problem. The mere fact that we tend to have things like Util, Shared or Common is an indication that we are basically throwing unrelated concerns together. It actually get worse if we have something in the common project that can be used for multiple projects. A good example of that would be the in memory database tests that Rhino Commons provide. I have used it in several projects, but you know what? It is freaking complex.
The post about rolling your own in memory test base class with NHibernate show you how simple it can be. The problem is that as timed went by, we wanted more & more functionality out of the rhino commons implementation, container integration, support for multiple databases, etc. And as we did, we piled more complexity on top of it. To the point where it is easier to roll your own than to use the ready made one. Too many requirements for one piece of code == complexity. And complexity is usually a bad thing.
The other side of the problem is that we are going to end up with a lot of much smaller projects, focused on doing just one thing. For example, a project for extending NHibernate’s querying capabilities, or a project to extend log4net.
Hm, low coupling & high cohesion, I heard that somewhere before…
Comments
Convert it to a bag of more or less useful code snippets for various occasions - just copy & paste.
Every project has its 'utils/tools/commons/shareds' and has to have one. Where else to put various helpers, wrappers, containers or adapters?
Coupling is exactly the reason why I could not use Rhino Security for my project.
My project (.NET 2.0) has already used StructureMap for IoC, but if I want to use Rhino Security, I have to use Windsor, plus Boo DSL, plus dependency on .NET 3.5, plus ActiveRecord. They're too much for just a security provider.
I did try extracting Rhino Security out of above mentioned dependencies, but the efforts was way too high compared to rolling out my own.
Just for clarification, my project does use NHibernate and I does use ActiveRecord in another project.
tvbusy,
FYI,
Rhino Security doesn't have a dependency on Active Record or Boo DSL
The Windsor dependency is pretty easy to abstract (although you would need to create a registry for it for StructureMap).
As for .Net 3.5, we aren't using any features from there, but we are using some syntax, again, that is pretty easy to resolve.
We tend to split utility functions in a project into a single location, later on we may refactor into more generalized areas.. we also follow a rule of re-use, given we think we've found a pattern that has existed in a few projects, we work to develop a more solid general 'product', if that product works we will 'brand' it under our company name..
The point is that we've pretty well isolated a pattern and created a product which is easy to name because its specific to a solution.. then it basically just gets its own namespace somewhere in our stack.
This way our reusable code base tends to stay clean and well versioned.
Our common framework classes are grouped into 'projects' around function, which is usually also what they have dependencies on, i.e. ORM, WCF, etc.
All our provider classes adhere to well-defined interfaces located in a different 'interface assembly' that have no dependencies.
We also provide an ILMerge of all our common implementation assemblies and interface assemblies into 2 distributed assemblies (interface and implementation) for convenience.
What about the .Net framework? It's a big collection of unrelated stuff, isn't it?
@configurator the .NET Framework isn't in one assembly, its arranged by funciton
Sounds to me like a packaging and organization problem. If Rhino.Commons is too big, now, you just need to break it up.
Rhino.Commons.Collections
Rhino.Commons.Configuration
Rhino.Commons.Threading.ThreadPool
etc.
Then you can still provide "Rhino.Commons" as the uber zip archive of all the DLLs.
Seems like a simple solution to me.
I have to agree with Bryan. The common library that I've written at work is packaged this way. There are five components: Common, Security, Logging, Reporting, and Web. It works quite well if I do say so myself.
I'll agree with you though that having one bin for all your stuff that you couldn't be bothered to find the proper place for isn't a good idea.
I think that rhino.commons also have a lot of infrastructure and is not compatible with this:
ayende.com/.../Infrastructure-Ignorance.aspx
Look at the nhrepository. I think the code is more reusable than the entire library.
"The sum of its parts is more than the whole."
I was ready to disagree based on the opening, but then I read the list of features in Rhino.Commons.
That's quite a diverse magic bag you've got there.
You are right. Cohesion does matter. I tend to build smaller magic bags which are organized by topic, and can be used separately.
It seems that Util dll's are often a fear of having many projects in your solution. I usually create separate projects for each unrelated piece of "common" functionality. This also helps at compile time, since a change in a base "common" lib only forces the dependent projects to recompile
Generally, i take the next step and move the interfaces of common pieces into separate interface dll's so that code changes that don't touch the contract don't require any recompile of the dependent assemblies. It's basically taking the IoC pattern to the assembly level.
The Graph Add-in for Reflector ( reflectoraddins.codeplex.com/.../View.aspx ) is a great tool for seeing how deep your dependencies go and help you flatten out those hierarchies.
@Rafal
Please, please do not do that...
What happens then when you need to fix MyFooSnippet which has been used in 72 different projects?
Just because someone (Ayende) has decided to use it as a catchall (it MIGHT be reusable one day so I'll put it in common) doesnt mean that common/util libraries are bad.
Oren,
Some of these things should be moved over to more appropriate places - like the Windsor DSL. Shouldn't that be a contribution to the Castle Project?
It is only a matter of componentization. In the Bryan's solution..
Rhino.Commons.Collections
Rhino.Commons.Configuration
Rhino.Commons.Threading.ThreadPool
etc.
the ONLY think that matter is oi make sure that each of these namespace are not statically dependent on each other, meaning you can take one of these, put it in its own project and recompile it immediately.
Now you might have dependencies between for instance,
Provide nicer semantics for using MultiCriteria with NHibernate
Provide Future queries support externally to NHibernate
because they must both rely in the same
Rhino.Commons.NHibernate
component.
Ok, folks. Where to put GuidCombGenerator? Almost every project I have seen recently contains this class...There is a theory that every software project evolves until it contains a crippled email client and a lisp interpreter - maybe we should add a guid generator to that list...
Rafal,
Most people steal it from NHibernate
@Rafal I typically put my Scheme interpreter from college into all my projects just to throw guys off =)
I don't think "common" libraries are bad. Sure you could break them up into assemblies with related functionality, but they you end up with lots of interdependent referencing. (To reference A you need to reference B & C & D because parameters from A or return types/exception definitions are defined in a different assembly now.) In any case I've heard it argued that fewer, larger assemblies utilizing meaningful namespace separation are better than many, smaller ones with only one or a few namespaces.
I've actually recently been pushing more functionality into "common" type assemblies. Though this is also because I'm a composition nut. My common library is one of my best tested blocks of code.
Seeing this post brought to mind this post by Jay Fields:
blog.jayfields.com/2009/03/kill-your-darlings.html, which in turn riffs on Michael Feathers: www.artima.com/weblogs/viewpost.jsp?thread=8826
Seems to me at least part of what we're talking about here is useful shared tools which keep getting extended as the userbase (and hence requirements) grow. Feature creep in the context of open source?
My response:
blog.jordanterrell.com/.../...-Libraries-Dead.aspx
Comment preview