This is pretty much exactly my thoughts also.. the community is full of preaching this aparent purity, but if you take an 'hour' to implement a real world system against those rules- you'll end up with monolithic migraine.. I'm really happy someone with what I believe to have sound judgement in this area said this.

I do a very similar thing, I have a wrapper class takes the IQueryable in the ctor and implements a couple of interfaces that support paging and sorting only, I return this wrapper when the query is passed to the dal.

When it is time to materialize the deferred query I now have a mechanism to intercept the execution which knows all about the UI usage so I can cache a compiled version of the query that takes the paging and sorting of the UI use cases into account.

You've explicitly mentioned sorting and paging which are the most common variations that the UI would want to layer on top of the original query. You've got me wondering though if I'm missing a trick and that you think it is useful to be allow the UI to further restrict the actual filter condition of the query as you have done by returning IQueryable directly?

Paul

If you have a controller which needs a list of late paying customers

1: At runtime how do you obtain the LatePayingCustomerQuery instance?

2: During testing how do you mock the list of Customer instances so that you don't need to go to the DB?

Paul, sure it is useful. Perhaps he only wants to see late paying customers with a balance of more than $1000 to make better use of his time. Or only at stuff that is 90 days past due. There's just no way you can guess every possible permutation of what the customer might want to look at.

Peter, unfortunately you can't mock session.Linq <customer() since it is an extension method. I hide nhibernate behind a generic repository. To mock i just do:

var dataAccess = MockRepository.GenerateStub <idataaccessor();

daraAccess.Stub(stub => stub.Query <customer()).Return(new Customer[]{...}.AsQueryable());

Ayende, how do you prefer to deal with remote database scenarios, specifically in a multi tenant app? Do you a) use .net remoting for ISession? or b) give the customer the connection string and be done with it?

Hi Jauo,

thanks for the reply. I use the specification pattern to define queries and I allow query composition logic such as: IUIResults results = Dal <invoices.Satisfy(new OverdueByDays(30) & new InvoiceValue(1000));

My stance was if you need a different subset of results then create a new specification/Query which may leverage existing queries using composition.

The examples you've quoted reasonably straight forward and therefore I agree not too much of a biggie if they're not implemented by a query. However if you wish to further restrict the data by some more complex crieria such as if the customer is a repeated bad payer then I would want to make that logic re-usable.

As query composition is possible using similar code to the example above I was just wondering if not forcing the devs to put this logic into queries (by returning IQueryable directly) was likely to lead to business logic to bleed into the UI

Paul

Here is an example of how my repository would look.

public interface IBlogEntryRepository

{

BlogEntry GetByID(ISession session, long id);

IEnumerable <blogentry GetAll(ISession session);

IEnumerable <blogentry GetPublished(ISession session);

}

This way I can do stuff like

repository.GetPublished(session).Skip(page * 10).Take(10);

or

repository.GetAll(session).Where(x => x.Deleted);

So I still use a Repository because that makes it very easy for me to mock in my consumer classes. I really don't think there is anything new here at all to be quite honest. Eric Evans calls this the Strategy Pattern and mentions it explicitly in use with the Repository pattern, it's just that these days it is implemented via LINQ and IEnumerable <t rather than some kind of IQuery parameter.

@Paul, the Specification pattern was mentioned in a previous post. It seems to be popular. I guess I'm just curious how much reuse do you get out of those things and where do they get parsed? It seems logical to encapsulate boilerplate so that you can repeat those types of activities throughout the entire module without writing additional code... but are you parsing them in your DL? Is there a third-party Specification parser that knows what your DAL needs in order to perform the final queries properly? Is there a component that you use that helps you with the code generation or is that hidden in the Specification definition class?

RE: nitpicking...

I think that he (the "UI Team") will not be happy about being referred to in that way and will violently reject that notion...

:)

@CaliCoder,

Simon Segal has a few good posts on the subject, see

www.simonsegal.net/.../entity-framework-reposit...

Basically specs just define predicates.

eg

ILatePayingCustomerSpecification : ISpecification <customer
{

bool IsSatisfiedBy(Customer cust)

}.

My implementation uses lamdas and works against Linq IQueryable sources. eg

Dal <customer.Satisfy(ISpecification spec) would return the logical equivalent of customers.Where(c => spec.IsSatisfiedBy(c))

All the heavy lifting is performed by the Linq provider

I agree 100% that the UI needs to be able to talk to both the business objects, and the DAL. However, there are a couple of tradeoffs to your specific approach that I see. One is that your DAL could become polluted with a lot of tiny classes if you use the one-DAL-class-per-query-type approach rather than the one-DAL-class-per-return-type approach. Also, since the DAL objects take an ISession, you've exposed to the outside world that your DAL uses NHibernate under the covers. These aren't necessarily bad decisions, but it is something to keep in mind when designing the DAL.

How would you handle multiple DBs under this scenario? Inject a SessionFactory instead of a Session?

Ayende, thanks for the reply. I agree further filtering is actually a new query.

As for the wrapper thing, I couldn't see another way to intercept the enumerator or ToList call so I can inspect the final usage and create a compiled version of the spec + order_by + paging + fetching_strategy (this is Linq2Sql or EF).

As you can only compile the final query I would have had to put many overloads to expose the final UI queries in my dal, which is a bad thing, which is kinda the point of your post :-)

Ayende:

What about the simple queries, like GetByID. Do you even create a class for those? Or do you duplicate the LINQ/Criteria code where you need it?

Im not sure new FindCustomerByIdQuery.GetQuery(_database){ID = 14}.FirstOrDefault(); reads better that (from c in _database where c.ID = 14 select c).FirstOrDefault();

Usually I make an Interface called IIdentifiable, with an ID propterty that all entities implement. So I guess I can use a generic FindByID.GetQuery <customer{ID = 14}. The generic part could even be inferred.

Whats your take on the simple quries?

Ayende. I can understand why using an in-memory DB is useful, I do it myself when using ECO from CapableObjects which comes with its own MemoryPersistence handler for this very reason.

BUT :-)

If your controller executes a query which fetches data from the DB then your DB needs to have data in it. To get the data into the DB it has to be valid in so far as any NOT NULL constraints must be met at least so setting up the context for the test is more work. Or does the in-memory SQLite database that gets automatically created not have any NOT NULL constraints applied?

As I understand this approach you are using, it is possible only for non remoting architecture, for example for web asp.net applications or desktop UI applications using database locally?

What I mean is, for UI client that has NHibernate session available. The examples that you cannot by my opinion (or am I wrong) use this approach are: Silverlight client, clients that call remoting method, or web service method on another domain.

In this cases you still have to write those facade methods that are problematic as GetInvoicesForUser(UserRequest request). Is this correct Ayende?

uh oh, I now have a bad feeling that I'm going to make a fool of myself on your blog but here goes anyway.

I did some tests and I found that the compiled queries with Linq2Sql were performing with nearly a 10 perf gain on the queries I was issuing. It appears that quite a lot of the cost of a linq2sql query is in the translation from expression tree to sql, as compiled queries skip that step by basically just caching the parameteized query in a func<> ready for you to supply the params, that seemed to be the way to go.

If I've understood correctly then you can only create a compiled query when you have the full expression tree available, which makes sense to me as any paging, sorting or change in fetching strategy after the fact effectively changes the sql that you would generate.

I also have the concept of filters, this is just another specification that can scope queries. I use using blocks to demarkate their use

e.g.

using(CustomerFilters.UK)

{

IResults results = CustomerRepsoitory.Satisfy(new LatePayingCustomers(60); //60days

results.OrderBy(c => c.Name);

results.Page(2, 10); //2nd page of 10

IList <customer customers = results.ToList();

}

In my initial comment, I said my result wrapper took an IQueryable, actually it doesn't it retains a copy of the spec used and the active filter if any. My wrapper (probably the wrong word) simply hangs onto the specifications and exposes interfaces and methods for ordering and paging only, much in the same way that IOrderedQueryable provides a variation of the IQueryable interface. These methods basically internally just store that the methods were called and nothing actually happens until the results are wanted for enumeration/ToList.

When the ToList is called I build a key to uniquely identify the combination of spec+filter+order_by+paging+fetching_strategy and see if I have a Func <dc,>

already cached.

If I don't have a cached version then I do the necessary expression rewriting to build a lamda that I can pass to the CompiledQuery.Compile method.

The specifications already expose a Expr <func<t,>

predicate property but that can't be used directly as the argument to a Linq IQueryable Where method in a compiled query, even though it works fine in the non-compiled version.

So going back to your question I suppose ultimately I just want to allow a query to be satisifed, while still allowing the UI code to page/sort/apply UC specific fetching, with the benefit of transparent caching of compiled queries for performance.

I don't have anyone to chat to about this kind of stuff really so you're my first sanity check :-)

I much prefer to introduce a Reporting layer that Greg Young is preaching. It would be a sibling of the Business layer but separate.

That way I can abstract away the underlying data access more cleanly. I can use completely different storage options to optimize various reporting or searching tasks. I can use denormalized data or materialized views for my reporting needs.

I don't like having custom query classes that create coupling between the data access and presentation layers.

Reporting is a separate abstract concept from the business layer and can be cleanly separated without leaking data access.

Ayende,

Something like the client has a desktop thin client that connects to my multi tenant application hosted on my server.

Ayende,

Say you want to cache the results of the LatePayingCustomerQuery.

Where do you implement caching with this approach?

Jake

Joao,

I would treat this as two separate apps.

In other words, this is a web app that has no UI, just output raw data.

The UI leaves on another app, but I treat it like you treat the browsers

"Creating the data is about as easy as creating it in memory for mocking."

I don't agree. With the repository...

User u = new User();

repository.Expect(x => x.GetBy......(.....)).IgnoreParameters().Return(u);

With the saving approach

1: Create the user

2: Set every property that maps to a NOT NULL column

3: Update the DB

otherwise you will get an exception telling you that a NOT NULL constraint has been violated. Now yes you can have a single method for creating a valid User but this method will have to be updated every time you change the User class and add a new property that cannot be NULL.

Apart from this inconvenience you are not testing the user you are testing something else which requires a user so spending time ensuring the user is valid is time wasted.

Now I don't disagree with you completely. In the ORM I use I often create objects and save them to an in-memory DB for testing purposes, but I am able to disable such constraints so I don't have to create a valid user to test something else.

Does the in-memory SQLite lack those constraints, or are you having to create a valid user? If you are then I think the approach is wrong because it will break when you change something unrelated (the User class).

Peter,

The problem that you describe doesn't exists to me.

I have UserBuilder or similar that I use, it takes care of things.

If I really cared, sure, I could disabled not null checks, but I never cared enough to do so.

Ayende,

Sorry if this seems like a stupid question, but where does the query get created or to ask it another way who requests the query be created?

And secondly how are the queries extended higher up?

you stated:

'In that way, my queries are closed for modification and open for extension, since they can be further manipulated by higher level code.'

I'm not clear on how you are achieving this?

@Stevie - He's achieving this by making small (arguable anemic), single purpose query classes that wrap a single type of query and it's parameters and returns the query result. The concept of being "open for extension, closed for modification" is that your class design is so small and single purpose, that once built it should rarely change (ie: closed for modification). But, that class can be extended by being inherited from (is-a relationship) or wrapped in other classes (has-a relationship) to give you flexibility to extend functionality. This is widely considered a good design for business (domain) objects, and for good reason. Unit testing and API stability are just two of the many benefits.

However, this isn't always an approach that extends well to the DAL because 1.) The types of queries needed aren't often very well defined as design time, 2.) The other layers may need to have way too much knowledge of the inner working of the DAL to achieve performance and proper transaction handling, 3.) This approach requires new classes for each new piece for functionality which may become unwieldy on moderate to large projects, and 4.) Business logic/rules have a tendency to slip into the DAL layer this way, which violates a proper separation of concerns and the DRY (don't repeat yourself) principle. These aren't necessarily reasons not to take this approach because there really aren't any DAL designs that don't have warts of their own.

Ayende's point seems to be that 1.) the UI should be able to talk to the DAL too, not just the business objects, and 2.) The OFMCFE design concept should be extended to the DAL.

PS: I think your diagram is incorrect. The Repository is most certainly supposed to be above the domain and not below it.

I think that generally the introduction of LINQ has mostly made the repository redundant, but I still find it useful especially for testing.

Hi Ayende, you made a point, I'm back to the "create a query class per usecase" rather than "add a method to that repository" camp.

Also, in my new attempt at it, I'm separating query parameters to the actual query/criteria builder implementation, it look like this:

public interface ICriteriaBuilder<TQueryParameters>

{

DetachedCriteria BuildCriteria(TQueryParameters parameters);

}

benefits are:

• parameters stay persistance ignorant and could be serialized over the wire if needed

• separation of concerns

• it make my criteria builder stateless and injectable to my service layer

Finally, I can't wait when I can switch from DetachedCriteria to IQueryable like you did in your sample

Peter,

I am not talking about repository here, I am talking about DAL and the common 3 tier arch.

I remember a thread I started on nh forum when I thought about this myself some time ago. What I was wondering/suggested was to split the NH assembly into smaller ones. Your approach would feel better if there was eg a "query assembly" (eg criteria interfaces) that could be used in higher layers, seperated from "nh core".

Call me old fashioned, but - for example - offering session.Connection on high layers feels... wrong.

Just an idea, along with the AST parser, couldn't the "query languages" using this parser be seperated to their own assemblies?

Don't you still have to update the GetQuery method to reflect the newly extended properties?

I might have got the wrong end of the stick but here go's:

If you had a new property that you need to add to the query you could extend the current implementation by adding that as a property to the class.

but you would still need to update (not extend) the getQuery method to allow for the new property. So GetQuery is not closed for modification.

an example:

public class LatePayingCustomerQuery

{

public DateTime? DueDate {get;set;}

public decimal? MinAmount {get;set;}

public deciaml MaxAmount {get;set;}

                public DateTime? OverDueTime {get;set;}


public IQueryable

<customer GetQuery(ISession session)

{

    var payments =  from payment in s.Linq

<payment()

                where payment.IsLate

                select payment;


    if(DueDate != null)

        payments = payments.Where( payment => payment.DueDate >= DueDate.Value );


    if(MinAmount != null)

        payments = payments.Where( payment => payment.Amount >= MinAmount.Value );


    if(MaxAmount != null)

        payments = payments.Where( payment => payment.Amount <= MaxAmount.Value );

if(OverDueTime != null)

        payments = payments.Where( payment => payment.DueDate == OverDueTime .Value );



    return  from customer in s.Linq

<customer
where payments.Any( payment => customer == payment.Customer )

            select customer;

}

}

So adding the OverDueTime param violates solid?

Could get query iterate over its properties building th query dynamically?

Whoops I meant violates Open closed!!

Aha I see so you stay within the Open/Closed principle by creating a new method for the query that would include the new param? So you are extending.

I've never seen a DAL that the model uses, only a model that the DAL uses. What a strange approach.

So to be clear you do not mind having explicit reference to NHibernate in your UI assemblies and having explicit calls to NHibernate objects in your UI logic?

Do you have an (even trivial) end-to-end sample for client application, (like the one on the screenshot) of that approach?

Ayende,

I still don't get it. You said:

If the requirement have changed, then yes, I'll change this class.

What the difference with changing this,

GetInvoicesForUser(userId, pageNum, pageSize, orderByCol, orderByDesc)

by adding column perhaps..

Change class - add property, change method - add column.

Thanks.

@Ayende,

That is approach that also like. Just a quick question. By saying "...how I approach data access using NHibernate", do you mean Linq2NH is production-ready? Which version do you use? That from NHContrib trunk?

I follow Sebastian ayende.com/.../...should-go-all-the-way-to-ui.aspx

Create a Query Layer sibbling to you domain layer.

The query layer provides concerns about querying the domain using all necessary tooling (ordering paging).

Data can come from replicated databases, precompiled database tables, cube...

I can definately see how this would eliminate a bunch of mindless repository query methods. But when it comes to persistance, it's hard to beat the flexibilty of a CustomerRepository.Save(customer).

What if I need to perform some additional workflow steps after I persist it the object (but only if it's successful)? Or synch with another system? Or what if I need to populate some additional customer information, like looking up and setting some delinquency schedule.

Ayende,

We're not talking about complicated logic here..just additional data tasks that are directly related to persistance. Surely I don't need the overhead of a real service layer (and possibly another interface) just to invoke a sproc or to call method in another repository when my object is saved, do it?

Are we getting into semantics here? Assuming that the session or the datacontext is my repository/data layer..if I call CustomerService.Save() instead of CustomerRepository.Save() even though both methods do exactly the same thing does that really change anything?

"It is also, incidentally, wrong."

That's a bit of a stretch, isn't it? Maybe a constructive provocation, though.

I doubt we want the data access any closer to the UI, but rather the domain queryability. That is of course possible with in-memory Linq, but that's hardly useful in all scenarios, so I guess that it's actually "efficient domain queryability" that could be the goal.

Just in case someone was offended by the "DAL to UI" bit... :)

Sorry if this is a bit of a necro but I found this post very useful in explaining some hard questions I have been having with how best to work with nHibernate so thanks for that.

Now in the diagram you have above how do you define the Business Layer vs. the DAL? Is the business layer the logic to return specific query results and the DAL just nHibernate?

Now are you suggesting, if I am understanding, that the Presentation Layer should be able to have direct access to the DAL? I am assuming that the query object you have defined is something the Presentation layer can access and instantiate, correct?

Finally, the query object you define, is it a part of the DAL or the Business Layer?

Sorry if some of these questions seem a bit naive.

One more perhaps stupid question. What is supplying the session to the query object? Is it coming from the DAL? Is it instantiated by the presentation layer? Where is the session maintained?

So you allow the presentation layer to directly reference nHibernate, manage the session, call session.Save() directly etc.?

The business layer then contains the domain/model entities, interfaces and also the query objects?

My initial design had the data layer being the only layer that referenced nHibernate with a similar design to this: www.codeproject.com/.../...rnateBestPractices.aspx

I have started by just making a modification to the AbstractNHibernateDao <t class based on here here>.

Interesting, sorry I feel like I am taking forever to fully understand this, but I do really want to make sure so I have one more clarification to ask of you.

Where does the ISession object come from here, the business layer? Which layer manages the session object and which passes it to the query object? In other words where/when is the session created and what layer calls GetQuery(ISession session).

Thanks, I think this is starting to make some more sense to me.

I am have to try to put all these ideas together in code and maybe build a sample application with it see if I am understanding the concept correctly. If I get something decent together I will post it up on CodeProject.

Thanks,

Chris

Thanks for this post - really enjoyed reading it! I too agree with the flexible querying up to the UI. However, that poses a question for me. Currently I'm using Linq2Sql as my DAL. Then in my Domain, I have some simple queries (returning IQueryable) which the UI will call and can also "extend" (paging, sort, etc.). Those queries will call the DAL (Linq2Sql), return a result set, and map those values to a separate Domain model, which types are exposed to the UI and get sent back up.

Because I'm doing a mapping in the Domain, I'm essentially executing the query there, before it gets to the UI. Hence, if the UI were to "extend" any more on that query (sort, page, etc), that portion of the query wouldn't be executed in Sql but rather in the assembly itself. Therefore, I wouldn't get the performance benefits of sorting/paging in Sql Server. Is that typically ok though and a good design to follow?