Ayende @ Rahien

It's a girl

Disconnected Mode Change Tracking

There is some interesting discussion going on right now about dealing with change tracking in disconnected mode. The whole discussion started when Andres complained that this making writing diconnected updates much harder than it should. You can check the links to see the whole discussion, what I am interested at is Udi's second reply, in which he suggest doing something like this:

void IBus.Send(params IMessage[] messages);

With the client code looking like this:

myBus.Send(addOrderLineMsg, delOrderLineMsg);

I am not sure that I agree with the choice of names (specifically, IMessage to me in an infrastructure level idea, IOrderMessage is something that would have a meaningful business sense), but I really like the idea. Then I started to think about it. Sending the server a bunch of changes, so it could execute them in a single transaction, thus reducing the chattiness of the conversation, I am pretty sure that I heard it before. Indeed, it is my favoriate pattern, Unit Of Work, expanded to include a multi layered, service based architecture. 

I really like this approach.

ASP.Net WTF?! (sorry, blog was down)

All of a sudden, I get an email telling me that my blog is down. I go and check it out, and it is indeed down. With the following error message.

Server Error in '/Blog' Application.


Could not load file or assembly 'App_Web_g5ujsn49, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.IO.FileNotFoundException: Could not load file or assembly 'App_Web_g5ujsn49, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.

I tried restarting the appication by renaming web.config, but it didn't work. The only thing that worked was changing the compilation debug mode from false to true, and then to false again.

Search for the error brought this post, and it looks like there my be a hotfix avialable.  The problem is that my scenario doesn't match the conditions on the hotfix, so I really don't know what to think. I am going to ping my host and check if there is anything that can cause file locks on the server (anti virus, indexing service, etc), although I would have expect this kind of error to occur much earlier if that was the case.

Wanted: MethodMissing

I am listening to this webcast, talking about dynamic dispatch on the JVM, and 19:30 it starts to get really interesting. The JVM is getting support for MethodMissing. As far as I understand, this is most relevant for compiler writers, but it is something that I really would like to have on the CLR (and exposed for C# & VB).

The reason for wanting this is very simple, it lets me handle advance scenarios very easily. Take a look at Boo's duck typing capabilities to see some of the stuff that can be done with it.

Skill vs. Knowledge

From here, actually talking about the value of certifications:

A skill is not as simple to acquire as knowledge: the learner has to perform the skill badly, recover from mistakes, do it a bit better, and keep repeating the whole process.

I like this definition.

The missing Linq: Stateful extentions methods

It seems to me that right now there is a lot of exploring going on with extention methods. People are doing some really cool stuff with them.

What I am afraid of is that this exploration is going to hit glass wall the moment people are going to try to do the interesting stuff with them. The reason for this is that beyond the simplest cases, I would really want to have some sort of a state between method calls.

Here is a code sample that demonstrate the issue:

object myRandomObj = new object();
myRandomObject.Tag("Linq ExtentionMethods");
myRandomObject.GetTags();

Tag() and GetTags() are extention methods. The problem is that as it stands now, there isn't really a way to implement this functionality using Linq. You need to keep the tags associated with the object, which usually means that you will use a hash table. This seems like a solution, but it runs into the issue of breaking the GC, since I would like to have the state freed by the GC when the object goes out of scope.

Hashtable based on WeakReference are one solution to this, but this is quite a bit more complex than it sounds, and you needs to scavengar that table every now and then, which means that you can either suffer wasted memory(bad) or have a separate thread that will clean the table (bad).

I can see two solutions to this problem. Either adding another field to System.Object "IDictionary ExtendedProperties" or supplying a weak hashtable implementation that ties into the GC in the same manner that WeakReference is there. The first solution is the simplest, but it is tying object into IDictionary, as well as increasing the size of System.Object significantly. The second solution is more complex, but it is much prefered, since it doesn't affect anything else. 

The problem is that both of those requires support at the framework level, it is not something that can be done externally.

MonoRail With SubSonic

In the Castle forums, it was asked if it is possible to use SubSonic as the DAL for a MonoRail application. The answer is absolutely yes.

You will usually see MonoRail samples with either Active Record or NHibernate, simply beacuse this is what most of the users of MonoRail are using, but MonoRail has no data access preference. (There are extention to MonoRail that make it work better with Active Record and NHibernate, but those are just that, extentions, the core is completely data layer agnostic).

So, how would you go about using SubSonic with MonoRail?

Here is the controller:

public class ProductsController
{
 public void List()
 {
  ProductCollection products = new ProductCollection();
  products.Load(); 
  PropertyBag["products] = products;
 }
}

And here is the view:

<ul>

<% for product in products: %>

<li> ${product.Name} </li>

<% end %>

</ul>

Query Objects vs. Methods On The Repository

Udi Dahan is talking about Query Objects vs. Methods On The Repository and raises some interesting ideas.

I am somewhere in the middle, since I am doing almost all queries directly on the repository, but I am passing it a query object. The main idea is that I don't really deal with the query object, it just happens behind the scenes:

return Repository<Customer>.FindAll( Where.Customer.City == "London" );

One important case for query objects is when you have complex searching, as I said, Querying is a Business Concern. The traditional case is for a search screen, where you may have several dozens criteria (and I don't kid about the several dozens) that you need to search for. But there are many cases where in order to perform a business operation, you need to perform a non trivial query. Those are often dynamic queries, changing according to business logic and the state of the moon.

In those cases, I generally tend to separate those out into explicit query objects, using [UseCaseName]Finder, which can then be use:

CustomersInDebtFinder finder= new CustomersInDebtFinder( CurrentEmployee )
     .DebtIsLessThan(CurrentEmployee.MaxDebtAllowedToHandle)
     .DebtIsOverDueAt( AppContext.CurrentApplicationDate );
if(CurrentEmployee.IsManager && shouldShowSubordinateItems )
    finder.FindCustomersWithDebtOverSubordinatesMaxDebt( CurrentEmployee );
return finder.Find();

The mechanics of the query is fairly complex, and a query object allows me to express them in a way that is much more intent revealing.

Tags:

Published at

Hibernating Rhinos - Episode #1: Rhino Mocks 101

To celebrate the Rhino Mocks 3.0 release, I produced a screen cast of about an hour, talking about how you can use it. This is the very first time that I am doing something like this, so the list of fubars in this episode is fairly long. For instance, I didn't realize that the Play/Pause bar would appear in the final movie, or thought about closing the IM during recording, etc.

At any rate, I think that this could provide a gentle introuction to Rhino Mocks, and help understand how to use it. The first few minutes are introductions and power point, but from then on, it is all code, so have fun.

You can watch it here.

Oh, and happy mocking.

Hibernating Rhinos: Introduction

Hibernating Rhinos is a series of screen casts that I am going to make, basically they include me, a microphone, code and Camtasia.The sound quality is going to be aweful, I presume, and it turn out that I speak English sloowly. It may very well be the case that I can type English faster than I can speak it.

I guess that I don't need to explain the choice of the name, but I am still looking for a logo / picture that I can use.

My editing skills are even more limited than my UI skills, so you are going to see some funny stuff, probably, but I hope to learn to do better. Feel free to send suggestions, comments, etc to me. No fixed time, not fixed subject.

The first topic is Rhino Mocks, basically an introduction to what it can do. The second topic is going to be about MonoRail. Beyond that, feel free to suggest topics, but I will say out front that I'll not be able to do them all.

When pressed for answers...

Q: How will you do this?
A: Web Service

Q: How will you do this?
A: BizTalk

Q: When will it be done?
A: Later

Q: When will it be done?
A: Count the links in the screenshot, I do a link a week.

The first two are usually good answer for a lot of questions, up to and including walking the dog and making coffee. The last two are useful for UI changes.

Rhino Mocks 3.0 Released!

I gave it a few weeks outside, and quite a few people downloaded and tried it, so I am feeling good about it. I had a surprised that kept me from release the 3.0 for a while, but I'll talk about it later.

The binaries and code are in their usual place, here.

(Image from clipboard).png

BREAKING CHANGES:

  • Moved all the constraints to Rhino.Mocks.Constraints namespace. This was done to avoid conflicts with NUnit and MbUnit's new assert syntax.

New features:

  • Generic Methods (at last!)
  • Support arrays of value types as out parameters
  • Support non-CLS Compliant value types such as UInt64
  • Better error messages when using generic delegate methods on generic types
  • Generally faster
  • Using Dynamic Proxy 2 - better performance, better code-base, full support for weird generic scenarios.
  • Text.* constraints can now be used to evaluate non strings as well, by calling the ToString() method. Useful for things such as the Criteria API in NHibernate.

Changes:

  • The CallOriginalMethod() was deprecated, you are now encouraged to use CallOriginalMethod(OriginalMethodOptions.NoExpectation) or CallOriginalMethod(OriginalMethodOptions.CreateExpectation) instead. The reasoning behind the change is that currently CallOriginalMethod looks like it create an expectation, but it doesn't really does it,  it is better to be explicit about the whole thing.
  • The build script will internalize almost everything no related to Rhino Mocks (Except AbstractInvocation) - prevent conflicts from users that uses both Dynamic Proxy and Rhino Mocks - i.e, me).

Bug fixes (from previous 3.0 Beta):

  • Fixed regression with closed generic types inheriting from open generic types that contains generic properties of the generic type, if the property is overriden
  • Fixed issues with the generic method return values.
  • Regression with closed generic types inheriting from open generic types that contains generic properties of the generic type, if the property is overriden
  • Fixed a problem with the new semantics of CallOriginalMethod
  • Fixed a problem mocking internal types that are not nested
  • Fixed a misleading exception when you pass a mock object of type that override ToString() to an unexpected method.
  • Fixing another issue with AbstractExpectation not checking the real return type of generic methods.
  • Fixed issue with generic method call on generic type for interfaces
  • Fixing an issue with BackToRecord no clearing originalMethodsToCall expectations.
  • Improved support for wacky generics scenarios.
  • Better support for types having non inheritable attributes that cannot be easily replicated.
  • Better support for types implementing interfaces by using a base class that is located in another assembly.

NHibernate Migration Guide: From 1.0 to 1.2

Billy McCafferty may have intended just to give a sample of how to move from NHibernate 1.0 + my NHibernate.Generics to NHibernate 1.2, but his post is actually a very good guide about migrating in general from 1.0 to 1.2.

The part that you really want to do is in the end, where you want to verify that:

  • Do updates to a child, via the parent, still work?
  • Do creations of new children, added to the parent, persist to the database?
  • Do deletions of existing children from the parent, or by deleting the child directly, work correctly?
  • Have other CRUD, cascade scenarios been tested?

I have said it before, I trust NHibernate to manage my objects, but I want to verify that what I think the mapping are, and what they are actually doing are the same. Cascading and lazy loading are two of the most important parts that you want to verify when you migrate, and you really should make sure that you have working tests before you do it. For the rest, check out Billy's post.

Tags:

Published at

Linq for NHibernate Updates: Adding a new committer

Following several good patches, I have added Bobby Diaz to the Rhino Tools project, which currently includes the Linq for NHibernate implementation as well. I expect that the new stuff will be up in the repository shortly.

Just to remind you, I am still looking for more volunteers to help with the implementation, so if you are into futuristic technologies, just dive right into the code and the implementation details. And be sure to send me (or Bobby) a patch or two.

Data Migrations Woes

For some reason, customers insist on putting their data in a database, which is all fine and dandy. But then they also insist on taking it out! Which is hardly fine at all.

The fun part starts when the source is a little known database called Oracle, and the destination is a new arival from out of town, who goes by the nick name SQL Server.

Fun stuff that I run into today:

  • PK in Oracle are Number, in SQL Server, it is interpreted as double or float. Do try to query for Id = 12324.432154
  • Wierd PK - such as 1,001,0004,001,000 - a real PK that I run into today. No idea how it got to this point.

I am using both P/L SQL Developer and SQL Plus, and I have to say that both have some distinct advantages over SQL Server Management Studio. Just Ctrl+Click is an invaluable assest when you try to figure out what you are doing.

WatiN + MbUnit

Verifying invalid input...

[RowTest]

[Row(null)]

[Row("")]

[Row("-12314132")]

[Row("foobar")]

[Row("very_long_string_very_long_string_and_so_on_for_another_512_chars")]

public void TryingToEnterWithInvlidPolicyNumberShowsError(string id)

{

    Open("PolicyRequests/NewRequest.aspx?Id="+id);

    AssertNoError();

    AssertContainsText("Can't find Policy with id #"+id,"Should give error message for invalid input");

}

NHibernate: Nullable DateTime Issues

System.DateTime is a value type in .Net, which means that it can never be null. But what happens when you have a nullable date time in the database, and you load it into a DateTime type?

Consider this simple example. Mapping:

<property name="UpdatedDate" nullable="True"  column="updated_date" type="DateTime" />

Property:

public DateTime UpdatedDate 
{
      
get { return m_dateTime; }
     
set { m_DateTime = value; }
}

When NHibernate loads a null value from the database, it cannot put a null in the UpdatedDate property, the CLR doesn't allow it. What happens is that the UpdatedDate property is set to the default DateTime value, in this case: 01/01/0001.

This can cause two major issues down the road. The first is that 01/01/0001 is not a valid date in SQL Server, so when you try to save the value, it will throw an exception. The second is that because of this issue, when NHibernate needs to track changes, it will check if null != 01/01/0001, and it will turn out that yes, this entity (which we never touched) has changed.

This can cause an extra update (and thus an exception) which can cause some fairly significant head scratching. The solution is simple, you need to let NHibernate know what to do with null values. This can be done by simply using a nullable type, such as:

public DateTime? UpdatedDate 
{
      
get { return m_dateTime; }
     
set { m_DateTime = value; }
}

Or by using the Nullables.dll library for 1.1, and specifying the correct type in the mapping:

<property name="UpdatedDate" column="updated_date" type="Nullables.NHibernate.NullableDateTimeType, Nullables.NHibernate" />

With this property:

public Nullables.NullableDateTime UpdatedDate 
{
   
get { return m_dateTime; }
   set { m_DateTime = value; }
}

Thanks for Sheraz Khan, for finding out and brining this to my attention, since then, I have run into the issue a couple of times, and I hope that if I blog about it, I will remember to match nullabilities.

Code smell

Why does it have to be like this?

if($F('<%= PolicyDescription.ClientID %>')==

    '<asp:Literal runat="server" Text="<%$ Resources:PolicyResources, DescriptionNotFound %>"/>')

{

    alert('<asp:Literal runat="server" Text="<%$ Resources:PolicyResources, EnterValidPolicyNumberOrDescription %>"/>');

    return;

}

 

Transactions and concurrency

I just had a head-scratching, hair-pulling, what-the-hell-is-going-on bug. Basically, a classic producer / consumer issue. Each work item is composed of several sub-items, which should always be processed as a single unit (exactly one time). The problem was that I suddenly started to get duplicate processing of the same work item, but with different sub-items each time.

I went over the code with a comb, and I couldn't see anything wrong, I investigated the database, and everything was fine there as well. I knew that I was in trouble when I considerred going down to the SQL Server protocol level and check if somehow network problems were causing this issue.

Here is a simplified version of the producer (and yes, I would never write this kind of code for production, test, or anything but a short and to the point demo). As you can see, it merely generate 500 records into a table.

private static void Producer()

{

       SqlConnection connection = new SqlConnection(connectionString);

       int count = 0;

       while (true)

       {

              connection.Open();

              SqlTransaction sqlTransaction = connection.BeginTransaction(isolationLevel);

              for (int i = 0; i < 500; i++)

              {

                     SqlCommand sqlCommand = connection.CreateCommand();

                     sqlCommand.Transaction = sqlTransaction;

                     sqlCommand.CommandText = "INSERT INTO t (Id) VALUES(@p1)";

                     sqlCommand.Parameters.AddWithValue("@p1", count);

                     sqlCommand.ExecuteNonQuery();

                     sqlCommand.Dispose();

              }

              sqlTransaction.Commit();

              Console.WriteLine("Wrote 500 records with count " + count);

              count += 1;

              connection.Close();

       }

}

And here is the consumer, which read from the table, and ensure that it reads in batches of 500:

private static void Consumer()

{

       SqlConnection connection = new SqlConnection(connectionString);

       while (true)

       {

              connection.Open();

              SqlTransaction sqlTransaction = connection.BeginTransaction(isolationLevel);

              SqlCommand sqlCommand = connection.CreateCommand();

              sqlCommand.Transaction = sqlTransaction;

              sqlCommand.CommandText = "SELECT COUNT(*) FROM t GROUP BY id";

              SqlDataReader sqlDataReader = sqlCommand.ExecuteReader();

              if (sqlDataReader.RecordsAffected != -1)

                     Console.WriteLine("Read: {0}", sqlDataReader.RecordsAffected);

              while (sqlDataReader.Read())

              {

                     int count = sqlDataReader.GetInt32(0);

                     Console.WriteLine("Count = {0}", count);

                     if (count != 500)

                           Environment.Exit(1);

              }

              sqlDataReader.Dispose();

              sqlCommand.Dispose();

              sqlTransaction.Commit();

              connection.Close();

       }

}

Note that I have painted the isolationLevel red in both. Here is the code that run both methods:

private static void Main()

{

       Delete();

       new Thread(Producer).Start();

       new Thread(Consumer).Start();

}

If I set isolationLevel to ReadCommited or RepeatableRead, this consistently fails within a couple of seconds, it manage to do partial read of the records that were inserted by the consumer. I I set the isolationLevel to Serializable or Snapshot, it behaves as expected.

I may be missing something, but I would expect ReadCommited to only allow the second transaction to read... well, commited rows. Just to point out, the real scenario doesn't involve aggregation, and I am seeing the same issue. I suppose that the new records were commited (and thus made visible) when the query already scanned that part of the table, and thus it missed some one the rows, while snapshot and serializable force either a static image all the way through or waiting till the end.

Any comments?

Dont Repeat Yourself: Pragmatic Approaches

Jeremy Miller has posted about the DRY principal, as well as the Wormhole anti pattern.  One of the pain points that was brought up was adding a field to the screen. We are currently extremely fluid with regard to the entity design, this means that new fields keep popping up and old fields are going away. I am also extremely fond of ReSharper, and I loath tables like "Cutsomers". The spelling mistake is intentional here, but I have seen systems that have this in productions. This make it fun to work with the system.

The process we use is to have a single authoritive source for our model. That model is the entities. Our entities are Active Record classes (but they do not inherit from ActiveRecordBase), so I get to specify what they do inside the code. Part of the build process would generate the database from our entities.

Assume that we are working with comments, and we have a new requirement, to show the commenter's IP. What we would need to do is:

  • Add the IP Property to Comment (then run the build):
    [Property(NotNull = true)]
    public virtual string Ip { get { ... } set { ... } }
  • When we create the comment, add the IP Address of the current user.
  • When we display the comment, show the IP Address as well.

In practice, we often add several fields at the same time, but it is the same principal. The only pain point in this process is the data, we have test data that needs to be modified when we re-generate the database, which can be a pain to deal with. I am searching for an answer in this matter, by the way.

Another thing that I had some experiance with is moving stuff from configuration to code. There should be only one truth, but the instict to put it in XML is the wrong approach, IMO. There are not tools to refactor XML, while code is extremely flexible under the right hands. I had success in pharsing code and generating additional code from it. In that case, it was to generate perfomance counters create/use code, from an abstract class that contain just their definitions.

At any rate, what I am trying to say is that you should aspire to a single truth as well, and you should carefully consider what kind of a truth it is. <Truth/> is not an easy apparoch.

How not to go about "Building user interfaces for object-oriented systems"

Update: This article is fron 1999, which I somehow missed, not news by far. Still wrong, even for its time, though.

This really annoys me, the author of the article tries to preach an OO methodology for writing UI. In general, I don't have an issue with that, but his arguments contains this:

An object-oriented solution tries to encapsulate those things [adding a field to a screen] that are likely to change in such a way that a change to one part of the program won't impact the rest of the program at all. For example, an object-oriented solution to the problems I just discussed requires a Name class, objects of which know how to both display and initialize themselves.

I am not sure where to begin. To start with, an entity should not have ties to the UI. Then we have this dream vision about editing a single class, and suddenly all the relveant screen sprout the new field, and we can start working on it. UI doesn't work like this. It doesn't work like this because adding a new field to screen can horribly break the screen. It might push the Ok/Cancel button outside of the visible realm of the screen, it might break the tab order, it will definately break the flow of the screen.

I am all for OO and encapsulation, but this is not a problem that you can generalize.

This just gets better when you keep on reading:

The other bugaboo that I want to put to death is the notion of different views into the same object, usually characterized by the question: "Suppose you need to display this data as a pie chart over here and a grid over there? How can you do this if the object displays itself?"

Now let's get real. How often in your work has this problem actually come up?

Um, all the time? The simplest version is showing items in a list and then in a details view, but I just finished talking about the different views that I present to the various aspects in my systems. Beyond that, I have different views for different screens. In one screen, I am showing an Employee's personal data, in another the Employee's salary data, etc.