Ayende @ Rahien

Refunds available at head office

Entity Framework 4.1 Update 1, Backward Compatibility and Microsoft

One of the things that you keep hearing about Microsoft products is how much time and effort is dedicated to ensuring Backward Compatibility. I have had a lot of arguments with Microsoft people about certain design decisions that they have made, and usually the argument ended up with “We have to do it this was to ensure Backward Compatibility”.

That sucked, but at least I could sleep soundly, knowing that if I had software running on version X of Microsoft, I could at least be pretty sure that it would work in X+1.

Until Entity Framework 4.1 Update 1 shipped, that is. Frans Bouma has done a great job in describing what the problem actually is, including going all the way and actually submitting a patch with the code required to fix this issue.

But basically, starting with Entity Framework 4.1 Update 1 (the issue also appears in Entity Framework 4.2 CTP, but I don’t care about this much now), you can’t use generic providers with Entity Framework. Just to explain, generic providers is pretty much the one way that you can integrate with Entity Framework if you want to write a profiler or a cacher or a… you get the point.

Looking at the code, it is pretty obvious that this is not intentional, but just someone deciding to implement a method without considering the full ramifications. When I found out about the bug, I tried figuring out a way to resolve it, but the only work around would require me to provide a different assembly for each provider that I wanted to support (and there are dozens that we support on EF 4.0 and EF 3.5 right now).

We have currently implemented a work around for SQL Server only, but if you want to use Entity Framework Profiler with Entity Framework 4.1 Update 1 and a database other than SQL Server, we would have to create a special build for your scenario, rather than have you use the generic provider method, which have worked so far.

Remember the beginning of this post? How I said that Backward Compatibility is something that Microsoft is taking so seriously?

Naturally I felt that this (a bug that impacts anyone who extends Entity Framework in such a drastic way) is an important issue to raise with Microsoft. So I contacted the team with my finding, and the response that I got was basically: Use the old version if you want this supported.

What I didn’t get was:

  • Addressing the issue of a breaking change of this magnitude that isn’t even on a major release, it is an update to a minor release.
  • Whatever they are even going to fix it, and when this is going to be out.
  • Whatever the next version (4.2 CTP also have this bug) is going to carry on this issue or not.

I find this unacceptable. The fact that there is a problem with a CTP is annoying, but not that important. The fact that a fully release package has this backward compatibility issue is horrible.

What makes it even worse is the fact that this is an update to a minor version, people excepts this to be a safe upgrade, not something that requires full testing. And anyone who is going to be running Update-Package in VS is going to hit this problem, and Update-Package is something that people do very often. And suddenly, Entity Framework Profiler can no longer work.

Considering the costs that the entire .NET community has to bear in order for Microsoft to preserve backward compatibility, I am deeply disappointed that when I actually need this backward compatibility.

This is from the same guys that refused (for five years!) to fix this bug:

new System.ComponentModel.Int32Converter().IsValid("yellow") == true

Because, and I quote:

We do not have the luxury of making that risky assumption that people will not be affected by the potential change. I know this can be frustrating for you as it is for us. Thanks for your understanding in this matter.

Let me put this to you in perspective, anyone who is using EF Prof is likely to (almost by chance) to get hit by that. When this happens, our only option is to tell them to switch back a version?

That makes us look very bad, regardless of what is the actual reason for that. That means that I am having to undermine my users' trust in my product. "He isn't supporting 4.1, and he might not support 4.2, so we probably can't buy his product, because we want to have the newest version".

This is very upsetting. Not so much about the actual bug, those happen, and I can easily imagine the guy writing the code making assumptions that turn out to be false. Heavens know that I have done the same many times before. I don’t even worry too much about this having escaped the famous Microsoft Testing Cycle.

What (to be frank) pisses me off is that the response that we got from Microsoft for this was that they aren’t going to fix this. That the only choice that I have it to tell people to downgrade if they want to use my product (with all the implications that has for my product).

Sad smile

Review: Microsoft N Layer App Sample, Part I

I was forwarded this link http://microsoftnlayerapp.codeplex.com/ with a request to review it. I thought about skipping it until I saw:

image

The main problem here?

If it is a simple business scenario, it doesn’t need DDD.

Now, just to be clear on this sample app. It is not someone working on their free time, this is something that is Official Guidance (shows up at: http://msdn.microsoft.com/es-es/architecture/en) and people are expected to read and follow this.

I haven’t reviewed the code yet, but I am scared just by looking at the architecture diagram.

LightSwitch on the wire

This is going to be my last LightSwitch post for a while.

I wanted to talk about something that I found which was at once both very surprising and Doh! at the same time.

Take a look here:

image_thumb[1]

What you don’t know is this was generated from a request similar to this one:

wget http://localhost:22940/Services/LSTest-Implementation-ApplicationDataDomainService.svc/binary/AnimalsSet_All?$orderby=it.Id&$take=45&$includeTotalCount=

What made me choke was that the size of the response for this was 2.3 MB.

Can you guess why?

The image took up most of the data, obviously. In fact, I just dropped an image from my camera, so it was a pretty big one.

And that lead to another problem. It is obviously a really bad idea to send that image on the wire all the time, but LightSwitch make is so easy, indeed, even after I noticed the size of the request, it took me a while to understand what exactly is causing the issue.

And there doesn’t seems to be any easy way to tell LightSwitch that we want to put the property here, but only load it in certain circumstances. For that matter, I would generally want to make the image accessible via HTTP, which means that I gain advantages such as parallel downloads, caching, etc.

But there doesn’t seems to be any (obvious) way to do something as simple as binding a property to an Image control’s Url property.

LightSwitch & Source Control

Something that I found many high level tools are really bad at is source control, so I thought that I would give LightSwitch a chance there.

I created a Git repository and shoved everything into it, then I decided that I would rename a property and see what is going on.

I changed the Animals.Species to Animals.AnimalType, which gives me:

image

This is precisely what I wanted to see.

Let us see what happen when I add a new table. And that created a new set in the ApplicationDefinition.lsml file.

Overall, this is much better than I feared.

I am still concerned about having everything in a single file (which is a receipt for having a lot of merge conflicts), but at least you can diff & work with it, assuming that you know how the file format works, and is seems like it is at least a semi reasonable one.

Nevertheless, as promised:

True story, I used to have a lot of ravens in my backyard, but they seem to have gone away single my dog killed one of them, about a week after RavenDB’s launch.

Analyzing LightSwitch data access behavior

I thought it would be a good idea to see what sort of data access behavior LightSwitch applications have. So I hook it up with the EntityFramework Profiler and took it for a spin.

It is interesting to note that it seems that every operation that is running is running in the context of a distributed transaction:

image

There is a time & place to use DTC, but in general, you should avoid them until you really need them. I assume that this is something that is actually being triggered by WCF behavior, not intentional.

Now, let us look at what a simple search looks like:

image

This search results in:

image

That sound? Yes, the one that you just heard. That is the sound of a DBA somewhere expiring. The presentation about LightSwitch touted how you can search every field. And you certainly can. You can also swim across the English channel, but I found that taking the train seems to be an easier way to go about doing this.

Doing this sort of searching is going to be:

  • Very expensive once you have any reasonable amount of data.
  • Prevent usage of indexes to optimize performance.

In other words, this is an extremely brute force approach for this, and it is going to be pretty bad from performance perspective.

Interestingly, it seems that LS is using optimistic concurrency by default.

image

I wonder why they use the slowest method possible for this, instead of using version numbers.

Now, let see how it handles references. I think that I run into something which is a problem, consider:

image

Which generates:

image

This make sense only if you can think of the underlying data model. It certainly seems backward to me.

I fixed that, and created four animals, each as the parent of the other:

image

Which is nice, except that here is the SQL required to generate this screen:

-- statement #1
SELECT [GroupBy1].[A1] AS [C1]
FROM   (SELECT COUNT(1) AS [A1]
        FROM   [dbo].[AnimalsSet] AS [Extent1]) AS [GroupBy1]

-- statement #2
SELECT   TOP ( 45 ) [Extent1].[Id]              AS [Id],
                    [Extent1].[Name]            AS [Name],
                    [Extent1].[DateOfBirth]     AS [DateOfBirth],
                    [Extent1].[Species]         AS [Species],
                    [Extent1].[Color]           AS [Color],
                    [Extent1].[Pic]             AS [Pic],
                    [Extent1].[Animals_Animals] AS [Animals_Animals]
FROM     (SELECT [Extent1].[Id]                      AS [Id],
                 [Extent1].[Name]                    AS [Name],
                 [Extent1].[DateOfBirth]             AS [DateOfBirth],
                 [Extent1].[Species]                 AS [Species],
                 [Extent1].[Color]                   AS [Color],
                 [Extent1].[Pic]                     AS [Pic],
                 [Extent1].[Animals_Animals]         AS [Animals_Animals],
                 row_number()
                   OVER(ORDER BY [Extent1].[Id] ASC) AS [row_number]
          FROM   [dbo].[AnimalsSet] AS [Extent1]) AS [Extent1]
WHERE    [Extent1].[row_number] > 0
ORDER BY [Extent1].[Id] ASC

-- statement #3
SELECT [Extent1].[Id]              AS [Id],
       [Extent1].[Name]            AS [Name],
       [Extent1].[DateOfBirth]     AS [DateOfBirth],
       [Extent1].[Species]         AS [Species],
       [Extent1].[Color]           AS [Color],
       [Extent1].[Pic]             AS [Pic],
       [Extent1].[Animals_Animals] AS [Animals_Animals]
FROM   [dbo].[AnimalsSet] AS [Extent1]
WHERE  1 = [Extent1].[Id]

-- statement #4
SELECT [Extent1].[Id]              AS [Id],
       [Extent1].[Name]            AS [Name],
       [Extent1].[DateOfBirth]     AS [DateOfBirth],
       [Extent1].[Species]         AS [Species],
       [Extent1].[Color]           AS [Color],
       [Extent1].[Pic]             AS [Pic],
       [Extent1].[Animals_Animals] AS [Animals_Animals]
FROM   [dbo].[AnimalsSet] AS [Extent1]
WHERE  2 = [Extent1].[Id]

-- statement #5
SELECT [Extent1].[Id]              AS [Id],
       [Extent1].[Name]            AS [Name],
       [Extent1].[DateOfBirth]     AS [DateOfBirth],
       [Extent1].[Species]         AS [Species],
       [Extent1].[Color]           AS [Color],
       [Extent1].[Pic]             AS [Pic],
       [Extent1].[Animals_Animals] AS [Animals_Animals]
FROM   [dbo].[AnimalsSet] AS [Extent1]
WHERE  3 = [Extent1].[Id]

I told you that there is a select n+1 builtin into the product, now didn’t I?

Now, to make things just that much worse, it isn’t actually a Select N+1 that you’ll easily recognize. because this doesn’t happen on a single request. Instead, we have a multi tier Select N+1.

image

What is actually happening is that in this case, we make the first request to get the data, then we make an additional web request per returned result to get the data about the parent.

And I think that you’ll have to admit that a Parent->>Children association isn’t something that is out of the ordinary. In typical system, where you may have many associations, this “feature” alone is going to slow the system to a crawl.

If I run Dev Div…

I found this post interesting, so I thought that I would try to answer it.

If I run Dev Div, I would:

  • Utilize CodePlex Foundation to a much higher degree. Right now CPF manages about 6 projects, all of them seems to originate from Microsoft. None of them are actually useful for CPF in the sense of creating an excitement around it. If I run Dev Div, and given the indication of the lack of will from within Microsoft to maintain the DLR, I would give the CPF the DLR, along with a small budget (1 million annually should more than cover that) and set it free. That 1 million? I would put it down as marketing, because the amount of good will that you’ll generate from doing that is enormous. And the level of publicity that you’ll get for CPF is huge.

 

  • Sit down with marketing and create a new set of personas, based on what is actually happening in the real world. Mort, Einstein and that Elvis dude might be good when you are building the product and designing the API, but there are other aspects to consider, and acceptance in the community is important. A lot of the public blunders that Microsoft has made recently has been just that, marketing mistakes more than anything else.

 

  • Build for each project a set of scenarios that appeal to each group. For the PHP crowd, point out how WebMatrix makes their life simple. For the ALT.Net crowd, show them how they can use IIS Express to run their unit tests without requiring anything installed. Right now what is going on is that pushing to one side alienate the other.
    Let me put this another way, when someone asks me why use NHibernate instead of Entity Framework, I am going to inform them that Microsoft continues their trend of new Data Access framework every two years, and they have gone retro with Microsoft.Data.

 

  • Kill duplicated effort before it goes out the door. Linq to SQL & Entity Framework are a great example of things that should have been developed concurrently (because either project had a chance of failure, so hedging your bets is good, if you can afford that), but they should have never been allowed to both see the light of day. The marketing damage from moving Linq to SQL to maintenance mode was huge.

 

  • Simplify, simplify, simplify! Microsoft does a lot of good things, but they tend to provide solutions that are wide scoped and complex. Often enough, so complex that people just give up on them. Case in point, if at all possible, I wouldn’t use WCF for communication. I would rather role my own RPC stack, because that makes things easier. That is a Fail for Microsoft in many cases, and one of the reasons that they are leaking developers to other platforms.

 

  • Forbid developers from arguing with customers publically about policy decisions. The reasoning is simple, devs don’t have a lot (if any) impact on policy decisions, they can explain them, sure. But when there is disagreement, having an argument about it tends to end up with bad blood in the water. Such discussions should be done at a level that can actually do something. Case in point, recent WebMatrix arguments in twitter & blogs.

 

  • Focus some love & attention toward the non entry level segment of the market. There has been a worrying set of events recently that indicate a shift in focus toward the entry level market, and the rest of developers are feeling abandoned. That shouldn’t be allowed to remain this way for long.

If you’ll not, except for the suggestion about turning simplification to an art form and applying it ruthlessly, most of the suggestions are actually about marketing and positioning, not about product development.

LightSwitch: The Return Of The Secretary

image Microsoft LightSwitch is a new 4GL tool from Microsoft, this is another in the series of “you don’t have to write any code” tools that I have seen.

Those are the tools that will give the secretary the ability to create applications and eliminate the need for coders. The industry has been chasing those tools since the 80s (does anyone remember the promises of the CASE tools?). We have seen many attempts at doing this, and all of them have run into a wall pretty quickly.

Oh, you can build a tool that gives you UI on top of a data store pretty easily. And you can go pretty far with it, but eventually your ability to point & click hit the limit, and you have to write code. And that is things totally breaks down.

LightSwitch is not yet publically available, so I have to rely on the presentation that Microsoft published. And I can tell you that I am filled with dread, based on what I have seen.

First of all, I strongly object to the following slide. Because I have the experience to know that working with a tool like that is akin to do back flips with a straightjacket on.

image

The capabilities of the tools that were shown in the presentation have strongly underwhelmed me in terms of newness, complexity or applicability.

Yeah, a meta data driven UI. Yeah, it can do validation on a phone number automatically (really, what happen with my Israeli based phone number?), etc. What is worse, even through the demo, I get the very strong feeling that the whole things is incredibly slow, you can see in the presentation multi second delays between screen repaints.

Then there are things like “it just works as a web app or windows app” which is another pipe dream that the industry has been chasing for a while. And the only piece of code that I have seen is this guy:

image 

Which makes me want to break down and cry.

Do you know why? Because this is going to be the essence of a SELECT N+1 in any system, because this code is going to run once per each row in the grid. And when I can find bugs from watching a presentation, you know that there are going to be more issues.

So, just for fun sake, since I don’t have the bits and I can rely only on the presentation, I decided to make a list of all the things that are likely to be wrong with LightSwitch.

I’ll review it when it comes out, and if it does manage to do everything that it does and still be a tool usable by developers, I’ll have to eat crow (well, Raven :-) ), but I am not overly worried.

Here are a few areas where I am feeling certain things will not work right:

  • Source control – how do I diff two versions of the app to see what changes? Are all changes diffable?
  • Programmatic modifications:
    • what happen when I want to write some code to do custom validation of property (for instance, calling a web service)?
    • what happen when I want to put a custom control on the screen (for instance, a google maps widget)?
  • Upsizing – when it gets to a 1,000 users and we need a full blown app, how hard it is to do?
  • Performance – as I said, I think it is slow from the demo.
  • Data access behavior – from what I have seen so far, I am willing to be that it hits its data store pretty heavily.

I fully recognize that there is a need for such a tool, make no mistake. And giving users the ability to do that is important. What I strongly object to is the notion that it would be useful for developers writing real apps, not forms over data. To put it simply, simple forms over data is a solved problem. There is a large number of tools out there to do that. From Access to Oracle Apex to FoxPro. Hell, most CRM solutions will give you just that.

My concern is that there seems to be an emphasis on that being useful for developers as well, and I strongly doubt that.

Microsoft.Data and Positioning

I just read a very insightful post from Evan Nagle about the topic. In it, Evan hit a key point:

Technically (and commercially) speaking, Microsoft has a slew of perfectly legitimate reasons for splattering the landscape with these "watered down" technologies. But, culturally speaking, it's painful to the professional Microsoft developer. It is. Because the Microsoft developer's personal and professional identity is tied up in the Microsoft stack, and that stack is now catering to a bunch of cat ladies and acne-laden teenagers. That's the real issue here.

If Microsoft thinks that they can get the market for bad developers, good for them. But that needs to be highly segmented. It should be clear that going with that route is leading you to a walled garden and that writing real apps this way is not good. The best comparison I can think of is Access apps in the 90s. There was a clear separation between “just wanna write some forms over data app over the weekend” and “I want to build a real application”. When you built an app in Access, you had very clear idea about the limitations of the application, and you knew that if you wanted something more, it would be a complete re-write.

That was a good thing, because it meant that you could do the quick & dirty things, but when things got messy, you knew that you had to make the jump.

The problem with things like Microsoft.Data is that there is no such line in the sand. And when you call it “Microsoft.*” you give it a seal of approval for everything. And when you have a piece of code that is explicitly designed to support bad coding practices, it is like peeing in the pool. If there is only one pool, it is going to affect everyone. There wouldn’t be nearly as much objection if it was called WebMatrix.Data, because that would clearly put it in someone else’s pool, and it that turn into a putrid swamp, I don’t really care.

There is another issue here, and it is not just that the community is afraid of inheriting horrible Microsoft.Data projects. It is much more than that.

Salary data is from the UK, because that is the first site I found with the numbers)

Now, I really can’t think of a good technical reason why VB.Net programmers are paid so much less, but those data points match what I have seen about salaries for developers in both positions.

In other words, VB.Net developers are getting paid a lot less for doing the same job.

Now, why is that? Is it because of the legacy of VisualBasic still haunts the VB guys? Because there is still the impression that VB is a toy language? I don’t know, but I think that at least in part, that is true. And that is what happen when a platform gets a reputation for being messy. People know in their bones that building stuff there is going to be costly, because maintaining this stuff is so hard.

Microsoft has repeatedly stated that they are after the people who currently go to PHP. Let me do the same comparison:

I don’t like those numbers. I really don’t like them.

Put simply, if Microsoft attempts to make the .NET platform more approachable for the PHP guys, it is also going to devalue the entire platform. I am pretty sure that this is something that they don’t want. Having a lot of hobbist programmer but fewer professional ones is going to hurt the Microsoft eco system.

Microsoft, if you want to create a PHP haven in the .NET world, that is great, but make sure that it is completely isolated, because otherwise you are going to hurt everyone who has a commercial stake in your platform.

I think that there is a lot of sense from commercial point of view in WebMatrix, but it should be clear that this isn’t .NET programming, this is WebMatrix programming. So if Microsoft succeed in gaining market share for this, it won’t be the .NET developers who would suddenly look at a 30% less money, it would be WebMatrix developers who would have to carry that stigma.

Tags:

Published at

Originally posted at

Comments (45)

TechEd Israel 2010 may only accept speakers from sponsors

This post is copied (with permission) from Roy Osherove. I don’t often do things like that but Roy’s post has pushed a lot of red buttons.

Let me just hand you over to Roy:

A month or so ago, Microsoft Israel started sending out emails to its partners and registered event users to “Save the date!” – Micraoft Teched Israel is coming, and it’s going to be this november!

“Great news” I thought to myself. I’d been to a couple of the MS teched events, as a speaker and as an attendee, and it was lovely and professionally done. Israel is an amazing place for technology and development and TechEd hosted some big names in the world of MS software.

A couple of weeks ago, I was shocked to hear from a couple of people that Microsoft Israel plans to only accept non-MS teched speakers, only from sponsors of the event. That means that according to the amount that you have paid, you get to insert one or more of your own selected speakers as part of teched.

I’ve spent the past couple of weeks trying to gather more evidence of this, and have gotten some input from within MS about this information. It looks like that is indeed the case, though no MS rep. was prepared to answer any email I had publicly. If they approach me now I’d be happy to print their response.

What does this mean?

If this is true, it means that Microsoft Israel is making a grave mistake –

  • They are diluting the quality of the speakers for pure money factors. That means, that as a teched attendee, who paid good money, you might be sitting down to watch nothing more that a bunch of infomercials, or sub-standard speakers – since speakers are no longer selected on quality or interest in their topic.
  • They are turning the conference from a learning event to a commercial driven event
  • They are closing off the stage to the community of speakers who may not be associated with any organization  willing to be a sponsor
  • They are losing speakers (such as myself) who will not want to be part of such an event. (yes – even if my company ends up sponsoring the event, I will not take part in it, Sorry Eli!)
  • They are saying “F&$K you” to the community of MVPs who should be the people to be approached first about technical talks (my guess is many MVPs wouldn’t want to talk at an event driven that way anyway )

I do hope this ends up not being true, but it looks like it is. MS Israel had already done such a thing with the Developer Days event previouly held in Israel – only sponsors were allowed to insert speakers into the event.

If this turns out to be true I would urge the MS community in Israel to NOT TAKE PART AT THIS EVENT in any form (attendee, speaker, sponsor or otherwise). by taking part, you will be telling MS Israel it’s OK to piss all over the community that they are quietly suffocating anyway.

The MVP case

MS Israel has managed to screw the MVP program as well. MS MVPs (I’m one) have had a tough time here in Israel the past couple of years. ever since yosi taguri left the blue badge ranks, there was not real community leader left. Whoever runs things right now has their eyes and minds set elsewhere, with the software MVP community far from mind and heart. No special MVP events (except a couple of small ones this year). No real MVP leadership happens here, with the MVP MEA lead (Ruari) being on a remote line, is not really what’s needed.

“MVP? What’s that?” I’m sure many MS Israel employees would say. Exactly my point.

Last word

I’ve been disappointed by the MS machine for a while now, but their slowness to realize what real community means in the past couple of years really turns me off. Maybe it’s time to move on. Maybe I shouldn’t be chasing people at MS Israel begging for a room to host the Agile Israel user group. Maybe it’s time to say a big bye bye and start looking at a life a bit more disconnected from that giant. I hear the people at Google are pretty Agile!

And now back to me. I had more discussions in the last two years with Microsoft UK than with Microsoft Israel. I think it says it all, and I am an Israeli MVP who spends most of his time in Israel.

Tags:

Published at

Originally posted at

Comments (5)

Microsoft Courier

There are some things that I just don’t get. The new MS Courier is one such thing. Check the link, this is basically a book size iPhone/tablet.

Go checkout the video, you’ll notice something that will kill this device.

It uses a pen, to write!

Leaving aside the fact that no OCR program has yet been able to figure out what I am writing (including the one in my brain), using a pen it annoying.

I write about three to four times as fast using a keyboard than using a pen (and I can use both hands). And I don’t write, using archaic pen & paper, much at all. That affect my writing readability when I am using a pen, of course, leading to a feedback cycle.

This pretty much turn me off of the device completely.

Hello!!! Is this ON?! [Connect Bugs sucks, again]

I want to point you to this bug:

image

Do you see the Won’t Fix there?

image

Let us take a look at what ExecutionEngineException is all about, okay?

image

Now, the real problem here is that along with this issue there is a failing sample, so it is not a case of not being able to reproduce the error. Granted, the customer has managed to find a workaround for their problem, but the bug in the framework is still there, and was not fixed. I would assume that CLR crashing bugs that “should never occur” would be given some priority, even if the customer managed to find a workaround.

Why Defer Loading in Entity Framework isn’t going to work

When I finished reading this post I let out a heavy sigh. It is not going to work. Basically, the EF is going the same way that NHibernate was in NHibernate 1.0 (circa 2005!).

Let me show you how. in the post, the example given is:

public class Category
{
    public int CategoryID { get; set; }
    public string CategoryName { get; set; }
    public virtual List<Product> Products { get; set; }
    ...
}

This looks like it would work, right? The way the EF is doing this is by creating a proxy of your class (similar to the way that NHibernate is doing that) and intercepting the call to get_Products.

It has a few problems, however. Let us start from the most obvious one, program to interface not to implementation. Exposing List<T> means that you have no control whatsoever about what is going on with the collection. It also means that they have to intercept the access to the property, not using the collection. That is bad, it means that they are going to have to do eager loading in all too many cases where NHibernate can just ignore it.

Let us more on a bit, and talk about the inability to support any interesting scenario. If we look at collections with NHibernate, we can take this:

Console.WriteLine(category.Products.Count);

To this SQL:

select count(*) from Products
where CategoryId = 1

With the approach that the EF uses, they will have to load the entire collection.

But all of those are just optimizations, not the make-or-break for the feature. Here is a common scenario that is going to break it:

public class Category
{
    private List<Product> _products;
    public int CategoryID { get; set; }
    public string CategoryName { get; set; }
    public virtual List<Product> Products { get { return _products; } set { _products = value; } }

    public void AddProduct(Product p)
    {
        // do something interesting here
        _products.Add(p);
    }
    ...
}

There is a reason why the default proxies for NHibernate force all members of the entities to be virtual. It is not just because we think everything should be virtual (it should, but that is not a discussion for now). It is all about being able to allow the user to use a real POCO.

In the scenario outlined above, what do you think is going to happen?

AddProduct is a non virtual method call, so it cannot be intercepted. Accessing the _products field also cannot be intercepted.

The end result is a NullReferenceException that will seem to appear randomly, based on whatever something else touched the Property or not. And however nice auto properties are, there are plenty of reason to use fields. Using auto properties just mask the problem, but it is still there.

Oh, and if we want to use our POCO class with EF, forget about the new operator, you have to use:

Category category = context.CreateObject<Category>();

Hm, this just breaks just about anything that relies on creating new classes. Want to use your entity as parameter for ASP.Net MVC action, to be automatically bounded? Forget about it. You have to create your instances where you have access to the context, and that is a database specific thing, not something that you want to have all over the place.

And I really liked this as well:

The standard POCO entities we have talked about until now rely on snapshot based change tracking – i.e. the Entity Framework will maintain snapshots of before values and relationships of the entities so that they can be compared with current values later during Save. However, this comparison is relatively expensive when compared to the way change tracking works with EntityObject based entities.

Hm, this is how NHibernate and Hibernate have always worked. Somehow, I don’t see this showing up as a problem very often.

Final thought, why is the EF calling it Defer Loading? Everyone else call it lazy loading.

How Microsoft should release guidance?

Phil Haack has a post about code sample taxonomy, in which he asks how Microsoft can ship high quality sample apps:

Obviously, this [shipping high quality samples] is what we should be striving for, but what do we do in the meantime? Stop shipping samples? I hope not.

Again, I don’t claim to have the answers, but I think there are a few things that could help. One twitter response made a great point:

a reference app is going to be grilled. Even more if it comes from the mothership. Get the community involved *before* it gets pub

Getting the community involved is a great means of having your code reviewed to make sure you’re not doing anything obviously stupid. Of course, even in this, there’s a challenge. Jeremy Miller made this great point recently:

We don't have our own story straight yet.  We're still advancing our craft.  By no means have we reached some sort of omega point in our own development efforts. 

In other words, even with community involvement, you’re probably going to piss someone off.

Um, no.

I feel that Jeremy’s point has been pulled out of context. While we may have disagreements about what constitute the future directions in software development, that isn’t actually what cause the community to reject the bad samples from Microsoft.

No one said a word about Kobe because it didn’t use NHibernate, or wasn’t built with messaging or using DDD.

What we reject in those sample app is that they are, put simply, bad coding. There is nothing amorphous about this, we aren’t talking about subjective feeling, we are talking about very concrete, objective and real metrics that we can apply.

Bad Code

Not some omega point.

Bad Code

Not lack of understanding of the platform because the developers writing that were new to ASP.Net MVC.

Bad Code

That would have been bad code if it was written 5 years ago or 15 years ago.

That is the problem.

And I think that there is quite a simple answer for that. Stop shipping official guidance package without involvement from the community.

Create a special CodePlex project where you are being explicit about putting things for review, not for publishing. After the guidance has been published, it is probably too late already.

Get some feedback from the community, then you can publish this in the usual places.

What do I expect from guidance?

Phil Haack has a post about code sample taxonomy. He suggests the following taxonomy:

  • Prototype
  • Demo
  • Sample
  • Production
  • Reference

I am not sure that I agree with his division. From my point of view, demo code need to focus on one single aspect, mostly because we are going to talk around that, and doing that means that I don’t have the time to deal with anything but the aspect that I need to talk about.

Sample code is a more fleshed out example.

I don’t make any distinction between reference and sample apps, maybe the size is different but that is about it.

What is important to follow is that for all of them, except maybe prototype (and I was bitten by that before), I have the same code quality expectations.

Note that I am saying code quality vs. production readiness. Most of the code that you see in demos or sample apps is not production ready. And that is fine.

Production readiness include such things as:

  • Parameter validation
  • Security
  • Error handling
  • Failover & recovery

And a lot more “taxes” that we need to deal with.

That does not include code quality. Code quality is a metric of its own, and it is one that you cannot give up. I would be personally ashamed to put out something that didn’t meet my own code quality metrics.

Sometimes, being production ready means that you have to give up on code quality, here is one example, that is understood and acceptable in certain circumstances.

What is not acceptable is to say that this is “Xyz” type of code sample, therefore code quality doesn’t matter.

And code quality isn’t really some amorphous thing. It is a very well defined metric that can be automatically checked. FxCop or NDepend are good tools to have, Simian will give you a lot of value all on its own.

When you put something out as an official Microsoft guidance release, the bar is higher, because whatever you do will be used by other people.

Microsoft Connect - FAIL (yet again)

I really think that Microsoft should close Connect. Because it is pretty obvious that they aren't managing that properly.

Let us take a look at yet another interesting bug report. This is related to a bug in System.Data that makes using System.Transactions even trickier than you would initially believe.

It was acknowledged as a bug by Miguel Gasca (from Microsoft), and a connect was reported.

That was in 2007(!), it was resolved, a month later, by "Microsoft", because it is "External" issue.

That bug is till here today, two years later, and still impacting customers. That is after a full release and SP1. The situation is that now I have to work around this bug because Microsoft cannot manage its own bugs database in a way that would allow it to... oh, I don't know, fix bugs!

FAIL

And you know what, I wouldn't be annoyed with this if this wasn't an ongoing pattern with Connect.

FAIL

MEF & Open Generic Types

I read Glenn' s post about MEF's not supporting open generic types with somewhat resembling shock. The idea that it isn't supporting this never even crossed my mind, it was a given that this is a mandatory feature for any container in the .NET land.

Just to give you an idea, what this means is that you can't register Repository<T> and then resolve Repository<Order>. In 2006, I wrote an article for MSDN detailing what has since became a very common use of this pattern. Generic specialization is not something that I would consider optional, it is one of the most common usage patterns of containers in the .NET land. IRepository<T> is probably the most common example that people quote, but there are others as well.

This is not a simple feature, let me make it clear. Not simple at all. I should know, I implement that feature for both Object Builder and Windsor. But that is not what I would consider an optional one.

I am even more disturbed by the actual reasoning behind not supporting this. It is a technical limitation of MEF because internally all components are resolved by string matching, rather than CLR Types. This decision is severely limiting the things that MEF can do. Not supporting what is (in my opinion) is a pretty crucial feature is one example of that, but there are other implications. It means that you can't really do things like polymorphic resolutions, that your choices in extending the container are very limited, because the container isn't going to carry the information that is required to make those decision.

I would advice the MEF team to rethink the decision to base the component resolution on strings. At this point in time, it is still possible to change things ( and yes, I know it isn't as easy as I make it seems ), because not supporting open generic types is bad, but not having the ability to do so, and the reason for that (not keeping CLR Type information) are even worse. I get that MEF needs to work with DLR objects as well, but that means that MEF makes the decision to have lousier support for CLR idioms for the benefit of the DLR.

Considering the usage numbers for both of them, I can't see this being a good decision. It is certainly possible to support them both, but if there are any tradeoffs that have to be made, I would suggest that it should be the DLR, and not the CLR, which would be the second class role.

Tags:

Published at

Originally posted at

Comments (3)

And now what?

It looks like the entire MSMQ .NET stack is riddled with threading bugs. At least if you think about using the async methods such as BeginPeek.

Nasty!

image

MessageQueue trouble continues

I mentioned that I got into some problems with MSMQ that I couldn’t reproduce later on. Well, here is the actual code that I am running that is causing a hung. As you can see, this is really strange.

image

More information of GC issue

After a lot more study, it looks like there are two separate issues that are causing the problem here.

  1. During AppDomain unload, it is permissible for the GC to collect reachable objects. I am fine with that and I certainly agree that this makes sense.
  2. Application_End occurs concurrently with the AppDomain unload.

Looking at the docs (and there are surprisingly few about this), it seems like 1 is expected, but 2 is a bug. The docs state:

Application_End  - Called once per lifetime of the application before the application is unloaded.

It may be my English, but to me before doesn’t suggest at the same time as.

So I do believe it is a bug in the framework, but not in the GC, it is in the shutdown code for ASP.Net. This is still a very nasty problem.

I am being stalked by CLR bugs

imageI just spent several hours tracking down a crashing but in my current project.

The  issue was, quite clearly, a problem with releasing unmanaged resources. So I tightened my control over resources and made absolutely sure that I am releasing everything properly.

I simply could not believe what was going on. I knew what they code is doing, and I knew that what I was getting was flat out impossible.

Yes, I know that we keep saying that, but this bug really is not possible!

The situation is quite clear, during the shutdown process of the application, an unmanaged resource’s finalizer would throw an exception because it wasn’t properly disposed.

The problem? It is most assuredly should be disposed. When debugging through the problem, I found out something extremely strange and worrying. The managed object’s finalizer was running while there were strong references to the object.

You can see that in the attached screenshot (click the image see it in full size).

That is, by the way, when you have the root in a static field, so it cannot be that the whole graph is free.

This is somehow related to threading, because debugging this would often change the way this works, but running without a debugger consistently fails.

The CLR semantics for finalizers clearly state that they can only be run after there are no more strong references to the instance. Cleary, this is not what is going on here.

The only thing that I can think of that can affect this is that there is something really strange going on with app domain unloads.

Now, I can’t figure if this is me being extremely stupid or if this is a real problem. I did manage to create a reproduction of the issue, however, which you can download here.

This is on VMWare Fusion, running Windows 2008 x64, with .Net 3.5 SP1.

To reproduce, start the application in WebDev.WebServer, wait for the page to load, and the close the WebDev.WebServer. If it crashes, you have successfully reproduced the problem.

Update – This is really interesting. both stack traces are operating on the same object, by the way.

image

Adding locking around the finalizer and dispose seems to have made the problem go away.

Hidden Windows Gems: Extensible Storage Engine

Did you know that Windows came with an embedded database?

Did you know that this embedded database is the power behind Active Directory & Exchange?

Did you know that this is actually part of Windows' API and is exposed to developers?

Did you know that it requires no installation and has zero administration overhead?

Did you know there is a .Net API?

Well, the answer for all of that is that you probably didn't know that, but it is true!

The embedded database is called Esent, and the managed library for this API was just released.

This is an implementation of ISAM DB, and I have been playing around with it for the last few days. It isn't as nice for .Net developers as I would like it to be (but Laurion is working on that).

I think making this public is a great thing, and the options that this opens up are quite interesting. I took that for a spin and came up with this tiny bit of code that allow me to store JSON documents:

https://rhino-tools.svn.sourceforge.net/svnroot/rhino-tools/branches/rhino-divandb

It is not done, not nearly done, but the fact that I could rely on the embedded DB to do so made my life so much easier. I wish I knew about that when I played with Rhino Queues, it would have made my life so much simpler.

Oxite: Open eXchangable Informative Troubled Engine

This is a post about Oxite, but it is actually not about the code base. There has been a lot said about this, and I think that I can solidly put myself on the side that spouts WTF and now seeks access to memory altering drugs.

What I wanted to talk about here is actually the way that this was handled. It didn't take long for people to start noticing the actual code quality of this in contrast to its marketing promise:

We heart you so much, that we thought of everything. Oxite was developed carefully and painstakingly to be a great blog sample, or a starting point for your own web site project with CMS needs. Its line-up of sexy attributes includes: provider-based architecture allowing you to swap out database and search providers (SQL Server DB, local and Live search providers included), built for testability and hence most likely to be voted "hottest in class" by TDD fans (repositories, everything has an interface, etc.)

The community reaction was mostly negative, especially since this was positioned as official sample application. The marketing for Oxite is actually quite amazing, when you consider that they have a channel 9 show, a visit mix site, a really nicely done codeplex site, a push from Microsoft evangelist and other things that I forget.

Very quickly, there was a response from more well known sources at Microsoft. It is interesting to watch the actual phrasing of the responses from those people. You have to admire the level of verbal acrobatics that they had do do. Again, I am more interested in the back story than in the code.

Scott Hanselman has this to say about the way Oxite was released:

I don't know the Oxite guys, and I didn't know they were working on anything. Thing is, anyone at Microsoft can put a project together, blog about it, and say "this aims to be a good example of...". Clearly it's not a good example and it shouldn't be advertised as one.

The fact that PhilHa, who owns ASP.NET MVC, didn't know it was going on, kind of makes that point. Any samples from Microsoft that intend to show a Best Practice should be reviewed by Phil, and certainly vetted by the community. David Nelson nailed that point clearly.

I think that this is really interested. "Anyone at Microsoft can put a project together and..."

Well, actually, no, they can't. Oh, they can do that for personal stuff, but they can't do that for Microsoft stuff. I had the chance to work with the P&P a couple of times, and I had the pleasure of seeing the release process from their side. Some of that is unique for P&P, I assume, but there are things that are certainly shared for all of Microsoft. A security review, for example, or the bloody annoying trademark stuff that give us names such as Windows Pillow Foundation Community Extended Express Edition for Professional Database Developers And Administrators 2008 R4.2.

There are other things, and among them, especially if you put out a best practice app, is to actually talk to the team that own this.

So, they can't do that. Or, to be rather more exact, they shouldn't have done that. Except that it appears that they did, I can see no other option given the information that is publicly available. Because there is absolutely no question that Oxite has been extremely damaging to Microsoft from a PR perspective. The worst part is that it was pushed as "we thought of everything" and "hottest in class".

We can compare those to Scott's BabySmash or to Rob Conery's Storefront (although I don't believe that even at their worst, they would have made me cry). Those were clearly labeled as learning projects, and that made all the difference.

Which bring us to the current state of affairs. Scott Hanselman said something that I utterly disagree with about this:

Oxite is Open Source. If you don't like it, and there are very valid reasons today for concern, I would sincerely encourage you to refactor it and share the results.

Open Source is not a way to get other people to fix bad code. I think that this is the wrong thing to do. However it got there, this was published with the impression that Microsoft is standing behind it. While my response would have been to kill the project, remove the site and deny that it has ever existed, Microsoft has chosen to keep it alive. I believe that by doing so, and I think that they have done so by not either killing the project or officially stating that this is not a Microsoft project, that Microsoft is now accountable for this project quality and its fitness to serve as an example application.

And yes, I do mean that what looks very much like a rouge project being released into the wild is now Microsoft's problem to fix. See above for the difference between stating the source (Microsoft vs. individual) and the effect that it has on the perception of the project.

Rob Conery's has picked up the ball and seems to be making rapid progress at it, judging from his twitter stream.

I think that this is the best solution all together. We get to see a good restructuring approach, and knowing Rob, we will end up with a good sample app. More to the point, that was the appropriate response to what happened.

It also has another side effect, which is pretty major one, it turning all that negative PR into a positive one, by stepping in, acknowledging the problems and fixing them.

Consenting Adults

David Kean has an interesting post here: "I can't believe Microsoft didn't make [Enter API Name Here] public"

In particular, he mentions this aging argument:

Developing good APIs is extremely difficult, especially the first time around. When we get things wrong (and we do get things wrong), trying to maintain backwards compatibility often stifles innovation as we try to correct these scenarios. Unfortunately, unlike others, we don't have the luxury to make only 95% of our new versions backwards compatible with our previous versions.

I have a lot of problems with this statement. Chief among them is the treating the users as people who should be cuddled. Actually, I don't really have a problem with cuddling, the real problem is that this is literally putting road blocks for progress.

The Microsoft backward compatibility strategy is a major reason why the default approach is to close down everything. Now, it is not as if there isn't a well defined way of saying: Touch that on your responsibility. This is quite pervasive, actually:

This API supports the .NET Framework infrastructure and is not intended to be used directly from your code.

But I think that this isn't actually the problem. The problem is that the rules that Microsoft has chosen to accept are broken. In that type of game, you are always losing. The only thing that you can do is to change the rate in which you are losing at.

Change the rules, treat the users as consenting adults, and you'll be able to do so much more, because the task ahead of you would be so much simpler.