Ayende @ Rahien

Error handling in the field

Mon, 17 Oct 2022 12:00:00 GMT

I’m not talking about this much anymore, but alongside RavenDB, my company produces a set of tools to help you work with OR/M (object relational mappers) such as NHibernate or Entity Framework as well as tracking what is going on with Cosmos DB.

The profilers are implemented as two separate components. We have the Appender, which runs inside the profiled process, and the Profiler, which is a WPF application that analyzes and shows you the results of the profiling. For the profilers, all the execution is done on the users’ machine.

We have crash reporting enabled and we are diligent in fixing any and all errors from the field. We recently ran into a whole spate of errors, looking something like this:

System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Windows.Controls.VirtualizingStackPanel.UpdateExtent(Boolean areItemChangesLocal)
   at System.Windows.Controls.VirtualizingStackPanel.ShouldItemsChangeAffectLayoutCore(Boolean areItemChangesLocal, ItemsChangedEventArgs args)
   at System.Windows.Controls.VirtualizingPanel.OnItemsChangedInternal(Object sender, ItemsChangedEventArgs args)
   at System.Windows.Controls.Panel.OnItemsChanged(Object sender, ItemsChangedEventArgs args)
   at System.Windows.Controls.ItemContainerGenerator.OnItemAdded(Object item, Int32 index, NotifyCollectionChangedEventArgs collectionChangedArgs)
   at System.Windows.Controls.ItemContainerGenerator.OnCollectionChanged(Object sender, NotifyCollectionChangedEventArgs args)
   at System.Windows.WeakEventManager.ListenerList`1.DeliverEvent(Object sender, EventArgs e, Type managerType)
   at System.Windows.WeakEventManager.DeliverEvent(Object sender, EventArgs args)
   at System.Collections.Specialized.NotifyCollectionChangedEventHandler.Invoke(Object sender, NotifyCollectionChangedEventArgs e)
   at System.Windows.Data.CollectionView.OnCollectionChanged(NotifyCollectionChangedEventArgs args)
   at System.Windows.WeakEventManager.ListenerList`1.DeliverEvent(Object sender, EventArgs e, Type managerType)
   at System.Windows.WeakEventManager.DeliverEvent(Object sender, EventArgs args)
   at System.Windows.Data.CollectionView.OnCollectionChanged(NotifyCollectionChangedEventArgs args)
   at System.Windows.Data.ListCollectionView.ProcessCollectionChangedWithAdjustedIndex(NotifyCollectionChangedEventArgs args, Int32 adjustedOldIndex, Int32 adjustedNewIndex)
   at System.Collections.Specialized.NotifyCollectionChangedEventHandler.Invoke(Object sender, NotifyCollectionChangedEventArgs e)
   at System.Collections.ObjectModel.ObservableCollection`1.OnCollectionChanged(NotifyCollectionChangedEventArgs e)
   at Caliburn.Micro.BindableCollection`1.OnCollectionChanged(NotifyCollectionChangedEventArgs e)
   at System.Collections.ObjectModel.ObservableCollection`1.InsertItem(Int32 index, T item)
   at Caliburn.Micro.BindableCollection`1.OnUIThread(Action action)
   at HibernatingRhinos.Profiler.Client.Sessions.SessionListModel.Add(SessionModel model)

And here is the relevant code:

This is called from a timer thread (not from the UI) one, and the Items collection in this case is a BindableCollection<T>.

The error is happening deep in the guts of WPF and it seems like it has been triggered by some recent Windows update. Here is the “fix” for this issue:

Basically, don’t report this error, and continue executing normally (the next UI operation will fix the UI state, usually within < 200 ms).

This is the right call in terms of development time and effort, but I got to say, this makes me feel quite uncomfortable to see a change like that.

NHibernate Profiler video

Wed, 17 Feb 2021 12:00:00 GMT

Entity Framework Profiler video

Tue, 16 Feb 2021 12:00:00 GMT

NHibernate Profiler & Entity Framework Profiler 6.0 RTMed!

Wed, 16 Dec 2020 12:00:00 GMT

I’m very happy to announce that we have recently released version 6.0 of Entity Framework Profiler and NHibernate Profiler. Here are some of the highlights:

This new version brings quite a lot to the table. We applied a lot of lessons to optimize the performance of the profiler so it can process events faster and in a more efficient manner. We also fully integrated it with the async/await model that became so popular. For Entity Framework users, we now support all versions of EF, running on .NET 4.7 all the way to .NET 5.0 and anything in between.

What I think is the crown jewels of this release, however, is the new Azure integration feature. We initially built this feature to allow you to profiler serverless code and Azure Functions, but it turned out that this is really useful.

The Profiler Azure Integration allows you to setup a storage container on Azure that would accept the profiled output from your application. Pretty simple, right? The profiler application will monitor the container and show you the events as they are written, so even if you don’t have direct access to profiler the code (very common on serverless / Azure scenarios), you can still profiler your code. That helps a lot as well when talking about running inside containers, Kubernetes, etc. The usual way the profiler and the application communicate is over a TCP channel, but given the myriad of network topologies that are now in common use, this can get complex. By utilizing Azure Integration, we can resolve the whole solution and give you a seamless way to handle profile your code.

On demand profiling of your code is just one part of the new feature. You can now do continuous profiling of your system. Because we throw the events into a blob container in Azure, and given that the data is compressed and cheap to store, you can now afford to simply record all the queries in your application and come back to it a few days or weeks later and look into interesting behaviors.

That is also a large part of why we worked on improving our performance, we expect to be dealing with a lot of data.

You are also able to specify a certain timeframe for this kind of profiling, as well as specify whatever we should remove the data after looking on it or retain it. The new profiling feature gives you a way to answer: “What queries run in production on Monday between 9:17 and 10:22 AM”. This comes with all of the usual benefits of the profiler, meaning that:

You can correlate each query to the exact line of code that triggered it.
The profiler analyze the database interaction and raise alerts on bad behaviors.

The last part is done not on a single query but with access to the full state of the system. It can allow you to find hot spots in your program, patterns of data access that leads to much higher latency and cost a lot more money.

To celebrate the new release, we are offering the profilers with 20% discount for the new quarter as well as offer a bundling option. You can get the Entity Framework Profiler or the NHibernate Profiler bundled with our Cosmos DB Profiler.

We’re hiring… come work for us

Wed, 13 Aug 2014 09:42:00 GMT

It seems that recently we have been going in rounds. Get more people, train them, and by the time they are trained, we already need more people .

I guess that this is a good problem to have. At any rate, we are currently looking for an experience well rounded developer.

This job availability is for our offices in Hadera, Israel. If you aren’t from Israel, this isn’t for you.

This job is primarily for work on our Profilers line of products. Here is the laundry list:

Awesome .NET skills
Experience in UI development using WPF, MVVM style
Understanding how computers work and how to make them dance
History with concurrency & multi threading (concurrent work history not required)
Architecture / design abilities

I would like to see open source history, or projects that you can share (in other words, your projects, not employer’s code that you try to give to look at).

Please contact us at jobs@hibernatingrhinos.com if you are interested.

Uber Prof 2.0 is OUT!

Tue, 01 Jan 2013 10:00:00 GMT

Exactly 4 years after the 1.0 release of Uber Prof, we have the 2.0 release for Uber Prof.

You can find it here: http://hibernatingrhinos.com/products/

New features include production and cloud profiling, performance improvement and better error detection & guidance.

Goodbye, 2012: Our end of year discount starts now!

Fri, 21 Dec 2012 05:02:00 GMT

Well, as the year draws to a close, it is that time again, I got older, apparently. Yesterday marked my 31th trip around the sun.

To celebrate, I decided to give the first 31 people a 31% discount for all of our products.

This offer applies to:

This also applies to our support & consulting services.

All you have to do is to use the following coupon code: goodbye-2012

Enjoy the end of the year, and happy holidays.

Production Cloud Profiling With Uber Prof

Tue, 18 Dec 2012 10:00:00 GMT

With Uber Prof 2.0 (NHibernate Profiler, Entity Framework Profiler, Linq to SQL Profiler, LLBLGen Profiler) we are going to bring you a new era of goodness.

In 1.0, we gave you a LOT of goodness for the development stage of building your application, but now we are able to take it a few steps further. Uber Prof 2.0 supports production profiling, which means that you can run it in production and see what is going on in your application now!

To make things even more interesting, we have also done a lot of work to make sure that this works on the cloud as well. For example, go ahead and look at this site: http://efprof.cloudapp.net/

This is a live application, that doesn’t really do anything special, I’ll admit. But the kicker is when you go to this URL: http://efprof.cloudapp.net/profiler/profiler.html

This is EF Prof, running on the cloud, and giving you the results that you want, live. You can read all about this feature and how to enable it here, but I am sure that you can see the implications.

Über Profiler v2.0–Private Beta is open

Thu, 16 Aug 2012 09:00:00 GMT

Well, it took a while, but the next version of the NHibernate Profiler is ready to go to a private beta.

We go to private beta before the product is done, because we want to get as much early feedback as we can.

We have 10 spots for NHibernate Profiler v2.0 and 10 spots for Entity Framework Profiler v2.0.

If you are interested, please send me an email about this.

What is next for the profilers?

Thu, 02 Jun 2011 09:00:00 GMT

We have been working on the profilers (NHibernate Profiler, Entity Framework Profiler, Linq to SQL Profiler, LLBLGen Profiler and Hibernate Profiler) for close to three years now. And we have been running always as 1.x, so we didn’t have a major release (although we have continual releases, we currently have close to 900 drops of the 1.x version).

The question now becomes, what is going to happen in the next version of the profiler?

Production Profiling, the ability to setup your application so that you can connect to your production application and see what is going on right now.
Error Analysis, the ability to provide you with additional insight and common solution to recurring problems.
Global Query Analysis, the ability to take all of your queries, look at their query plans and show your potential issues.

Those are the big ones, we have a few others, and a surprise in store

What would you want to see in the next version of the profiler?

Uber Prof & NuGet

Fri, 01 Apr 2011 11:00:00 GMT

Fitzchak has all the details in our company blog.

New Profiler Feature: Avoid Writes from Multiple Sessions In The Same Request

Tue, 01 Mar 2011 12:26:00 GMT

Because I keep getting asked, this feature is available for the following profilers:

This new feature detects a very interesting bad practice, write to the database from multiple session in the same web request.

For example, consider the following code:

public void SaveAccount(Account account)
{
    using(var session = sessionFactory.OpenSession())
    using(session.BeginTransaction())
    {
           session.SaveOrUpdate(account);
           session.Transaction.Commit();    
    }
}
public Account GetAccount(int id)
{
    using(var session = sessionFactory.OpenSession())
    {
        return session.Get<Account>(id);
    }
}

It is bad for several reasons, micro managing the session is just one of them, but the worst part is yet to come…

public void MakePayment(int fromAccount, int toAccount, decimal ammount)
{
    var from = Dao.GetAccount(fromAccount);
    var to = Dao.GetAccount(toAccount);
    from.Total -= amount;
    to.Total += amount;
    Dao.SaveAccount(from);
    Dao.SaveAccount(to);
}

Do you see the error here? There are actually several, let me count them:

We are using 4 different connections to the database in a single method.
We don’t have transactional safety!!!!

Think about it, if the server crashed between the fifth and sixth lines of this method, where would we be?

We would be in that wonderful land where money disappear into thin air and we stare at that lovely lawsuit folder and then jump from a high window to a stormy sea.

Or, of course, you could use the profiler, which will tell you that you are doing something which should be avoided:

Isn’t that better than swimming with the sharks?

New Uber Prof Feature: Too Many Database Calls In The Same Request

Mon, 28 Feb 2011 12:15:00 GMT

Recently, we added a way to track alerts across all the sessions the request. This alert will detect whenever you are making too many database calls in the same request.

But wait, don’t we already have that?

Yes, we do, but that was limited to the scope of one session. there is a very large set of codebases where the usage of OR/Ms is… suboptimal (in other words, they could take the most advantage of the profiler abilities to detect issues and suggest solutions to them), but because of the way they are structured, they weren’t previously detected.

What is the difference between a session and a request?

Note: I am using NHibernate terms here, but naturally this feature is shared among all profiler:

NHibernate Profiler

Entity Framework Profiler

Linq to SQL Profiler

LLBLGen Profiler

Hibernate Profiler

A session is the NHibernate session (or the data/object context in linq to sql / entity framework), and the request is the HTTP request or the WCF operation. If you had code such as the following:

public T GetEntity<T>(int id)
{
    using (var session = sessionFactory.OpenSession())
    {
         return session.Get<T>(id);
    }
}

This code is bad, it micro manages the session, it uses too many connections to the database, it … well, you get the point. The problem is that code that uses this code:

public IEnumerable<Friends> GetFriends(int[] friends)
{
   var results = new List<Friends>();
   foreach(var id in friends)
       results.Add(GetEnttiy<Friend>(id));

   return results;
}

The code above would look like the following in the profiler:

As you can see, each call is in a separate session, and previously, we wouldn’t have been able to detect that you have too many calls (because each call is a separate session).

Now, however, we will alert the user with a too many database calls in the same request alerts.

New Uber Prof Concept: Cross Session Alerts

Fri, 25 Feb 2011 10:05:00 GMT

We have recently been doing some work on Uber Prof, mostly in the sense of a code review, and I wanted to demonstrate how easy it was to add a new feature. The problem is that we couldn’t really think of a nice feature to add that we didn’t already have.

Then we started thinking about features that aren’t there and that there wasn’t anything in Uber Prof to enable, and we reached the conclusion that one limitation we have right now is the inability to analyze your application’s behavior beyond the session’s level. But there is actually a whole set of bad practices that are there when you are using multiple sessions.

That led to the creation of a new concept the Cross Session Alert, unlike the alerts we had so far, those alerts looks at the data stream with a much broader scope, and they can analyze and detect issues that we previously couldn’t detect.

I am going to be posting extensively on some of the new features in just a bit, but in the meantime, why don’t you tell me what sort of features do you think this new concept is enabling.

And just a reminder, my architecture is based around Concepts & Features.

Uber Prof New Features: A better query plan

Sat, 15 Jan 2011 10:00:00 GMT

Originally posted at 1/7/2011

Because I keep getting asked, this feature is available for the following profilers:

This feature is actually two separate ones. The first is the profiler detecting what is the most expensive part of the query plan and making it instantly visible. As you can see, in this fairly complex query, it is this select statement that is the hot spot.

Another interesting feature that only crops up whenever we are dealing with complex query plans is that the query plan can get big. And by that I mean really big. Too big for a single screen.

Therefore, we added zooming capabilities as well as the mini map that you see in the top right corner.

Uber Prof New Features: Go To Session from alert

Fri, 14 Jan 2011 10:00:00 GMT

Originally posted at 1/7/2011

This is another oft requested feature that we just implemented. The new feature is available for the full suite of Uber Profilers:

You can see the new feature below:

I think it is cute, and was surprisingly easy to do.

Uber Prof have recently passed the stage where it is mostly implemented using itself, so I just had to wire a few things together, and then I spent most of the time just making sure that things aligned correctly on the UI.

Happy New Year, and across the board profiler discount

Fri, 31 Dec 2010 09:11:56 GMT

To celebrate the new year, I decided to offer a single day coupon that will get you 35% discount for all my profiler products.

The coupon code is valid until the 1st of January 2011, but I won’t mention in which timezone, so you might want to hurry up.

The coupon code is:

HNY-45K2D465DD

You can use it to buy:

Happy new year!

What is Uber Prof’s competitive advantage?

Sat, 27 Nov 2010 10:00:00 GMT

Originally posted at 11/25/2010

In a recent post, I discussed the notion of competitive advantage and how you should play around them. In this post, I am going to focus on Uber Prof. Just to clarify, when I am talking about Uber Prof, I am talking about NHibernate Profiler, Entity Framework Profiler, Linq to SQL Profiler, Hibernate Profiler and LLBLGen Profiler. Uber Prof is just a handle for me to use to talk about each of those.

So, what is the major competitive advantage that I see in the Uber Prof line of products?

Put very simply, they focus very heavily on the developer’s point of view.

Other profilers will give you the SQL that is being executed, but Uber Prof will show you the SQL and:

Format that SQL in a way that make it easy to read.
Group the SQL statements into sessions. Which let the developer look at what is going on in the natural boundary.
Associate each query with the exact line of code that executed it.
Provide the developer with guidance about improving their code.

There are other stuff, of course, but those are the core features that make Uber Prof into what it is.

Profiler Usage Analysis

Tue, 17 Aug 2010 09:00:00 GMT

I have been doing some studying of how people are using the profiler, and it shows some interesting results.

Typical profiler session is :

NH Prof : 1:15 hours
Hibernate Profiler: 1:05 hours
EF Prof: 42 minutes
L2S Prof: 50 minutes

83% of the profiler users have used it more than once. In fact, here is the # of usages:

So we have over 50% that use it regularly.
Most people use it predominately to view the statements executed:

This means that the reports are getting comparatively little attention.
The results per geographical location are also interesting:

Hibernate Profiler New Feature: Parameters Values

Tue, 09 Mar 2010 12:12:00 GMT

One of the annoying things about the Hibernate port of the profiler was that JDBC didn’t provide us with the parameters values.

Eric has just fixed and that is now live:

Enjoy…