Ayende @ Rahien

Refunds available at head office

Code Reviews are not just about the code

Usually when I am doing a code review, I ask not just access to the code, but to the source control repository as well. There are quite a lot that you can learn from the source control of a project.

Here is something that is both good and bad. It is bad because we have no checkin comments. It is good because we can see that we have very frequent checkins.

image 

Another thing that you want to check is the bug tracking system. Is there such a thing (including email thread or a post it notes or a story wall)? Is it relevant?

Is there a defined build process? How do you deploy to production? How do you manage DB changes? What is you back off strategy from an ambitious feature? How do you rollback a failed deployment? Do you have a physical deployment plan?

I don't expect all projects to have detailed answers to the all the questions I put above, but I do expect to have a response. The worst possible thing that could happen is that there is no thought given to those.

My standard answer to "How do you manage DB changes?" is: I create a diff using SQL Compare and apply that when needing to update DB versions.

The most important thing in my opinion is the actual interaction in code reviews. I tend to have two stages of code review. The first is going over the code with only general idea about what is going on. This is to find if there are things that require special attention, if there are obvious issues there.

In general, you want to be able to view this from a clean mind. Then, you want to have a conversation with the developers. Based on that initial review. This allows you to capture more than just the code, it let you capture the actual dynamic of the team and how they work. During this phase I also get a lot more

The second phase of the code is much more in depth, I tend to try to go over every line in the code at least once. After that, I can sit with the developers again and discuss the overall impression.

Probably the most value in the process comes at this second state, I think.

Logging - the AOP way

Logging is probably the most mentioned sweet spot of AOP. Probably because it is the simplest and most straightforward example most people can think of. I remember going over a piece of code that handles billions of transactions a day(that is for you, Jdn), and seeing something like this:

public void DoSomethingInteresting(string arg1, int arg2)
{
	LogGateway.Enter("DoSomethingInteresting", arg1, arg2);
	try
	{
		// actual method code 
		LogGateway.Success("DoSomethingInteresting", arg1, arg2);
	}
	catch(Exception e)
	{
		LogGateway.Error(e, "DoSomethingInteresting", arg1, arg2);
		throw e;
	}
	finally
	{
		LogGateway.Exit("DoSomethingInteresting", arg1, arg2);
	}
}

That was repeated for each and every method in the system. It was horrible.

I don't care how you do it, but there are at least 7 AOP approaches in the CLR, at least three of them are applicable in your environment.

I am going to show how to handle that with Windsor's AOP, but the same thing is applicable using other approaches. First, our service:

public interface ISmsSender
{
	int Send(string to, string msg);
}

public class SmsSender : ISmsSender
{
	public int Send(string to, string msg)
	{
		if(msg.Length>160)
			throw new ArgumentException("too long","msg");
		return to.Length;
	}
}

And now we need to define the interceptor:

public class LoggingInterceptor : IInterceptor
{
	public void Intercept(IInvocation invocation)
	{
		var logger = LogManager.GetLogger(invocation.TargetType);
		try
		{
			StringBuilder sb = null;
			if (logger.IsDebugEnabled)
			{
				sb = new StringBuilder(invocation.TargetType.FullName)
					.Append(".")
					.Append(invocation.Method)
					.Append("(");
				for (int i = 0; i < invocation.Arguments.Length; i++)
				{
					if (i > 0)
						sb.Append(", ");
					sb.Append(invocation.Arguments[i]);
				}
				sb.Append(")");
				logger.Debug(sb);
			}

			invocation.Proceed();
			if(logger.IsDebugEnabled)
			{
				
				logger.Debug("Result of " + sb + " is: " + invocation.ReturnValue);
			}
		}
		catch (Exception e)
		{
			logger.Error(e);
			throw;
		}
	}
}

The code is really simple, we get the logger instance ( in this case, from log4net ), check if logging is enabled and if so, write the method and parameters to the log. We then execute the code and log the return value as well.

If there was an error, we log that as well.

Here is my test code:

var container = new WindsorContainer()
	.Register(
		Component.For<LoggingInterceptor>(),
		Component.For<ISmsSender>().ImplementedBy<SmsSender>()
			.Interceptors(new InterceptorReference(typeof(LoggingInterceptor))).First
	);
BasicConfigurator.Configure(); // configure log4net
var sender = container.Resolve<ISmsSender>();
try
{
	sender.Send("ayende", "short");
	sender.Send("rahien", new string('q', 161));
}
catch (Exception)
{
	
}

And the output for that is:

241 [1] DEBUG ConsoleApplication9.SmsSender (null) - ConsoleApplication9.SmsSender.Int32 Send(System.String, System.String)(ayende, short)
272 [1] DEBUG ConsoleApplication9.SmsSender (null) - Result of ConsoleApplication9.SmsSender.Int32 Send(System.String, System.String)(ayende, short) is: 6
272 [1] DEBUG ConsoleApplication9.SmsSender (null) - ConsoleApplication9.SmsSender.Int32 Send(System.String, System.String)(rahien,
    qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
    qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
    qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq)
273 [1] ERROR ConsoleApplication9.SmsSender (null) - System.ArgumentException: too long
Parameter name: msg
   at ConsoleApplication9.SmsSender.Send(String to, String msg) in C:\Users\ayende\Documents\Visual Studio 2008\Projects\ConsoleApplication9\ConsoleApplication9\ISmsSender.cs:line 15
   at ISmsSenderProxyb24fc36182fb434fa7f466d148259c48.InvocationSend_1.InvokeMethodOnTarget()
   at Castle.DynamicProxy.AbstractInvocation.Proceed() in e:\OSS\Castle\Tools\Castle.DynamicProxy2\Castle.DynamicProxy\AbstractInvocation.cs:line 140
   at ConsoleApplication9.LoggingInterceptor.Intercept(IInvocation invocation) in C:\Users\ayende\Documents\Visual Studio 2008\Projects\ConsoleApplication9\ConsoleApplication9\Program.cs:line 65

Simple, elegant, and doesn't clutter my code.

Why are you showing such a contempt for my time?

This post just went over the line...

image

I mean, how do you expect me to respond to such a message?

In the old tradition of, don't give the man a fish, here is how you find out.

There is this tool called Google. You can s-e-a-r-c-h things there. I used the advance technique of putting the search terms in the text box and clicked on Search.

Here is the result:

image

The third result, surprisingly enough, is from my site. You can see it right there in the link. Clicking on the link will lead you to this page:

 

image

Do you see the big red arrow? It point to a new fangled concept called a link. If you click on it, it takes you somewhere.

Here is where it takes you in this situation:

image

Have fun sharing this with your team, but I suggest taking a Google 101 course at the same time.

Entities dependencies best practices

The question came up in the ALT.Net list, and it is fairly common in the DDD circles as well.

I have an User with Email address, which I want to keep encrypted in the database, but visible to the application. How do I handle this scenario in the entity?

We can do it in our service layer, and encrypt / decrypt that outside the entity, but that has two issues. It couple the entity to the service and it means that we can forget to encrypt the email at some point.It also smells.

Another option, which I call the brute force option, is to pass the dependency to the entity and let it handle its own concerns. Here is implementation:

image

Yes, it is ugly, and even using this approach you can improve upon it, but let us focus on the interesting tidbits. First, not only does our entity know about an infrastructure service, it is also aware of how to create it. It has to do so, because we have to provide an empty ctor for NHibernate to use.

We can generalize this a bit, leading us to this scenario:

image

This piece of code is akin to nails on board to me. It isn't how I want to see code written.

A better way exists. We will start by making a slight modification to the User class, we remove the empty ctor. Well, we can't remove it at the moment, there is check in NHibernate for this, which I need to find and remove, so for now, we have this:

image

Note that this also requires that you'll specify the unsaved value of the identifier in the configuration.

Now that we have established that we have no way of creating the User using the empty ctor, we let NHibernate know how it can create it:

public class EntityDependenciesInterceptor : EmptyInterceptor
{
    public override object Instantiate(string clazz, EntityMode entityMode, object id)
    {
        if(entityMode!=EntityMode.Poco)
            return base.Instantiate(clazz, entityMode, id);
        if(clazz != typeof(User).FullName)
            return base.Instantiate(clazz, entityMode, id);
        return new User(CryptoProvider.Instance);
    }
}

Just for User, we override the way NHibernate create entities. Now we can register it in the session, and we are done:

using (ISession session = sessionFactory.OpenSession(new EntityDependenciesInterceptor()))
{
    IList list = session.CreateCriteria(typeof(User)).List();
}

This is still not something that I particularly like. I don't see encryption as part of the responsibilities of the entity. It is just something we need in order to save encrypted values to the DB. So let make it explicit.

NHibernate has the notion of IUserType, which let you take actions on save / load of values from the DB.

First, we make User POCO again, remove all traces of the previous ugliness. Now, we change the mapping for User so the email property is defined so:

<property name="Email" type="Samples.EncryptedStringUserType, Samples" />		

And now we need to create the user type:

public class EncryptedStringUserType : IUserType
{
    public bool Equals(object x, object y)
    {
        return object.Equals(x, y);
    }

    public int GetHashCode(object x)
    {
        return x.GetHashCode();
    }

    public object NullSafeGet(IDataReader rs, string[] names, object owner)
    {
        object r = rs[names[0]];
        if (r == DBNull.Value)
            return null;
        return CryptoProvider.Instance.Decrypt((string) r);
    }

    public void NullSafeSet(IDbCommand cmd, object value, int index)
    {
        object paramVal = DBNull.Value;
        if (value != null)
            paramVal = CryptoProvider.Instance.Encrypt((string) value);
        IDataParameter parameter = (IDataParameter)cmd.Parameters[index];
        parameter.Value = paramVal;
    }

    public object DeepCopy(object value)
    {
        return value;
    }

    public object Replace(object original, object target, object owner)
    {
        return original;
    }

    public object Assemble(object cached, object owner)
    {
        return cached;
    }

    public object Disassemble(object value)
    {
        return value;
    }

    public SqlType[] SqlTypes
    {
        get { return new SqlType[]{new StringSqlType()}; }
    }

    public Type ReturnedType
    {
        get { return typeof(string); }
    }

    public bool IsMutable
    {
        get { return false; }
    }
}

And that is it, you can now get transparent encryption / decryption when you are going to the DB, at no cost to the application design.

There are some things a good developer SHOULD know

Adi has posted a post that I am not in agreement with: there are some things a good developer is NOT required to know.

[Quoting Jeff Atwood and Peter Norvig] Know how long it takes your computer to execute an instruction, fetch a word from memory (with and without a cache miss), read consecutive words from disk, and seek to a new location on disk

I did learn these things, but after reading Jeff's post I have been trying to remember a project I worked on during the past 10 years which required this type of knowledge - and I got nothing.

Well, I used knowledge about reading & seeking to / from disks the last two weeks. They are extremely important when you are thinking about high perf large I/O. Fetching words from memory, and the implications of ensuring good locality have huge affects on the type of performance that you are getting to get. As a simple example, a large part of the reason that thread switching is slow is that the CPU cache is mostly misses when you switch a thread.

You should know those things, and you should know how to apply them. No, they don't come often, and I can think of a lot more things that I would like a developer to know (OO, to start with) more than those low level details, but those are important to know.

From annoyance to laugher in two seconds

This is a screen shot from NDepend (I have a review forthcoming), I run it on a code base, and I went through the issues it found.

These rules are things that I disagree with.

image

So I clicked to modify it, and I got this, which had me laughing.

image

I like the spirit.

Useless Java

This article (The 'Anti-Java' Professor and the Jobless Programmers) has really annoyed me. Not because of its content, but because the professor being interview is so narrowly focused that he is completely out of it.

Let us take this quote:

“Furthermore, Java is mainly used in Web applications that are mostly fairly trivial,” Dewar says, with his characteristic candor. “If all we do is train students to be able to do simple Web programming in Java, they won't get jobs, since those are the jobs that can be easily outsourced. What we need are software engineers who understand how to build complex systems.”

You know what, I might agree with this, trivial web programming is something that doesn't require highly skilled worker. But the context in which he says this makes all the difference in the world. Here is the next statement:

“By the way Java has almost no presence in such systems. At least as of a few months ago, there was not a single line of safety-critical Java flying in commercial or military aircraft. I recently talked to a customer who had a medium-sized application in Java, which was to be adapted to be part of a safety-critical avionics system. They immediately decided that this meant it would have to be recoded in a suitable language, probably either C or Ada.”

So, if it is not aircraft code, it is "simple Web programming"? Excuse me?!

Leaving aside the part about Java's license not being suitable for life critical systems, it annoys me that this is the only criteria that he think is worth mentioning. His bank is running Java, as well as the stock exchange, it is likely that the payroll system in his university as well. Relegating Java to "simple Web programming" is a huge mistake. Java sweet spot is building complex systems, where a lot of the design decisions that make it hard to do the simple stuff pay off tremendously.

I can think of quite a few mission critical (people life on the balance) that are written in C# (for the purpose of this post, equivalent to Java). That is not to say that aircraft systems shouldn't be written in Ada or C, I never written one, so I won't comment, but to say that this is the only field of any complexity is showing a remarkable amount of ignorance and bigotry.

There was another part that caught my eye, this professor interviewing's technique:

Dewar says that if he were interviewing applicants for a development job, he would quickly eliminate the under-trained by asking the following questions:

1.) You begin to suspect that a problem you are having is due to the compiler generating incorrect code. How would you track this down? How would you prepare a bug report for the compiler vendor? How would you work around the problem?

2.) You begin to suspect that a problem you are having is due to a hardware problem, where the processor is not conforming to its specification. How would you track this down? How would you prepare a bug report for the chip manufacturer, and how would you work around the problem?

“I am afraid I would be met by blank stares from most recent CS graduates, many of whom have never seen assembly or machine language!” he says

I can tell quite a lot about what he is doing, just by the questions he ask, but let me try answering them.

1) Well, I would start by creating an isolated test case, seeing if I can still see the odd behavior. I would remove all code that is even half way smart, trying to reduce the amount of work that the compiler has to do. I would get the language spec and pore over it, trying to make sure that I am not relying on unspecified behavior or misunderstanding the spec. Hopefully, I have a decompiler as well, since that would be an independent corroboration that there is or isn't an issue. Then I would take the output and verify disassemble it, comparing it to what is supposed to happen. I will take that info and submit to the compiler vendor. Working around this issue... by now I have some understanding on what triggers it, so I would change the code accordingly, simplifying it, removing any smart tricks, etc. I'll also leave a big comment about why this is done.

2) Try it on a different machine with the same processor, try it on another machine with a different processor. That rules problems with a specific machine and problems with my code, respectively. Reduce the problem to the smallest thing that can repro it, go over the spec, verify understanding. Eventually I should have a small program that demonstrate the issue. This can be incredibly hard to do if you run into something like memory barriers do not work in some scenario on SMP machines, for example. Work around it... depending on context...

In Belgium Next week

I am going over to Belgium next week, to do some work. Bart Reyserhove is organizing a beers night there.

Where: Sint-Gorikshallen: Sint-Goriksplein 1, 1000 Brussel
When: Tuesday august 5th @ 21:00 hours.

Writing begets writing

I am working on the versioning chapter for the DSL book, and it is going amazingly well. When I started it I had only the vaguest of idea how to go about writing it, but I wrote 10 pages today, and it is going very fast, and the flow seems to be good. What creeps me out is that I spent months thinking about this, without getting any concrete idea about how to express myself, and than it all flow at once.

The problem, of course, is that it doesn't really flow into the book, here is my current blogging queue (unordered):

  1. NHibernate Interception with Post Sharp
  2. Entities dependencies best practices
  3. Creating logging interceptor with Castle Windsor
  4. Evaluating Prism
  5. Usage scenarios for Cloud Computing
  6. Evaluating Kym
  7. Distributed Hash Tables: Locks usage best practices
  8. Distributed Hash Tables: Locality
  9. Distributed Hash Tables: Namespaces
  10. Distributed Hash Tables: Time sensitive updates
  11. Distributed Hash Tables: Lookup by property
  12. Distributed Hash Tables: Lookup by range
  13. Code Critique: Transactional Queue
  14. Rhino Queues
  15. Useless Java
  16. A DSL trip to the Zoo
  17. I'll get to your application in a week- First, we need to build the framework
  18. Production Quality: A DSL Sample

And I am probably forgetting a few along the way. Just getting this turned around would be hard enough, but I also have three webcasts that I need to record & edit:

  • First Steps With Rhino Commons
  • Building Domain Specific Language with Boo
  • Production Quality Software

Way too much on my plate, I am going to bed.

Building Domain Specific Languages in Boo: Chapter 9 TOC

Thoughts?

  • Starting from a stable origin
  • Planning our DSL's versioning story
    • Implications of modifying the DSL Engine
    • Implications of modifying the DSL API and Model
    • Implications of modifying the DSL Syntax
    • Implications of modifying the DSL Environment
  • Regression Test Suite
  • Versioning cut off point - where to apply versioning concerns
  • Versioning strategies:
    • Abandon Ship Strategy
    • Glacial Change Strategy
    • Additive Change Strategy
    • Tower of Babel Strategy
    • Runtime Adapter Strategy
    • Compilation Adapter Strategy
    • Auto Modification Strategy
    • Self Modification Strategy
  • Versioning best practices:
    # actual content
    • Closed world
    • Regression test suite

Onion Architecture

image This post from Jeffrey Palermo has managed to distilled the reasons for a lot of things that I consider important.

From persistence ignorance to dependency injection.

To be rather more exact, Jeffrey has managed to put it in words in a way that I don't think that I would have ever could.

Go and read it.

The concepts will probably not be earth shattering, but the ability to communicate clearly about this is very important.

 

 

 

 

.

A DSL is also its usage

The title says it all, I think. Okay, it doesn't, I admit.

There is a lot of focus with DSL about the syntax. And there is some focus on the engine. There is very little focus about how both the environment and the usage of the DSL affect the DSL itself. Here are a few examples:

  • Naming convention
  • Script ordering
  • Execution location (when you are running the scripts)

All of those are of particular importance in many DSL, not only for the actual execution, but for how the DSL itself is written.

Not all bytes weight exactly 8 bits

Or, pay attention to how you write to the disk. Here is a simple example:

static void Main(string[] args)
{
	var count = 10000000;

	Stopwatch stopwatch = Stopwatch.StartNew();

	using (var stream = CreateWriter())
	using (var bw = new BinaryWriter(stream))
	{
		for (var i = 0; i < count; i++)
		{
			bw.Write(i);
		}
		bw.Flush();
	}
	stopwatch.Stop();
	Console.WriteLine("Binary Writer: " + stopwatch.ElapsedMilliseconds);

	stopwatch = Stopwatch.StartNew();

	using (var stream = CreateWriter())
	{
		for (var i = 0; i < count; i++)
		{
			var bytes = BitConverter.GetBytes(i);
			stream.Write(bytes, 0, 4);
		}
		stream.Flush();
	}
	stopwatch.Stop();
	Console.WriteLine("BitConverter: " + stopwatch.ElapsedMilliseconds);


	stopwatch = Stopwatch.StartNew();

	using (var stream = CreateWriter())
	using (var ms = new MemoryStream())
	{
		for (var i = 0; i < count; i++)
		{
			var bytes = BitConverter.GetBytes(i);
			ms.Write(bytes, 0, 4);

		}
		var array = ms.ToArray();
		stream.Write(array, 0, array.Length);
		stream.Flush();
	}
	stopwatch.Stop();
	Console.WriteLine("Memory stream: " + stopwatch.ElapsedMilliseconds);


	stopwatch = Stopwatch.StartNew(); 
using (var stream = CreateWriter()) { byte[] buffer = new byte[sizeof(int) * count]; int index = 0; for (var i = 0; i < count; i++) { buffer[index++] = (byte)i; buffer[index++] = (byte)(i >> 8); buffer[index++] = (byte)(i >> 16); buffer[index++] = (byte)(i >> 24); } stream.Write(buffer, 0, buffer.Length); stream.Flush(); } stopwatch.Stop(); Console.WriteLine("Single buffer: " + stopwatch.ElapsedMilliseconds); } private static FileStream CreateWriter() { return new FileStream(Path.GetTempFileName(), FileMode.Create, FileAccess.Write, FileShare.Read, 0x10000, FileOptions.SequentialScan | FileOptions.WriteThrough); }

And the results:

Binary Writer: 1877
BitConverter: 1985
Memory stream: 1702
Single buffer: 1022

Not all objects are created equals

I found something extremely surprising while profiling a project. Take a look at this piece of code:

Stopwatch stop = Stopwatch.StartNew();
for (int i = 0; i < 1000000; i++)
{
	new WeakReference(null);
}
stop.Stop();
Console.WriteLine("WeakRef: " + stop.ElapsedMilliseconds);

stop = Stopwatch.StartNew();
for (int i = 0; i < 1000000; i++)
{
	new string('a', 5);
}
stop.Stop();
Console.WriteLine("'aaaaa': " + stop.ElapsedMilliseconds);

On my machine, this has the following output:

WeakRef: 980
'aaaaa': 35

Creating a WeakReference is much more costly than creating a normal object. Not surprising, when you think of it, WeakReference has deep ties to the CLR, but I couldn't really believe it when I saw it the first time.

Some finalizer pitfalls

The use of finalizer to ensure eventual release of resources is recommended, but one thing that is not discussed often is the implications of adding a finalizer.

No, I am not talking about the extra wait time before the class memory is released, this is fairly well known. I am talking about two major additional issues that don't get enough attention, in my opinion.

Specifically, invalid instances and threading.

An object that has a finalizer will may be executed in more than one thread. Luckily, the CLR ensure that our finalizer will not be able to run if there is any code that is using the class, so it is not as bad as it seems at first.

However, something that make absolute sense was a bit of a surprise to me, when I figured it out, finalizers will run for invalid instances as well. What do I mean by that? Invalid instance is an object whose ctor threw an exception. It is in invalid state.

But, before it threw an exception, it might have acquired resources that needs to be released, so it goes to the GC finalizer as well. In fact, since you can't call Dispose on such an object, it must go to the finalizers for cleanup.

Remember the invalid state bit part? This is important!

Your object is in invalid state, and it is asked to cleanup itself. If it can't do so even in invalid state, an exception will be thrown. This exception on the finalizer thread will kill the application.

Here is the proof of concept code:

public class BadClass : IDisposable
{
	FileStream fs;

	public BadClass(int i)
	{
		if (i%2 == 0)
			throw new InvalidDataException("evens are bad for your health");
		fs = new FileStream(Path.GetTempFileName(), FileMode.Create);
	}

	~BadClass()
	{
		fs.Dispose();
	}

	public void Dispose()
	{
		fs.Dispose();
		GC.SuppressFinalize(this);
	}
}

And the code to execute it:

for (int i = 0; i < 1000; i++)
{
	try
	{
		using (new BadClass(i))
		{

		}
	}
	catch (Exception)
	{
	}
}

Trying to figure out what is going on in this scenario is tough. Going back to the threaded aspect of it. What this means is that such an object is a time bomb, and it will blow up at some future date to kill your application, but you are not likely to know what exactly caused it to be this way.

In my situation, I figured it out by putting new StackTrace(true) in the constructor of the object, and breaking into the debugger on the finalizer.  But until it occurred to me to look for it there...

Challenge: What does this code do?

Without compiling this, can you answer me whatever this piece of code will compile? And if so, what does it do?

var dummyVariable1 = 1;
var dummyVariable2 = 3;
var a = dummyVariable1
+-+-+-+-+ + + + + + +-+-+-+-+-+
dummyVariable2;

Oh, and I want to hear reasons, too.

Code Critique: Transactional File

Update: To clarify, the only purpose of this code is to give atomic writes, it either complete successfully or it is rolled back.

Okay, this looks like it is too simple to work, thoughts?

/// <summary>
/// Allow to overwrite a file in a transactional manner.
/// That is, either we completely succeed or completely fail in writing to the file.
/// Read will correct previous failed transaction if a previous write has failed.
/// Assumptions:
///  * You want to always rewrite the file, rathar than edit it.
///  * The underlying file system has at least transactional metadata.
///  * Thread safety is provided by the calling code.
/// 
/// Write implementation:
///  - Rename file to "[file].old_copy" (overwrite if needed
///  - Create new file stream with the file name and pass it to the client
///  - After client is done, flush & close the stream
///  - Delete old file
/// 
/// Read implementation:
///  - If old file exists, remove new file and rename old file
/// 
/// </summary>
public static class TransactionalFile
{
	public static void Read(string name, Action<Stream> action)
	{
		if (File.Exists(name + ".old_copy"))
		{
			File.Delete(name);
			File.Move(name + ".old_copy", name);
		}

		using (
			var stream = new FileStream(name, FileMode.Open, FileAccess.Read, FileShare.None, 0x1000,
										FileOptions.SequentialScan))
		{
			action(stream);
		}
	}

	public static void Write(string name, Action<Stream> action)
	{
		File.Delete(name + ".old_copy");

		if (File.Exists(name))
			File.Move(name, name + ".old_copy");

		using (
			var stream = new FileStream(name, FileMode.CreateNew, FileAccess.Write, FileShare.None, 0x1000,
			                            FileOptions.WriteThrough | FileOptions.SequentialScan))
		{
			action(stream);
			stream.Flush();
		}

		File.Delete(name + ".old_copy");
	}
}

The exception I hate the most

And just for fun, it is reproducable if you run 824 tests, but not if you run a single test!

image

Update: Found the guilty test, and it is one that used to work, damn it!

High performant way for waking REALLY fast

Precondition: Extremely sleepy.

Solution: Go to a coffee shop, order their strongest coffee, add lots of sugar. Go to the table and take a sip. Spill the coffee on top of the hideously expensive MacBook Pro.

Result: Instant wakefulness.

The MBP seems fine, and I am using it to type this, but that was a big scare.

The Sewer Architecture

imageOn Friday, I had an encounter with the sewers at my house. On Call team has came and fixed the issue, which was a plugged pipe somewhere down the street very fast, so no harm was done (although checking the sewers just after taking a shower is not my idea of fun).

That did, however, let me learn a bit about the way sewers are constructed.

Broadly, the sewer is based on the idea of pipes and pools ( at least, those are the Hebrew terms, translated ). The idea is that the sewer flows in the pipes, toward where ever it goes. But if we have the pipes, why do we need the pools as well?

The pools are there for capacity control. They help to ensure that even if there is increase demand for the sewer, it won't back fire on you (literally, yuck!) when used.

If the downstream pipe cannot accept the sewer that we have locally, it is gathered in the pool, and will trickle downstream based on whatever "bandwidth" it will get.

Usually, you have several such pools, each of which can take quite a bit of sewer water, so you tend not to have issues with the sewer unless it is a real problem. Reduced capacity for the sewer is not noticed, because the pools absorbe the spike in usage and moderate it.

In my house, we had the following issue:

image

 

Three pools down from the actual drain (about 100 meters from my house, actually), we finally found the culprit, a semi plugged pipe that wasn't draining properly. The interesting thing here is that it is likely that it wasn't functioning correctly for a long time (weeks, at least).

Now, last I checked, this blog was mainly technical stuff, and while I am sure that there are Sewer Geek somewhere, I am not part of that particular subculture. Why am I telling you that?

Because there are a lot of parallels with software. Specifically, with building software that can withstand the harsh environment called production. The idea of pools for the sewers closely match the idea of queued messaging for communication, for example.

And I have seen application survive any number of transient failures as a result of this type of architecture.

So, this was a trip in the sewers, hope you held your noses.

No testing, no software

I just tried to spike something, and as usual, I created a console app and started hacking.

It is a non trivial spike, so I started refactoring it to allow proper separation and actually let me handle the complexity that I am trying to contain.

I couldn't continue the spike. Literally. I had no idea how to go about it.

I am currently in the process of moving the spike code into a proper environment, one that has tests, so I can actually work in small increment, and not try to implement the whole thing in a single go.

About an hour later, I have this mostly complete and working, and I can see how the tests helped me get into a situation where I can actually make a small set of changes are get things working.

Mocking the file system

I have the following piece of code:

image

And I wasn't sure about how I can test this thing. For production, this will use a File Stream, and getting those to produce errors is non trivial in most situations. A bit of thinking, however, told me that I can write a test for the error handling section of this fairly easily, like this:

 

image

I love having powerful tools at my reach.