Design
Using Lucene – External Indexes
Lucene is a document indexing engine, that is its sole goal, and it does so beautifully. The interesting bit about using Lucene is that it probably wouldn’t be your main data store, but it is likely to be an important piece of your architecture. The major shift in thinking with Lucene is that while indexing is relatively expensive, querying is free (well, not really, but you get my drift). Compare that to a relational database, where it is usually the inserts that are cheap, but queries are usually what cause us issues. RDBMS are also very good in giving...
Black box reverse engineering speculation
Terrance has pointed me to some really interesting feature in Solr, called Facets. After reading the documentation, I am going to try and guess how this is implemented, based on my understanding of how Lucene works. But first, let me explain what Facets are, Facets are a way to break down a search result in a way that would give the user more meaningful results. I haven’t looked at the code, and haven’t read any further than that single link, but i think that I can safely extrapolate from that. I mean, the worst case that could happen is...
Nice process, but what about the engineering bits?
Software processes has always been a popular topic of discussion in our industry. Those can get quite heated, with advocates of the “stable / stale” Waterfall method pointing fingers toward “rapid / rabid” Agile methods, with the CMMI people throwing documents around and Lean people standing on the sidelines muttering about Waste. This isn’t a post about a specific software process, I’ll defer that to another day. Instead, I want to focus on a flaw in the basic building blocks in many* software building processes. They ignore the actual building the software. ...
It is less expensive to do it inefficiently!
This is a continuation of a twitter conversation that I had with Karl Seguin. One of the problems for developers is that we tend to have a hard time distinguishing between the right thing from a technical perspective and the right thing from a business perspective. One of the known issues with the profiler from the very start was the how to handle large amount of data. The profiler use to keep all data in-memory, which put a hard limit to how much data it can...
Encapsulation is the enemy of the user interface
I got this question a while ago from Kyle, and I think is is a great one. It is especially great since it is an exchange of emails that resulted in the following (all of which are Kyle words): I've been annoyed lately by the MVVM pattern. It seems like it requires that the data on your business classes be public so that the view-model can get at it, and that completely breaks encapsulation and goes against standard OO design theory (in my opinion). The UI layer should be allowed to reference...
Rejecting Dependency Injection Inversion
Uncle Bob has a post about why you should limit your use of IoC containers. I read that post with something very close to trepidation, because the first example that I saw told me a lot about the underlying assumptions made when this post was written. Just to give you an idea about how many problems there are with this example when you want to talk about IoC in general, I made a small (albeit incomplete) list: The example is a class that has two dependencies, who themselves has no dependencies. There...
When the design violates the principle of least surprise, you don’t close it as By Design
I don’t actually have an opinion about the actual feature, but I felt that I just have to comment on this post, from Brad Wilson, about the [Required] attribute in ASP.Net MVC 2. Approximately once every 21.12 seconds, someone will ask this question on the ASP.NET MVC forums … The answer is the title of this blog post. ([Required] Doesn’t Mean What You Think It Does) If this is the case, I have to say that the design of [Required] is misleading, and should be change to match...
Time transitions should be explicit
Let us talk about time for a second, okay? We deal with in just about every application we write, but we treat it quite dismissively. But let me give an example first. We need to build a notification system, the system is based on timed notifications that should be displayed in a web page. Thinking about it, I came up with the following design: And this query: SELECT TOP 3 Id, PublishAt, Title, Content FROM Notifications
WHERE PublishAt > GETDATE()
ORDER BY PublishAt DESC
Using a service bus for queries
Yep, another forum question. Unfortunately, in this case all I have is the title. Even more unfortunately, I already used the stripper metaphor before. There are some questions that I am really not sure how to answer, because there are several underlying premises that are flat out wrong in the mere asking of the question. “Can you design a square wheel carriage?” is a good example of that, and using a service bus for queries is another. The short answer is that you don’t do that. The longer answer is that you still don’t do...
Message passing concurrency and shared state
Mike Rettig has left a somewhat snarky comment on a post detailing a deadlock issue that I run into: Locking on shared state? I thought you were a proponent of message based concurrency. This post demonstrates exactly why concurrency combined with shared state is so hard. Looking forward to your next thread race or deadlock, The problem with message passing concurrency is that the underlying assumption here is that there isn’t any shared state. But in my situation, that is no a valid assumption. Let us see if I...
How to lead a convoy to safety
I recently run into a convoy situation in NH Prof. Under sustained heavy load (not a realistic scenario for NH Prof), something very annoying would happen. Messages would stream in from the profiled application faster than NH Prof could process them. The term that I use for this is Convoy. It is generally bad news. With NH Prof specifically, it meant that it would consume larger and larger amounts of memory, as messages waiting to be processed queued up faster than NH Prof could handle them. NH Prof uses the following abstraction to...
Two strikes, and you are out
I don’t have a lot of patience for repeated bugs. I just got a bug report that turn out to be in the result of the same feature as a previous bug I fixed. I could have fixed the bug. But I didn’t bother. A repeated bug in the same area for the same reason usually indicate a fragile design. In this particular case, the feature wasn’t important, so I just ripped it all out, root & branch. If it was important, I would have still ripped the whole thing apart, and then I would rebuild it from...
Soft Deletes aren’t Append Only model
There seems to be some confusion regarding my post about soft deletes, in particular, people brought up the idea of append only models. I had the chance to work on both types of systems, and I can tell you that I would much rather work with append only model than with soft deletes. The append only model means that you can only ever insert, never delete or update. Thing about the way your bank account works. If I had the clerk transfer money from one account to another, and he had a typo and send tens times the...
Ayende’s Razor
This is a response to a comment on another post: Oren, in all seriousness, I thought that problems that were "(a) complex, (b) hard to understand (c) hard to optimize" were the kinds that folks like you and I get paid to solve... Given two solutions that match the requirements of the problem, the simpler one is the better.
Concepts & Features: A concept cover the whole range, a feature is constrained
A while ago I mentioned the idea of Concepts and Features, I expounded it a bit more in the Feature by Feature post. Concepts and Features is the logical result of applying the Open Closed and Single Responsibility Principles. It boils down to a single requirement: A feature creation may not involve any design activity. Please read the original post for details about how this is actually handled. And a the specific example about filtering with NH Prof. But while I put constraints on what a feature is, I haven’t talked about what...
Concepts & Features in NH Prof: Filtering
A while ago I mentioned the idea of Concepts and Features, I expounded it a bit more in the Feature by Feature post. Concepts and Features is the logical result of applying the Open Closed and Single Responsibility Principles. It boils down to a single requirement: A feature creation may not involve any design activity. Please read the original post for details about how this is actually handled. In this case, I wanted to show you how this works in practice with NH Prof. Filtering in NH Prof is a new concept, which allows...
A design question: What do you do with inaccurate data?
I intentionally don’t intend to give out enough information about this problem, I want to see what your opinion is. I have an application where a certain action invalidate some of the data that the user is shown. It is quite expensive to recalculate that data, so we can’t just recalculate it right then and there, and in many cases, it will be the exact same data as the user is currently shown. The question is, what should we do with this data? Ignore the invalidation and just show the (possibly invalid) data to...
Virtually everything
This is a short reply to Ward Bell’s post, Do Not Make Every Method Virtual. More specifically, I have an issue with this statement: My argument is with opening up the class blindly and totally by making everything virtual. Suddenly nothing in the class is truly “closed for modification.” The “virtual” keyword announces to the world “here is my extension point.” When every method is virtual, the world is invited to change every method. I think that there is some confusion here. The original class is closed for modification. I can’t change it. I...
Taking advantage on the Data Transfer Tier
When talking about RIA applications, we usually have the following physical architecture: That is, we have the application actually running on the client’s machine, we access the host server in order to perform operations that cannot be made locally and we save to some persistent storage, usually an RDBMS. The great advantage that in a RIA application, we have a real platform at the client side, and not the hack that is HTML + JavaScript. That make it a much more pleasant experience to actually work with them. However, we then encounter a very interesting...
The Least Common Denominator approach
I was talking with a team about their project, and doing a very minor code review. Naturally, one of the things that I checked first is the data access portion of the application. I looked at the data access code in a sense of shock. I was on a phone call with the team, with one of them speaking and I interrupted him to say something along the lines of: “Guys, the nineties called and they want their handed rolled data access back!” Just to give you an idea, when I am saying that the code has...
Let the fighting commence!
Another interesting problem that I run into in Jamie Farser’s blog, he is building a game and run into a problem designing the object model for it. The problem can be sum up pretty easily in two pictures: And here is what Jamie has to say about it: Each interface defines several methods or properties (for example, Attack or AttackRanged). However this feels a bit wrong to me - I'd ideally like just a single Attack method, which would then do something to determine whether the unit was capable of ranged attack ...
Watermarking and Security
I was reading Jamie Fraser’s blog when I run into this post. In the post, Jamie talks about how he want to make it to fake data that the user is able to print from a site. The example given is a payslip, which we don’t want the user to be able to successfully change. The solution that Jamie came up with is to watermark the image. His example is: Unmodified Watermarked ...
OR/M suitability
A few days ago, I run into this tweet: My answer to that is composed of two parts. People usually think that I am being facetious when they ask me when they should use NHibernate, and I answer: “Whenever you are using a relational database”. I am not, I am quite serious in that regard. Well, there is a small subtext here in which I assume that you are talking about OLTP rather than reporting or other BI activities. The reason for that is quite simple, NHibernate (and other OR/Ms, I guess) is doing...
Infrastructure is king: The To Do app
I spent the last few days writing the NHibernate in Desktop Apps sample. Since I didn’t feel like doing something adventurous, I choose to use the age old ToDo app as the sample. Now that I have finished it, I run some stats on it, and it is… interesting. Here is the number of lines split into infrastructure and application code: The funny part about this is that this is for an application that has three forms altogether and very little behavior. The not so funny part is that I fully expect...
Your domain model isn’t in the Entity Relationship Diagram
As usual, reading Udi’s articles is an interesting experience, and his latest one, Employing the Domain Model Pattern is no exception. I am actually a proponent about thinking twice before approaching a domain model architecture, but not because I see it as a way to manage change. I see it as a way to manage complexity, and you need a fairly high threshold of complexity before the domain model pattern starts to justify itself. And I enthusiastically agree about the domain events model. But probably the most important topic raised in the article is that when...
Feature by feature
I commented before that a feature is usually composed of several classes, running in different places and at different times, all working together to achieve a single goal. This post is meant to expand on this notion a bit. Let us go back to the type of architecture that I favor right now. Note that this is a physical diagram only, without actually going into the details about how each of those is implemented. Storage, for example, may be a relational database, a key value store, a distributed hash table or something else, depending on ...
The Tale of the Lazy Architect
So, there is the interesting debate in the comments of this post, and I thought that it might be a good time to talk again about the way I approach building and architecting systems. Let me tell you a story about a system I worked on. It started as a Big Project for a Big Company, and pretty much snowballed into getting more and more features and requirements as time went by. I started up as the architect and team lead for this project, and I still consider it to be one of the best projects that I worked...
Don’t castrate your architecture
Note: I thought about trying to find a more PC title for this post, but I don’t think that I can really find a good one that express the same emotional content and the punch that this title have. A few weeks ago I got my first interesting cold call (someone calling me from the number in the blog without some previous acquaintance), cold calling doesn’t happen very often (maybe 5 – 7 times in the 5 years I had the number there), but that was the first time that I actually got to talk...
Avoid externalizing decisions from your domain model
I recently saw an entity that looked something like this: public class Prisoner
{
public virtual bool CanBePutInIsolation() { ... }
public virtual bool IsEligibleForVacations() { ... }
public virtual bool CanSendToWork() { ... }
}
I won’t show the logic in those methods, but it was fairly involved and very business focused. It is also, incidentally, caused my rhino sense to tingle. It didn’t surprise me to find out that the UI looked like this:
Well, the logic is well encapsulated, and...
Analyzing a performance problem – Is a prisoner dangerous?
Recently I run into a performance problem in an application that I was reviewing, and I thought that it would make a great post. Since I can’t use the actual application model, I decided that I am tired of using the same old online shop model and turned to the one domain in which I am a domain expert. Prisons. Let us imagine that this is part of the Prison 10’s* Dashboard. It looks pretty simple, right? Let us talk about this as SQL, ignoring all layers in the middle. We can express this as:...
Can you justify your actions with regards to this pattern?
Probably the most common problem with unintended complexity in software is the unthinking application of patterns. I see a lot more code that has became more complex through that practice than code that has became simpler. David Wheeler said: “All problems in computer science can be solved by another level of indirection;” However, we tend to forget the extra bit that Kevlin Henney's tacked at the end, "...except for the problem of too many layers of indirection." Most patterns templates come with a clearly defined sections for “when to use” and “what are the disadvantages”....
Who is the responsible party?
Object Oriented design is all about placing behavior and data together, of making the object responsible for its own operations. The problem then becomes identifying which object is responsible for what. This is usually pretty easy, since the natural owner of an operation is usually clear. Note that I don’t much care for the choice people make between things like parent.AddChild(child) vs. child.AddTo(parent). Both are equally good from my point of view. What I don’t want to see is this: parent.Children.Add(child); child.Parent = parent; The reason for that is quite...
The Stripper Pattern
In 2006 (if I can recall correctly), Greg Young and I discussed the concept of the Stripper Pattern, we never really had the chance to formulate it into a real pattern, but we kept discussing this whenever we met (and had beer available). This post is about the stripper pattern, and there is a good reason why I selected the accompanying image for this post, it is the spookiest stripper image that I can put in the blog. Because the Stripper Pattern is an anti pattern. It all started from a fairly innocent question, asked in...
How design rot
I am currently doing some big modifications on the NH Prof code base, getting it ready for v1.0 status (and reserving some surprises as well). Among other things, it means that I am trying to touch as much of the code base as possible. It also means that I am finding stuff that I generally don’t see when using the application normally. Here is a good example: The internals of NH Prof are built on the notion of pub/sub, mostly because I wanted to be able to have good separation...
The buddy classes are drowning DRY
The notion of buddy classes has appeared at first in ASP.Net Dynamic Data, and now has made its way to RIA services. I don’t really bother to keep track with all the new tech from Microsoft, but this post came to my attention. What is a buddy class? Let us take this example as an example, which was generate by a tool: public partial class MyClass
{
public int MyProperty { get; set; }
}
And say that we need to add some attribute to that, to...
Let us burn all those pesky Util & Common libraries
This is a post that is riling against things like Rhino Commons, MyCompany.Util, YourCompany.Shared. The reason for that, and the reason that I am not longer making direct use of Rhino Commons in my new projects is quite simple. Cohesion: In computer programming, cohesion is a measure of how strongly-related and focused the various responsibilities of a software module are. Cohesion is an ordinal type of measurementand is usually expressed as "high cohesion" or "low cohesion" when being discussed. Modules with high cohesion tend to be preferable because high cohesion is...
The Repository’s Daughter
Keeping up with the undead theme, this post is a response to Greg’s. I’ll just jump into the parts that I disagree with: The boundary is not arbitrary or artificial. The boundary comes back to the reasons we were actually creating a domain model in the first place. it seems what Oren is actually arguing against is not whether “advances in ORMs” have changed things but that he questions the isolation at all. The whole point of the separation is to remove such details from our thinking when we deal with the domain and to make...
Night of the living Repositories
This is a response to Greg’s post in reply to mine. I can’t recall the last time I had a blog debate, they are usually fun to have. I am going to comment on Greg’s post as I read it. So this part is written before I did anything but skim the first paragraph or so. A few things that I want to clarify: His post was originally intended to be an “alternative to the repository pattern” which he believes “is dead”. Of course this...
Where is the ROI?
Justin has a post called: Keep your IQueryable in check, in which he basically says that while he finds value in the Query Object pattern that I recently talked about, he is concerned by the fact that I am returning an IQueryable (or ICriteria or IQuery) from the query object. The problem is that the UI layer can then modify the query in potentially undesirable ways. The example that he gave is: latePayingCustomQuery
.Where(c => c.LastName.StartsWith("E"))
.Skip((page - 1) * itemCount)
.Take(10);
Which perform a query...
The DAL should go all the way to UI
My recent post caused quite a few comments, many of them about two major topics. The first is the mockability of my approach and the second is regarding the separation of layers. This post is about the second concern. A typical application architecture usually looks like this: This venerable structure is almost sacred for many people. It is also, incidentally, wrong. The main problem is that the data access concerns don’t end up in the business layer. There are presentation concerns that affect that as well. Let us take a look...
Designing Rhino DHT - A fault tolerant, dynamically distributed, hash table
of the more interesting requirements that came my way recently was to redesign Rhino DHT to support dynamically distributed network of nodes. Currently, the DHT support a static network layout, and while it can handle node failure (and recovery) via replication, it cannot handle node additions on the fly. That is a problem. But another problem is that the configuration for the DHT is getting complex very fast. And we start putting more and more requirements on top of it. This post is going to articulate the requirements we have for the DHT, how the current design for solving...
Application structure: Concepts & Features
I spoke about this topic quite a bit in ALT.Net Seattle. This is mostly relating to application architecture and how you structure your application. This is the logical result of applying the Open Closed and Single Responsibility Principles. Feature may not be as overloaded a term in our industry as a service, but it still important to define exactly what I mean. A feature, in my eyes, is a discrete part of the application that perform some operation. That is pretty vague, I know, but I hope that the examples I am going to go through later would make...
“Just don’t do that” is an acceptable answer
Recently I found myself facing several pretty tough problems. Solving the problem or the generic case would have been hard if not impossible. In all of those cases, I was able to redefine the problem to make the solution trivially simple. Problem #1 – SQL display in NH Prof We had an issue with NH Prof regarding scrolling a big list. The problem was that the performance for big lists isn’t that great. WPF has a solution for that, virtual lists, which means that we only get to bind to the visible portion of the list, which significantly...
Opinionated API
Recently I found myself doing a lot more API design that use the following method. A method that takes a delegate which accepts a parameter that allow you to perform the actual operations that are required. This is very good for things like handling semantics, and since we only allow access to the desired operation through the parameter to the delegate, we can be pretty sure that we will have a consistent operation experience. Let us take the following example: ...
Social Engineering in Software Development: Imitating Actions
The absolutely hardest thing in most projects is to (and I am going to be blunt), to get the customer to shut up and let the team start working. I have been in several projects where the actual move from blah blah talking to actual doing was a painful and drawn out process. In one particular example, I was brought on as a team lead for a team that had 4 months to produce a system. I was stunned to discover that the last two years were spent in meetings on top of meetings on top of meetings. At some...
Fighting the architecture astronaut
It is surprising how many other things I have to deal with in order to get a product up and running. Right now I am working on the web site for NH Prof, and I found myself tackling a problem in an interesting way. The problem itself is pretty simple. I need to show a user guide, so people can lookup help topics, view alerts, etc. My first instinct was to create a HelpTopics table and do a simple data driven help site. Literally just showing a list of the topics and allowing to view them. I got that working,...
Discussion: OO 101 Solutions and the Open Close Principle at the architecture level
This post is based on the transcript of a conversation between me and Luke Breuer, regarding the application of the Open Close Principle to the system architecture. The quoted sections are Luke's, and my own responses are shown underneath them. This is a different style from my usual posts, because this is a conversation, which was transcriptted and only slighted edited. I hope that this would help all those who wanted to know more about what I meant about using OPC at the architecture level and how to structure systems where you only ever add new things. Let us get...
Architecture Advice: Scale the architecture to fit the application
I was just reviewing an application that was obviously built upon a lot of the best practices advice for using an NHibernate application. I am currently in the process of ripping apart much of that application and then putting it back together again. This involve things like collapsing projects, removing abstractions and deleting tests. The main reason for that is quite simple, the architecture is too big for the application. We can create a much more stream lined application is we don't burden it with an architecture that is suitable for bigger and more complex applications. Selecting an inappropriate architecture...
My baseline ASP.Net MVC modifications
We start by killing ViewData. I don't want it, and I want to fail hard if someone is trying to use it: Likewise in the views, I hate ViewData.Model, so I remove it from there as well: For Javascript, I don't want to use <script/> tags, they are prone to path problems, and I might need to go back and change them (using compression, combining scripts, etc). I use this approach: The page title is set by the view, the way it should: Or, I just set it up in the master...
Do not put presentation concerns in the controllers!
I thought it would be obvious, but I am currently cleaning up an application that has mixed them in a very annoying way. I decided to track the reason for this coupling and I found this on the sample controller in ASP.Net MVC: I am pretty sure that setting the page title is a presentation concern, as such, I don't want to see this in the controller, and I certainly don't want to see it in the base sample from which everyone is going to start.
When your extensibility strategy is OOD...
You get to have really simple solutions. One of the reasons that I like NHibernate so much is that is allow me to use Object Oriented solutions to my problems. Case in point, we have the Rhino Security library, which provide a facility for asking security questions about your domain. Bart had an issue with Rhino Security, he wanted to extend the library to also contain a type. The original idea was to add a int field called AppSpecific, which will let each app define additional information on top of the existing domain model. That made me feel so Win32...
Accessibility Concerns
Kelly Stuard called me out with regards to the NH Prof user interface: I'd just like to weigh in and say that I am completely against the WPF-murder of this application's UI. There are several hundred reasons to go away from the OS's colors and standard dialogs; I do not see a single one here. In choosing to shove your choices upon your users you are potentially alienating people who may need a particular color contrast or text size(color blind / low vision) or need to only use the keyboard (blind with reader). It looks very sexy, I will...
The database as a service anti pattern
A while ago I stated: The problem is supporting change tracking in disconnected scenarios. In particular, you take an object from the database and send it to the presentation layer, some time later, you get the object from the presentation layer and persist it to the database again. There is a whole host of bad practices and really bad design decisions that are implicit in the problem statement, but we will leave that alone for now. I was asked what I meant by that. I think that just the title of this post should suffice as an answer,...
Thinking with blinders
Many of the comments to my previous post were some variation on this one: Have you consider what would happen if they actually followed your advice?Do you really want to rewrite major parts of your application every time a new version comes out? I carefully re-read my post, and I couldn't find anything there to suggest that to me in the post. What I suggested is relaxing the mandate of absolute backward compatibility, which cause so many problems in both design and maintenance modes. I never suggested that there should be no compatibility, and I certainly didn't suggest "let...
Shopping Cart sample design considerations
While it is highly likely that you have created a shopping cart in the past, this shopping cart design is slightly different. One thing that should be obvious is that this is still a simplified example, and in the real world you are likely to need to do more, but it is a good example of an important concept. In the image, you can see that we have the fairly standard Shopping Cart, Product and Item classes, where item represent the quantity of a particular product in the shopping cart. But why do we need such...
Designing a shopping cart
I needed to create a sample shopping cart, and this is what I came up with. Thoughts?
Aspects of Domain Design
On of the things that I like most about conference is that they are the place to meet and exchange a whole boat load of ideas with a large number of people. Most of the time, this is actually done on a normal level, just far more frequently than usually happen. Just this is a reason enough to visit conferences. Sometimes, however, you get into a conversation that really open your mind. That is what Aslam Khan managed to do to me in about five minutes of hallway conversation. This short talk literally opened up for me a whole...
Why NH Prof isn't functional
One of the things that I wanted to do with NH Prof is to build it in a way that would be very close to the Erlang way. That is, functional, immutable message passing. After spending some time trying to do this, I backed off, and used mutable OO with message passing.
The reason for that is quite simple. State.
Erlang can get away with being functional language with immutable state because it has a framework that manages that state around, and allow you to replace your state all the time. With C#, while I can create immutable data structures, if I...
The fallacy of IRepository
I didn't have a controversial title in a few days :-) The cause for this post is a post by Rob Conery where he suggests removing the RDMBS from the equation, at least during development. In particular, he suggests using an OODB during development and switching to RDBMS late in the game, when going to production. The reason for doing so in order to reduce the friction of having to maintain a database during development. One thing that I feel that I have to point out. DB is a friction point in one of several conditions. If your tool...
What questions should you ask at the beginning of a project?
Regardless of the actual project, I usually ask the following questions. Scalability Requirements - How many users are we expecting? Expressed as users per day. As part of that, however, we also need to consider spikes in traffic, and if we need to handle them. Distribution Requirements - How many data centers are we going to run on? How many machines?Numbers I want to hear: 1, a few, lots. Security Requirements Authorization Requirements - Role based? Data driven? Dynamic? Rule based? Sensitive data - Do we store any? If so, how secure do...
How NHibernate forced me to OO design
At some point of my code, I had this method. Remember, this is spike code, so I didn't care for breaking things like encapsulation or getting ugly code. In essence, the code above is breaking encapsulation and stomps all over DRY and Liskov substitution principle. It worked for a while, then it stopped. The reason that it stopped is that I started passing it lazy instances. And a lazy instance is a proxy, which is neither an ImageItem or HtmlItem. So now ContentItem has the following: As an aside, notice the RecordApperances method. That is the...
How do you test a system that self optimize itself?
I have a system in which one of the core parts is required to work in a non deterministic fashion. In particular, some part of the system is behaving randomly (by design). Here is a small description: 90% of the time, we select only items that are above the score limit. Selection between those is made randomly, with no bias 10% of the time, we select only items that are _below_ the score limit. Selection among those is done based on the oldest last shown date To skip "rotten apples", we calculate the median of the...
Zones of Quality
In most of my applications, I tend to work according to a method that I call zones of quality. Case in point, I am currently writing an application that does [doesn't really matter]. This application needs data entry screens to get data in, but while data entry is a big and complex topic, it is not something that is even remotely related to the real business value in the system. For that matter, a lot of the domain logic of the system is secondary and only there as a supporting role. So I give them both the exact same...
Messaging Concepts - The Starbucks example
During KaizenConf, we discussed the notion of messaging as a first level concept quite a bit. I have already spoken about this at length, but I think that there is still a lot of room to talk about actual implementation and architecture details.
The example that was chosen as the reference implementation for Mass Transit reference implementation was Starbucks. This is a concept that should be familiar to most programmers, while at the same time offering us a good way of exploring quite a few messaging concepts.
The scenario in the example is simple, I want to order a venti hot chocolate....
When select IS broken (or just slow)
Usually, "select" isn't broken is a good motto to follow. Occasionally, there are cases where this is case. In particular, it may not be that it is broken, it may very well be that the way it works doesn't match the things that we need it to do.
I spoke about an optimization story that happened recently, in which we managed to reduce the average time from 5 - 10 seconds to 5 - 15 milliseconds.
What we needed was to walk a tree structure, which was stored in a database, and do various interesting tree based operations on it. The most...
Safe for multi threading...
The easiest way of getting there is to have no mutable state. And here is a simple test to ensure that. Seeing how CouchDB code works and how erlag handle things is quite educating in this regard.[TestFixture]
public class EnssureTypesSafeForMultiThreadingTestFixture
{
[Test]
public void TypeIsSafeForMultiThreading()
{
var visitedTypes = new List<Type>
{ // immutable types, partial list
typeof(int),
...
Building isolated components
In this webcast, I spoke about an architecture where all the different components of the systems are inherently separated and isolated from each other. Here is a good example of this: The issue is how to implement such a feature. Well, that is actually not the issue. The major issue is how to think about such a feature. For a start, let us discard the notion that we can have a WhizBang solution for all the components in the system. The behavior for a database down vs. a 3rd party service violating is response time SLA are not...
Designing Erlang#
I am currently reading another Erlang book, and I am one again impressed by the elegance of the language. Just to be clear, I don't have any intention to actually implement what I am talking about here, this is merely a way to organize my thoughts. And to avoid nitpickers, yes, I know of Retlang. Processes Erlang's processes are granular and light weight. There is no real equivalent for OS threads or processes or even to .Net's AppDomains. A quick, back of the envelope, design for this in .Net would lead to the following interface: public class ErlangProcess
{
public ErlangProcess(ICommand cmd);...
Getting Things Done, On Time
Today I implemented refactoring support for a DSL.
Basically, it is Extract Business Condition, and it was explicitly modeled after the way R# handles Extract Variable. It even share the same shortcut, ctrl+alt+v. I also took a stub in implementing automatic pattern recognition, so when the system recognize a common usage pattern, it will automatically refactor it to a high level abstraction. It works, although I think that I can make it even more flexible than it is now. Now, if someone from the Resharper team is actually reading this, they would know that I am lying. There is no...
Does you application has a blog?
I was having dinner with Dru Sellers and Evan Hoff and Dru brought something up that really sparked my imagination. To put it in more concrete words, what Dru said started a train of thought that ended up with another mandatory requirement for any non trivial application. Your application should have a blog. Now, pay attention. I am not saying that the application team should have a blog, I am saying that the application should have one. What do I mean by that? As part of the deployment requirements for the application, we are going to setup a...
JFHCI: Quote of the Day
This is probably the best quote that I can find about the motivation for JFHCI: Enabling developers to write "bad" code quickly and safely.
Strategies for editing DSL scripts in production
Another interesting question from Chris Ortman: So I write my dsl, and tell my customer to here edit this text file? How do I tell them what the possible options are? Intellisense? This is a web app, and my desire to build intellisense into a javascript rich text editor is very low. It might be a good excuse to try out silverlight but even then it seems a large task. Or I put...
Meaningless code quality, part 2
Here is the next slide in my presentation. Code quality is a hard metric, it can be backed up by numbers, if you need to. Fairly often, it is measured by gut feeling and "this is a mess". One interesting problem with code quality is that you generally don't have a good way to measure the maintainability of a particular solution, regardless of its current code quality.
Code quality is meaningless
This is part of a presentation that I am preparing: What do you think about that?
JFHCI: Considering Scale
There are some really good comments in this post, which I am going to respond to. Before that, I want to emphasis that I am still playing devil's advocate here. Joe has two very insightful comments, regarding the just hard code everything and don't provide any admin interface for the parameters: When you're making software that will be used by more than one customer, you can't hardcode these things in. And: Again, why not? Why can't Company A use the same software as Company B and A wants...
JFHCI: The evil that is configuration
Chris Ortman asks a very good question about my just hard code it post: I really like the simplicity of the approach, but does this impose a requirement that in order to change the discount percentage of the system I must be able to write c# code? I think it would not take very long for someone to ask me for and 'admin' screen to do that. I run into this all the time with new features that would be trivial to implement if not for the...
Enabling change by hard coding everything: the smart way
Okay, hopefully this post has caused enough interest in the subject. Let us explore what I said: I consider hard coding as much as possible as a key concept in enabling change. The first part that you need to notice is "as much as possible". This is another way of saying YAGNI. That is, if you don't really need it flexible, just hard code it. JFHCI is my new motto. When accepting an order, we want to give 5 percent discount for preferred members. How are we going to handle this scenario? We...
Enabling change by hard coding everything
I had a discussion today about how to make change easier. My take on that was a bit of a surprise to the other guy, "Just hard code everything". Yes, I consider hard coding as much as possible as a key concept in enabling change. I am going to leave this as is for now, and post a continuation for that tomorrow, just to see what kind of response this has.
Multi Tenancy - Approaches and Applicability
Previously on the Multi Tenancy series: Multi Tenancy - Definition and scenario. Reviewing Litware HR - P&P Multi Tenancy Sample. Multi Tenancy - The Physical Data Model Multi Tenancy - Extensible Data Model Multi Tenancy - Extensible Behaviors Multi Tenancy - Development Structure Multi Tenancy - Where do you put variability? Multi Tenancy - Scriptability and DSL Multi Tenancy - multi tenant apps and frameworks Multi Tenancy...
Multi Tenancy - User Interface
Previously on the Multi Tenancy series: Multi Tenancy - Definition and scenario. Reviewing Litware HR - P&P Multi Tenancy Sample. Multi Tenancy - The Physical Data Model Multi Tenancy - Extensible Data Model Multi Tenancy - Extensible Behaviors Multi Tenancy - Development Structure Multi Tenancy - Where do you put variability? Multi Tenancy - Scriptability and DSL Multi Tenancy - multi tenant apps and...
Multi Tenancy - Keep It Simple, REALLY Simple
Previously on the Multi Tenancy series: Multi Tenancy - Definition and scenario. Reviewing Litware HR - P&P Multi Tenancy Sample. Multi Tenancy - The Physical Data Model Multi Tenancy - Extensible Data Model Multi Tenancy - Extensible Behaviors Multi Tenancy - Development Structure Multi Tenancy - Where do you put variability? Multi Tenancy - Scriptability and DSL Multi Tenancy - multi tenant apps and...
Mutli Tenancy - multi tenant apps and frameworks
Previously on the Multi Tenancy series: Multi Tenancy - Definition and scenario. Reviewing Litware HR - P&P Multi Tenancy Sample. Multi Tenancy - The Physical Data Model Multi Tenancy - Extensible Data Model Multi Tenancy - Extensible Behaviors Multi Tenancy - Development Structure Multi Tenancy - Where do you put variability? Multi Tenancy - Scriptability and DSL Tobin Harris asks: I'm interested in learning the difference between...
A question of Scale
One of my routine set of questions when I am coming to a new codebase is: "What is the expected load on the system?" or "How big do you need this to scale?" I have several variant of the question, including, "What precentage of your budget is dedicated for scaling scenarios?" In plenty of cases, the answer that I get is vague. There is not definite scaling requirements, but the client want to be able to scale. Here are a few rule of thumbs that will help you to know if scaling is even an issue...
Multi Tenancy - Scriptability and DSL
Previously on the Multi Tenancy series:
Multi Tenancy - Definition and scenario.
Reviewing Litware HR - P&P Multi Tenancy Sample.
Multi Tenancy - The Physical Data Model
Multi Tenancy - Extensible Data Model
Multi Tenancy - Extensible Behaviors
Multi Tenancy - Development Structure
Multi Tenancy - Where do you put variability?
It probably would not surprise you that I a seeing a place for a DSL in a Multi Tenant application. Indeed, one might almost say that I have DSL on my mind a lot these days.
Multi Tenant applications are all about managing variability...
Multi Tenancy - Where do you put variability?
Previously on the Multi Tenancy series: Multi Tenancy - Definition and scenario. Reviewing Litware HR - P&P Multi Tenancy Sample. Multi Tenancy - The Physical Data Model Multi Tenancy - Extensible Data Model Multi Tenancy - Extensible Behaviors Multi Tenancy - Development Structure Paul Cowan has left an interesting comment on my post about Multi Tenancy - Development Structure. I have migrated the app. from ASP and the ASP approach was to...
Multi Tenancy - Development Structure
Previously on the Multi Tenancy series: Multi Tenancy - Definition and scenario. Reviewing Litware HR - P&P Multi Tenancy Sample. Multi Tenancy - The Physical Data Model Multi Tenancy - Extensible Data Model Multi Tenancy - Extensible Behaviors You might ask, what is the relation between multi tenancy and the development structure... And you would be wrong to assume that there isn't any. In particular, I am talking about designing the development structure and the tenants. No, I don't...
Multi Tenancy - Extensible Behaviors
Previously on the Multi Tenancy series: Multi Tenancy - Definition and scenario. Reviewing Litware HR - P&P Multi Tenancy Sample. Multi Tenancy - The Physical Data Model Multi Tenancy - Extensible Data Model When talking about multi tenancy, one of the greatest challenges is to set the system in such a way that each tenant can define their own behaviors about the way the system behaves. In fact, this is my main criteria for whatever a system is truly a multi tenant app or simply...
Patterns for using Distributed Hash Tables: Conclusion
Well, it looks like I finally had completed all I wanted to say about DHTs. I can now go back to talking about multi tenancy :-) The previous ones are: Distributed in memory cache / storage Patterns for using distributed hash tables: Groups Patterns for using distributed hash tables: Locking Patterns...
Patterns for Distributed Hash Tables: Range Queries
Next to last, in this series on using Distributed Hash Tables (DHT)! The previous ones are: Distributed in memory cache / storage Patterns for using distributed hash tables: Groups Patterns for using distributed hash tables: Locking Patterns for Distributed Hash Tables: Locality ...
Patterns for Distributed Hash Tables: Lookup by property
Will this series on using Distributed Hash Tables (DHT) ever end? The previous ones are: Distributed in memory cache / storage Patterns for using distributed hash tables: Groups Patterns for using distributed hash tables: Locking Patterns for Distributed Hash Tables: Locality ...
Patterns for Distributed Hash Tables: Cheap Cross Item Transactions
And yet another post in my series on using Distributed Hash Tables (DHT). The previous ones are: Distributed in memory cache / storage Patterns for using distributed hash tables: Groups Patterns for using distributed hash tables: Locking Patterns for Distributed Hash Tables: Locality ...
Patterns for Distributed Hash Tables: Item Groups
And yet another post in my series on using Distributed Hash Tables (DHT). The previous ones are: Distributed in memory cache / storage Patterns for using distributed hash tables: Groups Patterns for using distributed hash tables: Locking Patterns for Distributed Hash Tables: Locality Right now I want...
What is wrong with SortedList<T>
Today it has arouse my ire. Take a look at the IndexOfKey, IndexOfValue and RemoteAt methods. You would expect a list to have a way to get to item by index, certainly based on methods such as RemoveAt and IndexOfXyz. It doesn't have such a way. You have to violate the Law of Demeter in order to get it to work, list.Values[i], yuck!
Patterns for Distributed Hash Tables: Locality
Here is another post in my series on using Distributed Hash Tables (DHT). The previous ones are: Distributed in memory cache / storage Patterns for using distributed hash tables: Groups Patterns for using distributed hash tables: Locking Now I want to talk about locality, and why it is important. First, the idea of locality is very simple. Put related items together, so...
Multi Tenancy - Extensible Data Model
Continuing on the multi tenancy theme. We have the following problem, I now want to focus on the data model, and how you can create an extensible data model.
In particular, I am thinking about the ability to define something like a base line Customer entity, and allow a tenant to extend that with new fields and behavior. Note that we are talking about both data and behavior here, not on just extending the data. But let us try to take a look at that as well, just to see what solutions we can come up with.
The scenario that we have...
Multi Tenancy - The Physical Data Model
Continuing on the multi tenancy theme, let us talk about the actual physical structure of the data store. In general, you have two basic options.
The TenantId column and separated data stores. The TenantId is simply adding a column to all the columns, showing what tenant the row belongs to.
The separate data store calls for a much higher level of separation, where there is no shared storage for the tenants.
There are variation on those, such as having separation in the schema level instead of DB level, etc, but they all end up being down to one of...
Reviewing Litware HR
Note: the last time I did even an off the cuff review of a Microsoft application, the whole CAB argument was born. I am writing this statement before looking at the code, mind you, so I have no idea what the review will be. For a change, let us start by talking about the documentation, which I am reading while the code is downloading. Multi-Tenant Data Architecture is a very interesting read, I agree to most of what they are saying there, just not with the way they are pushing things and the...
Multi Tenancy
Wikipedia defines Multi Tenancy as:
Multitenancy refers to the architectural principle, where a single instance of the software runs on a software-as-a-service (SaaS) vendor's servers, serving multiple client organizations (tenants). Multitenancy is contrasted with a multi-instance architecture where separate software instances (or hardware systems) are set up for different client organizations. With a multitenant architecture, a software application is designed to virtually partition its data and configuration so that each client organization works with a customized virtual application instance.
My definition of multi tenant includes a single instance service multiple tenants and multiple instances, one for each tenant. In fact,...
Beyond mere software design
No, I am not going to talk about how to design a system, I am going to take it as granted that you are aware on all the traditional checklists for good design and are actively applying them. What I wold like to talk about are the parts that a lot of people tend to forget until they find out about it after the system is already at the hands of the users.
I am going to talk about Rhino Security as an example to that. The predecessor for Rhino Security was a proprietary piece of software that worked on top...
It isn't scalable!
That was the first thing out of my mouth when I saw a diagram the day before yesterday. The diagram was of a domain model and included four entities.
No, I can't see the future and extrapolate from customer/order/order lines whatever the system is going to scale or not. I wasn't talking about the system scalability at all. I was thinking of using the diagram as the main means for expressing concepts. A single diagram is just not a good solution for showing any sort of a complex concept. At that particular case, the diagram is a view on top of...
Logging, Auditing and Alerts
Udi has a few things to say about logging, in response for my logging with AOP post. As usual, I agree with Udi. :-) In particular, I am in agreement with this: In broader terms, all logging goes in framework-level code. For smart clients, one really good place to put logging is in your Command infrastructure - every time a command is invoked, log it and the args. For data access, well, any decent O/R Mapper has a lot of logging already, just use it. For communication, ditto. Funny that just last week this was one...
Thinking about Cloud Computing
A few weeks ago I had a fascinating discussing with a guy about usages of cloud computing. Over the course of an hour, we covered quite a lot of very interesting options. The traditional drive for using cloud computing is the desire to avoid having to manage your own infrastructure, and the problems that one have to face when dealing with expanding one's infrastructure. Being able to hand all of that to a 3rd party is a very attractive offer, especially if you feel like this 3rd party is someone that you can rely on to do better job...
Reversibility
No, I am not talking about Jeremey Miller's reversibility. I am talking about another matter all together. I have an API that perform an operation. The problem is that based on the result of this operation, I may need to cancel that operation. The main issue here is that in order to reverse this operation, I need to expose too much information to the client, which I rather not do. Let us take the following interface as an example: public interface IActionStack
{
ActionSpec Pop();
void Push(ActionSpec action);
}
The code I would like to write is something like...
Onion Architecture
This post from Jeffrey Palermo has managed to distilled the reasons for a lot of things that I consider important. From persistence ignorance to dependency injection. To be rather more exact, Jeffrey has managed to put it in words in a way that I don't think that I would have ever could. Go and read it. The concepts will probably not be earth shattering, but the ability to communicate clearly about this is very important. .
The Sewer Architecture
On Friday, I had an encounter with the sewers at my house. On Call team has came and fixed the issue, which was a plugged pipe somewhere down the street very fast, so no harm was done (although checking the sewers just after taking a shower is not my idea of fun). That did, however, let me learn a bit about the way sewers are constructed. Broadly, the sewer is based on the idea of pipes and pools ( at least, those are the Hebrew terms, translated ). The idea is that the sewer flows in the pipes,...
The difference between best effort and assurance
Let us take a messaging system as an example. One messaging system will make a "best effort" attempt to avoid delivering duplicate messages. A second system make the assurance that it will not do so. What is the difference between the two? Best effort means that the system will make reasonable attempts to prevent it, but there are situations where this may still happen. Assurance means that it cannot happen. You can rely on it happening. Another example of a best effort vs. assurance is with caching vs. storage. A caching system will make a best...
The importance of context
Brian Chavez asks about adding multi tenancy using contextual sessions: A topic for a future post perhaps, it would be really interesting to see how you would architect a web application with a contextual session management that supports multi tenancy that is a scalable keeping in mind the possibility of web farms / load balancers. I spoke about this in my DevTeach talks about advanced OR/M and IoC. Let us start by defining multi tenancy. It is the ability to serve several customers on the same installed application. What do I mean by that? ...
The scope of an application
I am using the term "application private scope" quite often, and it seems that I need to make this clear. Note that I am talking about logical layout here, physical layout doesn't really matter here (except with rolling updates, which are beside the point). An application is all the code and functionality that is required to perform a specific task. Let us take a recent project of mine, which dealt with scheduling people to do work. The Application consisted of two different web UI, one that was based on MS CRM, the other that was based...
Default Architecture: Layers
The question came up in the ALT.Net group, and it seems like a good idea to post my thought about it. You can see my overall default design for most just about any system. No, this isn't the end all be all design dogma, it is just something that has served me well in a wide number of applications. This architecture has only one thing at its code, it is focused on clearly defining responsibilities between different parts of the application. On the face of it, this looks like the traditional Model View Controller approach, but while...
[Unstable code] How a blocking remote call can take down an application
I mentioned that this line has the potential to destabilize an application, because it is a remote blocking call. var cart = customerSrv.GetShoppingCart(customerId); Neil Mosafi left the following comment: I've never experienced other threads being blocked whilst making a sync service call. Even an Async call is essentially a sync call but done in another thread or using an iocompletion port. Or are you saying we should be making duplex service calls to avoid possible problems? Let us start by saying that I am talking about pathological scenarios,...
Patterns for using distributed hash tables: Locking
Here is another post in my series on using Distributed Hash Tables (DHT). The previous ones are: Distributed in memory cache / storage Patterns for using distributed hash tables: Groups In this post, I would like to handle locking. As I mentioned in the previous post in the series, updating a single item safely can be done using optimistic concurrency techniques. Updating more than a single item is... harder. Let...
Patterns for using distributed hash tables: Groups
Yesterday's post called them distributed in memory cache / storage, but I was reminded that the proper term for what I am talking about is distributed hash tables (DHT). I presented the problem of dealing with DHT in this post, mainly, the fact that we have only key based access and no way to compose several actions into a single transaction. I'll let you go read that post for all the gory details, and continue on with some useful patterns for dealing with this issue. As a reminder, here is the API that we have: ...
Distributed in memory cache / storage
Let us start by defining the difference between cache and storage. A cache may decide to evict items whenever it feels like, a storage will only do so at well defined points. As a simple example, this is legal for a cache: PUT "foo", "item data"
result = GET "foo"
assert result is null
The cache contract means that it "might" preserve values, or it might not, it is the cache choice. A storage contract make the above code illegal. It may never happen. Most cache solutions have a way to specify priorities, including the "do not...
Obsolete in Isolation
The question of how to deal with a brown field project came up in the ALT.Net mailing list. Here is the description of the project: You've inherited a software project that has deeply rooted dependencies, business logic embedded in the presentation layer (and elsewhere), 'unit' testing that tests databases with hard-coded strings for primary keys, no actual unit testing but contrived integration/functional tests... Sounds like one of my projects, from several years back, as a matter of fact. Oh, the things we did to System.Web.UI.GridView... Oh, I was supposing to talk about how...
Putting the container to work: Refactoring NServiceBus configuration
One of the most common issues when people are building frameworks and applications that rely on a container is that they are not giving the container enough to do. Basically, they use the container to create some components, but they are doing a lot of things that the container could do for them outside of the container. Note: Code for this post can be in the Scratch Pad. NServiceBus and Mass Transit are good examples of that. I detailed some of the issues that I had with Mass Transit a while ago. Udi and...
Public vs. Published
One of the thing that you see some people agonizing about is what they should be exposed to the outside world. On the one hand, people have heard all about encapsulation, and how it is going to save the world. On the other hand, we want to expose functionality to the outside world, be it for testing or for advance usages of your software. Something that I wish that C# had is the published alias to the public keyword, that would have expressed intent in a much easier fashion than just public. That said, I found a very...
Zero Friction: Creating a test model
I am writing some integration tests at the moment, using WatiN, and I am really enjoying the process. The last time that I tried to use WatiN it was in a WebForms environment, and it was... hard. Using it with MonoRail is a real pleasure. Here is a simple test: [Test]
public void Can_submit_new_values_for_webcast()
{
browser.GoTo(appUrl + "webcast/edit/" + webcast.Id);
var newTestName = "New test webcast name";
browser.TextField("webcast_Name").Value = newTestName;
var newDesc = "There is a new webcast description";
browser.RichTextEditor("webcast_Description_Editor").Type(newDesc);
browser.Button("webcast_save").Click();
browser.Url.ShouldContain("webcast/view/" + webcast.Id);
ReloadWebcastFromDatabase();
webcast.Name.ShouldEqual(newTestName);
webcast.Description.ShouldEqual(newDesc);
}
For a while, I was happy with this, but I have over two dozen such tests, and it is getting very...
My design process
Vijay Santhanam asks about the design process I takes for building: How much analysis/OO modeling you do before you start writing real code? With tools like R#/SVN is the up-front design pretty much dead for garage projects? My answer to that is complex. I don't do design. I don't think that I ever had. What I would do is something I call envisioning. I have posted several such things in the past, most recently it was Architecting Twitter. A good way of describing that is this picture: I am not...
Deciding on the correct syntax for UI permissions
Going back to why I hate new projects, I am trying to figure out how to perform UI level security. Right now I need to dynamically decide whatever or not a user can edit a webcast. I came up with the following options. Explicitly allow this for administrators only: <% component adminOnly: %>
${ Html.LinkTo( "Edit", "webcast", "edit", webcast.Id) }
<% end %>
Allow this by operation:
<% component securedBy, {@operation: '/webcast/edit': %>
${ Html.LinkTo( "Edit", "webcast", "edit", webcast.Id) }
<% end %>
The second is much more flexible, but I am not sure that I am happy about putting...
My BDD experiment
I am trying to get BDD. As such, I started with building a list of specifications. Here is what I have so far. Now you get the chance to tell me why I am wrong... As a user I want to be able to see the latest webcast summary on the homepage So that I have great visibility on what is new ------------------- Acceptance criteria: - Can see the name of...
When an error is a valid response
Udi is talking about occasionally connected clients, and point to several challenges that you have to face when you are building such applications. One of the major issues with such clients is that you have to take into account the fallacies of distributed computing and how to deal with them. But, and this is important, having to deal with those can be expensive in terms of time and complexity. There are many scenarios when telling the user that the system is not operational is a valid choice. No, it is not always valid, and there are as many application...
Architecting Twitter
There are some interesting discussions about the way Twitter is architected. Frankly, I think that having routine outages over such a long period is a sign of either negligence or incompetence. The problem isn't that hard, for crying out loud. With that in mind, I set out to try architecting such a system. As far as I am concerned, twitter can be safely divided into two separate applications, the server side and the client side. The web UI, by the way, is considered part of the client. The server responsibilities: Accept twits from the...
Patterns vs. Blueprints
Recently I had several discussions about the usage of patterns, and application thereof. Usually, there are recommended implementation for a pattern, often this is the way it is presented in its pattern catalog. The discussion had to do with the amount of fidelity that you had in implementing the pattern. My view on that is that the usage of a pattern doesn't require a one to one match with the common implementation, but leave room for change while keeping the concepts of the pattern in use. Patterns are the overall shape of the solution, they are not a...
Request/Reply vs. Pub/Sub
I am having an interesting discussion in email with Evan Hoff, about the benefits of pub/sub architecture vs. request/reply. I want to state upfront that I see a lot of value in async one way messaging for a lot of the interactions in an application and across services. However, I am still skeptic about the push for pub/sub all the way. To me, it seems like the request/reply model is a very natural one in many common scenarios. In most backend processing systems, I would tend to use pub/sub with sagas as both the easiest and most scalable solution....
SOA Data Access
Yesterday I posted about designing an SOA system. I identified the following service: Available candidates - given a set of requirements, will attempt to find all verified candidates that match the requirements. Additional responsibilities include logging search history (important for noticing what requirements are common and desirable), and recording which requirements has no matching physicians in the system (should be passed for human inspection, to decide if head hunting should begin, or this should be ignored). BIll Poole has left a comment regarding this post, which included this statement: For...
An exercise in designing SOA systems
This question came up in the NServiceBus mailing list, and I think it is very interesting question to try to solve. The standard disclaimers are that I haven't build a real system using this approach, although I am leaning toward it more and more. I would like to thank Brad for agreeing to post this example here. I would like some advice as to how to properly identify the services I need to create. I know that Udi has said that creating services for each department is a good starting point, but I keep feeling...
Code Generation and the Open Close Principal
Vladan Strigo made a really inspiring comment about code generation in the NH Users list. I didn't want to use codesmith for that because it would de-OCP-fy me in my future efforts :) That makes a lot of sense, and manages to touch on what bothers me the most with code gen as an architectural approach. It gives up a very important concept, and that affect a lot of the stuff that is dependant on that.
Non Invasive API, take 2
A few days ago I presented my typical way of handling container driven polymorphism. // only use for selection
public interface IRegion<T> : IRegion {}
public void SetRegion(DependencyObject containerElement, string regionName)
{
IRegion region = (IRegion) IoC.TryResolve(typeof(IRegion<>).MakeGenericType(containerElement.GetType()));
if (region != null)
_regions.Add(regionName, region);
}
I got an email with some good questions, and it is worth a second post to clarify. The questions:
The only thing I don’t like is that a lookup in the form of TryResolve(typeof(IRegion<>).MakeGenericType(containerElement.GetType())) would infer an explicit wrapper type for ever possible UI container type. On one hand it gives me the fine grain control I need...
You should hard code the idea, not the value
I am reading NServiceBus' code at the moment, and I run into this idea: [Serializable] [Recoverable] [TimeToBeReceived(“0:01:00.000")] public class ScheduleInterviewMessage : IMessage { public Guid InterviewerId; public Guid CandidateId; public DateTime RequestedTime; } I don't like hard coding this value in the message. Obviously this is something that is tied very closed...
A maintainable environment - anti corruption as a way of life
Recently I had a number of conversations about how to create a maintainable environment, not only for advanced developers, but an environment that enables beginners and intermediate developers to create good, maintainable, scalable solutions.
This problem is not a simple one, and there aren't really one off solutions. However, there is quite a lot that can be done, and I am very disturbed by the assumption that nothing can be done about it.
I noticed that a major part of the problem seems to be in the underlying assumption that in order to make it possible for beginners/intermediate developers to work with...
Non invasive API - Now with an IoC container
I talked about ways to avoid invasive API design, and a lot of people asked about how to handle this with a container. First, I want to make it clear that the previous example assumed that you can't rely on a container, so it used Poor Man IoC to do that. Now that we can assume that there is a container, this is another matter entirely. The API design now shifts to allow me to select the proper implementation from the container automatically. This generally ends up being either as a naming convention on top of a fixed interface, or...
Avoiding Invasive API design
I have a strong dislike for invasive API and framework. I dislike them because they require a lot of work, avoid existing options, limit my ability to work with them, etc. They also tend to tie you to a platform for very little reason. Here is a simple (and fictional) example. Let us assume that we want to write a library to expose our domain model in a version way. So we can do something like this: svn://localhost/Customers/15[@revision=3] Now, in order to handle this scenario, I have devised the following schema. I can create the end point using the...
SOA Future Batches
Pair<CustomerDto, ICollection<OrderDto>> GetCustomerAndOrders(int customerId);
Today I had a very interesting lunch conversation, about designing the backend of a web side, SOA style. The initial suggestion called for an interface similar to this one:
public interface ICustomerService
{
CustomerDto GetCustomerById(int id);
ICollection<OrderDto> GetOrdersForCustomer(int customerId);
}
One the surface,...
Yet another internal issue
I want to give a nice error message when windows integration is enabled. Now, WCF has this ability, and while it took me a while, I managed to track down the way they do it to this method:
HostedTransportConfigurationManager.MetabaseSettings.GetAuthenticationSchemes(base.HostedVirtualPath);
Which, of course, is internal.
Bad WCF, no cookie for you.
This is something really useful that I could make use of.
But, of course, Framework Design Guidelines says that you should make everything internal.
Argh!
Design Corruption
A short time ago, I commented on the problematic decision to move from interfaces to base classes in the MS MVC framework. Hammett has left the following comment there: Now its abstract class, then they will protected internal by default, and in the end people will get the usual MS tool, while this could be a break from the old habits. Phil Haack, today (emphasis mine): This is why there tends to be an emphasis on Abstract Base Classes as I mentioned before within the .NET Framework. Fortunately, when...
Continuous Environment Validation
For some reason, it seems like I am talking about Release It a lot lately, to a lot of people. As I said when I reviewed it, that book literally changed the way that I approach problems. It also made me much more aware of the failure scenarios that I need to deal with. A while ago I sat down in one of Jeremy Miller's talks and he mentioned that he had added the ability to do Environment Validation to StructureMap, so when the application is starting up, it can verify that all its dependencies are in a valid...
Zero Friction and why Defaults Matters
Several times recently I had to answer why I care so much for defaults if they are something that I can change. My answer is usually a derivative of "defaults matters". I have come to understand that this is not a sufficient answer and I need to clarify it. I have talked extensively about Zero Friction environment and why I care so much about it. It is about how much I have to fight the tools that I have in order to get what I want. I spent much of yesterday moving Rhino Tools from MsBuild to...
Reviewing Unity
I am sitting at the airport at the moment, having to burn some time before I can get on a flight, and I decided to make use of this time to review the Unity container from the P&P group. A few things before I start. Unity is a CTP at the moment, it is not done, and that is something that should be taken into account. I have several design opinions that conflict with the decisions that were made for Unity. I am a contributor to Windsor. Overall, I am biased. Please take that into account. I am...
Ayende's Design Guidelines: Rule #1
When creating a generic method, strongly prefer to supply an overload that is not generic, can accept the types manually, and is externally visible. Reasoning: The generic version often looks significantly better than the non generic version, but it comes with a cost. It assumes that you know, up front, what the types are. When you are writing any type of generic code, this is almost always not the case and your generic method is useless in that scenario. Example: myContainer.RegisterComponent<IService, ServiceImpl>(); is a good syntax to have, but the problem with that is that it cannot be...
Handling dependencies in a one assembly
There were several comments for my recent post about the disadvantages of creating a single assembly for a project, instead of the more common multiply ones. The main one seems to be related to dependency management. How you can be sure that the UI isn't accessing the database, and other fun violations of layering. I have to say that I don't really have much issue with that. I want good layering, but I find that code style in a project tend to keep consistent over a long period of time in any reasonable time. As such, once you...
The two project solution
Back to technical content :-) This post from Kyle Baley got me thinking about my last few projects. In them, I trended toward the assembly per logical layer of the app. So I had things like: MyApp.Model MyApp.Web MyApp.Services MyApp.Infrastructure MyApp.Tests It worked, but at one point I found myself managing 18 projects solutions, and that was only because I was committed to reducing the number (otherwise it would have been much higher). I hear about people talking about 30 - 60 projects in a single solution, and there are people with more. Considering the cost of just managing...
Re: Versioning Issues With Abstract Base Classes and Interfaces
Phil Haack is talking about why the MS MVC team changed IHttpContext to HttpContextBase. I follow the argument, but at some point, I just lost it. This, in particular, had me scratching my head in confusion: Adding this method doesn’t break older clients. Newer clients who might need to call this method can recompile and now call this new method if they wish. This is where we get the versioning benefits. How on earth does adding a new method to an interface would break an existing client? How on earth does adding a new method to an interface require...
Refactoring for Separation Of Concerns: A real world example
I just did a major refactoring to Rhino DSL, to add orthongality to the DslEngine. It was an interesting refactoring, and something that is worth talking about. Let us take a look at the before image, first: We have two classes that are involved in creating and managing DSL. Notice the FileFormat, CreateInput, GetMatchingUrlsIn and NotifyOnChange methods on the DslEngine? Or the GetFromCache, RemoveFromCache, SetInCache methods? The firsts are related to the storage of the DSL scripts on disk and the second batch are related to caching, they have nothing to do with the actual work performed by...
Infrastructure Ignorance
Recently there have been a long discussion in the ALT.Net mailing list about Auto Mocking Containers. We covered a lot of ground there and it is interesting to boot. What I wanted to talk about comes from this discussion. A while ago there was a big stick about Persistence Ignorance. I won't reiterate the arguments, but I want to expand the idea to other concepts as well. It is not just persistence that your code should be ignorant of, it is all the infrastructure concerns in general. I don't care how good your infrastructure code is, it is still...
Setting Up Zero Friction Projects - Data Access
After talking about the advantages setting up environments where the laziest approach is also the faster approach, I want to talk a bit about the details of actually doing it. Zero Friction approach is very important for the overall success of a project, because it means that the right thing is the easy thing, so that is what will get done. Something to note is that this doesn't excuse ignorance, the developers using this approach should be able to at least understand the concepts and methods that they use. I am saying that there is no excuse for making...
Don't make me THINK!
I am having a good discussion in the ALT.NET mailing list about how to get buy in from developers for advance concepts. From my point of view, this is based solely on making it easy, transparent and simple. If you can set up a system that means that the zero friction path is also the path for good design, you are golden. In other words, A while ago I was asked what I think was a good quality for an architect. My reply was that he should set things up in such a way where the developers...
Academia in Real World Development
Frans Bouma has decided to unsubscribe from the alt.net mailing lists. I'll miss him there, I usually disagree with him, but the debates are interesting, educated and fun. What I want to talk today is about something that he pointed out in the post: What surprised me to no end was the total lack of any reference/debate about computer science research, papers etc. except perhaps pre/post conditions but only in the form of spec#, not in the form of CS research. Almost all the debates focused on tools and their direct techniques, not the computer science behind them. In...
Code that read and modify code, code that read codes and output XML
If you can follow that sentence, congratulations. I spend today dealing with an interesting problem. We have a set of commands that operates in response to a message from another system. This system is configured by an XML file. One of the parameters for configuration is which fields in the message should be sent. I consider this an abomination unto Nuggan, but this is a key performance optimization technique. Otherwise the external system may spend as much as ten seconds just gathering the data for this message. We started out using the simplest approach of just give us all...
Task Scheduling Improvements
I took some time to see how I can improve the sample implementation of the task scheduling that I posted yesterday. You can checkout the code here. What I wanted to check is the ability to scale the solution to many tasks and to IO bound tasks. In a stress test that I made, the library held its own while scheduling 9,864,100 tasks, at one point we had ~3,500,000 concurrently queued tasks. Memory usage hovered at the ~500Mb range. The next stage was to see what happens when we have tasks that takes a long amount of time to execute,...
Concurrency Solutions
I am not sure why, but recently I have became interested in concurrency again. I blogged about both the pipeline and the tasks scheduling approaches recently, and I was repeatedly pointed to Retlang, Parallel FX and the Concurrency and Coordination Runtime. I thought that it would be interesting to compare those approaches. The pipeline The pipeline can be easily parallelized, because there is an explicit handoff of objects as they pass from one stage of the pipeline to another. The only synchronization needed for this would be in terms of passing the objects between the stages. Task Scheduling Task scheduling...
Erlang processes on the CLR
I mentioned that getting to Erlang like processes is something that I really want to get to. I started thinking about how I can make it work with the IEnumerable<T> implementation. As it turned out, that was fairly simple, once I got the idea. Take a look at this class diagram: We have tasks, and we register them in the scheduler. We can also wait for them without blocking the thread. Check out this trivial example. We have the following long running task: public class MultiplicationTask : AbstractTask
{
private readonly int value;
...
Fluent Pipelines
I am having a discussion with Jon Skeet about the merits of using Linq for pipelines and delegates/lambda instead of classes. I kept saying that I don't really see the point, so I went ahead and implemented this: GenerateData(10000)//enumerator from 0 .. 10000
.Where(i => i % 3 == 0)
.Transform(i => (i * 2).ToString() )
.Act(i => Console.WriteLine(i))
.Execute();
This uses the same approach as my previous pipeline, but it does it in C# 3.0, so it can use things like extension methods, which make this nicer. The same in C# 2.0 is possible, but take some ridiculous amount of code to do.
This code...
Pipes and filters: The multi threaded version
Another advantage of using the pipes and filters approach is that it is naturally thread safe. Only a single filter is handling an item at a given time, even if the pipes are all running on multiply threads. This means that we have an very simple way to introduce threading into the system, without any hassle whatsoever. Let us take a look at the single threaded pipeline: public class SingleThreadedPipeline<T>
{
private readonly List<IOperation<T>> operations = new List<IOperation<T>>();
public SingleThreadedPipeline<T> Register(IOperation<T> operation)
{
operations.Add(operation);
...
Pipes and filters: The IEnumerable appraoch
Pipes are very common in computing. It is a very good way to turn a complex problem to a set of small problems. You are probably familiar with the pattern, even if not explicitly. The ASP.Net Http Pipeline (Begin_Request, Authorize_Request, etc Compiler Pipelines (Parse, ProcessTypes, SaveAssembly, etc) Command Line piping (ps -ax | grep Finder) What I wanted to talk about today was how to implement this in code. I did several implementation of pipes and filters in the past, and they all were overly complex. I took this weekend to look at the problem again, and...
Code base size, complexity and language choice
Via Frans, I got into these two blog posts: Steve Yegge - Code's Worst Enemy Jeff Atwwok - Size Is The Enemy In both posts, Steve & Jeff attack code size as the #1 issue that they have with projects. I read the posts with more or less disbelieving eyes. Some choice quotes from them are: Steve: If you have a million lines of code, at 50 lines per "page", that's 20,000 pages of code. How long would it take you to read a 20,000-page instruction manual? Steve: We know this because twenty-million line code bases are already moving beyond...
Fuzzy smart selections
Not, I am not particulary hungry at the moment, despite what this image may suggest. Let us assume that I am hungry, and I would like some sort of a Vegetable to eat. Now, I know that I would really like to eat a carrot. Problem. I don't want to try all the possibilities in the kitchen. That would be real horror. But do I have any other choice? Assume that I have the time to do whatever I want, how would you handle the question of "give me something like carrot". Please remember that...
Don't like the API, go ahead fix it, or not?
Over in the alt.net mailing list, Charlie Poole has said that he thinks that the Playback() method for Rhino Mocks would be better named Monitor(). Without thinking about it, I sent this reply: public class CharliePreferences
{
public static IDisposable Monitor(this MockRepository mocks)
{
return mocks.Playback();
}
}
After I sent it, I started thinking about the implications. Both for legacy code and for preferences. I am not sure whatever this is a good approach is a good...
MVP Pattern guidance from patterns & practices
In have tried to find a clever way of saying this, but I couldn't really find a good one. Saying that I am happy about it should suffice. The P&P's guidance package for MVP goes over the pattern, forces, limitations and reasoning behind the pattern. The MVP quick starts comes with a CWAB and POCO versions, and it is interesting to compare the two. Frankly, I consider the manual hooking of the presenter in the view to be an anti pattern. It is a PITA of the first order, to start with, and it mixes concerns. There aren't really...
Thinking about repositories
For a long time, I have used the idea of IRepository<T> as my main data access method. It is a very good approach, and I have used a static gateway to be able to resolve that effectively. The code to use that read like this: Repository<Account>.Save(account); But, I have an issue with that, because this is an opque dependency. I don't have a way to look at the declaration and just get what this needs. It makes testing a lot harder than it should be, because you now need to do indirect dependency injection. There is the additional problem...
A vision of enterprise platform: A database that you don't hide in the attic
For some reason, the moment that peole start working on Enterprise Solutions, there is a... tendency to assume that because we are going to build a big and complex application, we can afford to ignore best practices and proven practices. That is just wrong. And it applies in many manners to many parts of the application, but it never applies to something as obiviously as it applies to the database. This is just the results of a very short research: The metadata driven approach, AKA the 5 tables DB The Blob The 5th normalized form DB ...
Code review: The PetShop Application
I gave a talk about ReSharper today, and I used the PetShop demo app as the base code. I have purposefully avoided looking at the source code of the sample until today, because I wanted to get a gueniue experience, rather than a rehearsed one. I don't think it went as well as it could have, but that is not the point of this post. The point is to talk about just the code quality of the PetShop application. First, let us see what the PetShop application is: The Microsoft .NET Pet Shop 2.0 illustrates basic and advanced coding...
Slipping under the radar
I am having quite a few interesting discussions at DevTeach, and one of those had to do with introducing projections and processes against opposition. For myself, I am a... bit forceful about such suggestions, especially in face of stupid opposition. One of the things that came up was simply to do it, the old "it is easier to ask for forgiveness than permission". I am both supportive for that and not really comfortable with the idea. I support it because it is a way to actually get things done, but I got a really good example of why it is...
Graphical Domain Specific Languages
Another form of DSL is the graphical domain specific language. What do I mean by that? I am talking a DSL that is not textual, but rather uses shapes and lines in order to express intent. A good example would be UML. That is a DSL for describing software systems. In fact, quite a lot of money and effort has been devoted to making UML the One True Model, from which you can generate the rest of the application. There has been also a lot of effort invested in making it possible to write your own graphical DSL. Microsoft has the...
A vision of enterprise platform: Hot & Distributed Deployment
If deployment means a week with little sleep and beepers going off just as you nod to sleep, then you are doing it wrong. I am saying that as someone whose standard deployment procedure has two steps, "svn up" and "build.cmd". Actually, I lied, I have a single step, deploy.cmd, which runs them both for me. Now, you can't always get this procedure to work, this is especially true if you need to deploy several different parts of the same application at the same time. The classic example is making a change that requires modifying the database as well. More...
A false sense of security
After reading my post about enterprise security infrastructure, Comlin asks: Ayende, What would you say if I give you a real world scenario:1. Server running Microsoft Windows hosts a web site which includes your security features.2. Separate server hosts database. Well, common thing isn't it?We are pretty sure that the first server(it's on the Internet) would be hacked sooner or later and the hacker will acquire admin privileges. And all we have is your "entity security" inside the BL, who needs it anyway, you will take the connection string and perform all the malicious actions inside the database! I...
A vision of enterprise platform: Security Infrastructure
I have been asked how I would design a security infrastructure for my vision of an enterprise platform, and here is an initial draft of the ideas. As anything in this series, no actual code was written down to build them. What I am doing is going through the steps that I would usually go before I actually sit down and implement something. While most systems goes for the Users & Roles metaphor, I have found that this is rarely a valid approach in real enterprise scenarios. You often want to do more than just the users & roles,...
A vision of enterprise platform
A while ago I asked what kind of constraint an enterprise platform should deal with, I have been thinking about this ever since. Let us go over the list of constraints that have been brought up, and then I am going to discuss how I think about solving them. Extensible in an easy manner - note that this holds for business analysts and for developers, both are groups that are likely to do work on the system. Ideally we can have some sort of a common interface that would make both people happy. New entities User...
Platform should have source control, and that is not optional
Gary has a post that talks about something that I have been thinking about for a while. Platforms, defined as applications that you extend, must have source control built in. That is not an optional feature, that is a Must Have. I am working on such a system now, and the amount of effort that it took to get to the point where I can have sort-of seamless source control means that I am not going to work with the product again, period. I am toying with the idea of an enterprise platform that does hurt, that is possible to...
Hasty notes on Big Refactoring
I am going to write a more full featured post about this, but I think that the jotted notes for that are also interesting. Big rafactoring is what I call to a refactoring that touces large swaths of the code base, changing the implementation of a core service is one such example, or changing key archtictual patterns. Must have tests at first. Change is too big, will affect too much of the system to change in place. Can't change the system and the test at the same time Taking as granted that you have source control. Ability to rollback...
A question of scale
I am using a different meaning to the terms "scaling" and "scalable". I am usually not really worried about performance or scaling out a solution. I am often thinking about how I can take a certain approach and scale it out to the more complex scenarios. Dragging a table to the form as an example, doesn't scale. It doesn't scale because when you need to handle logic, you need to go into significant complexity just to get it. NHibernate does scale, because so far it handled everything that I threw at her without increasing the complexity of the solution beyond...
Embracing fluent interfaces
Jeff Atwood is talking about languages in languages, and suggest that you would avoid using fluent interfaces to hide a language semantics. I completely disagree. Let us talk about the example. The first one deals with regular expressions: //Regular expression
Pattern findGamesPattern = new Pattern(@"<div\s*class=""game""\s*id=:"(?<gameID>\d+)-game""(?<content>.*?)
<!--gameStatus\s*=\s*(?<gameState>\d+)-->
");
// Fluent interface
Pattern findGamesPattern = Pattern.With.Literal(@"<div")
.WhiteSpace.Repeat.ZeroOrMore
.Literal(@"class=""game""").WhiteSpace.Repeat.ZeroOrMore.Literal(@"id=""")
.NamedGroup("gameId", Pattern.With.Digit.Repeat.OneOrMore)
.Literal(@"-game""")
.NamedGroup("content", Pattern.With.Anything.Repeat.Lazy.ZeroOrMore)
.Literal(@"<!--gameStatus")
.WhiteSpace.Repeat.ZeroOrMore.Literal("=").WhiteSpace.Repeat.ZeroOrMore
.NamedGroup("gameState", Pattern.With.Digit.Repeat.OneOrMore)
.Literal("-->");
I can read the fluent interface, but I need to parse the regex pattern....
Template Controllers
Sharing common functionality across controllers is something that I have run into several times in the past. It is basically needing to offer the same functionality across different elements in the application. Let us take for a moment a search page. In my current application, a search page has to offer rich search functionality for the user, the ability to do pattern matching, so given a certain entity, match all the relevant related entities that can fit this entity. Match all openings for a candidate. Match all candidates for an opening. That is unique, mostly, but then we have a...
Deal Breakers
Occasionally I get to try something that sounds really good, but it falls on the execution because of the small details. Let us take FireFox Prism as an example. The idea is good, I get a separate fire fox window just for gmail, or other always on apps. Note that I am using it as an example only, because I just run into it. Prism is still in beta, so it is just an example. This means that if the browser crash I don't lose my unsaved email, and more importantly, I can close FireFox (and thus release the memory) without...
Specifying Specifications
Specification: An explicit statement of the required characteristics for a matching entity / object. Specifications are very useful concepts when you need to think about searches. They are useful because they seperate what you are searching from how you are searching. In the example to the right, we have a specification for a job opening. We can use it to search for matching job openings. But couldn't we do it directly? Yes we could, but then we would dealing with both what and how at the same time. Let us say that we want to search for an opening...
The IoC mind set: Validation
I think that I mentioned that I believe that having a good IoC container at your disposable can make you really shift the way you are structuring your code and architects solutions. Let us take for instance Validation as an example. The two examples that jump to mind are Castle.Components.Validation and the Validation Application Block. They both have the same general approach, you put an attribute on a property, and then you use another class to validate that the object meet its validation spec. This approach work, and it is really nice approach, for very simple input validation. Required fields,...
Where are the people?
Jeff Brown correctly points out that I have not included the people when I talked about Enterprise Software: Enterprise systems generally incorporate a very large installed base of internal and external stakeholders that have widely varying perspectives and needs. These people are accustomed to a particular conception of the business operating environment. They have developed a specialized vocabulary to describe their doings. They have adopted work habits to get them through the day. To observers they often appear to behave in strange, mysterious and chaotic ways. Changes run the risk of inducing systematic paralysis, shock, mass-hysteria and cannibalism. Read the...
From demo to production: Handling the edge cases that aren't there
There is a difference between getting a scenario to working and actually finishing it. The first part is just making sure that the tech stuff work, that is generally easy, the hard part is getting all the stuff that you need to take care of taken care of. The biggest problem is that often you don't really see those issues. Let us start with an example from today. Let us have this user story: As part of the nightly maintenance routines, two days before the event, the system will remind all the participants about the event using SMS....
Thinking about Enterprise Platform Constraints
I mentioned that I hate aimless bitching before, and I have been doing that for a while about Microsoft CRM. I would still like to lob the GoF, the PoEA and the AntiPatterns books at the team responsible for the CRM, but the new goggles are wonderful, it barely hurt at all. * Just to be clear, and for Scott's peace of mind, I don't have any intention to build Rhino CRM. That is beyond the scope of my interest. That doesn't mean that I can't amuse myself by thinking out loud the ideal architecture for such as system as...
ALT.NET is a divisive thing
Note, this is a partial (and the final) response to Sam Gentile post about ALT.NET. Now to the part that actually talks about ATL.NET: ALT.NET is a divisive thing. No matter what they tell you, they are full of negative energy, they sneer at others that don't buy into their view and sneer at the "enterprisey" folks. ALT.NET is divisive, of course. Like any idea, it divide people into those who agree and those who don't. The ALT.NET doesn't include a value statement about those who don't agree, I am feeling forced to point out. And I am...
Architects & ALT.NET
Note, this is a partial response to Sam Gentile post about ALT.NET.
One of the things that Sam said that I fully agree on is this one:
...walking into a Fortune 50 bank with 200 systems, using 20 different technologies, totally non-integrated, and listening to the business needs and using solution technologies like BizTalk, Neuron, WCF, MOSS, Sharepoint and such putting together a solution that requires me to have intimate knowledge of the business as well as single sign on, multi-domain security trust domains, asynschronous message brokering and lots more
Solving this kind of problems is non trivial and requires understand both tech...
Enterprisey vs Enterprise Software
Note, this is a partial response to Sam Gentile post about ALT.NET.
Enterprisey - This is a derogatory term, used to refer to a system whose design or implementation is overly complex compared to its supposed function. Often the term refers to the usage of anti patterns such as Everything is configurable, Executable XML, etc. Those systems are consider bad because a simpler design or implementation would have achieve the same purpose, at far lower cost, with greater maintainability overall.
Enterprise Software - A common term used to refer to systems that usually run a core part of a business. Often those are mission critical systems, which...
More on Data Layer Componentization
A while ago I commented on Alex's data-layer componentization idea. Alex replied, and Diego as well, but I didn't get around to answering them until now. To remind you, the basic idea is: A data-layer component simply allows for information to be shared between different applications. By supporting re-use data layer componentization allows you to break the duplication habit. You can still have silos, but the natural tendency to duplicate data disappears. Each new silo will instead contain only new types of data and where old types are required, simply pointers, or cross silo foreign keys. I am...
Skinny Controller, Fat Model
Scott Bellware has a post about Responsible SRP, he presents the Ruby's idiom of skinny controller & fat model vs. the .Net one of a service layer. He links to this post from Jamis Buck, which explains the Ruby concepts much better. This is also something that is annoying me about my current application, Rhino Igloo doesn't really provide the same level of separation that I consider essential, and as a result my controllers are doing way more than I wish them to. I probably should have broken at least some of them up to services. I find myself troubled, because...
More reasons for interfaces
Usually when I start a system, I want to focus on the shortest scenario that would bring me a live system. That is usually the login page, since that is almost universally identical. My approach to that is fairly simple, Login Controller that collaborate with an authentication service and an authorization service. The first to verify who the user is, the second to verify that the user even has the right to log in. I always use interfaces for those, so I would have IAuthorizationService and IAuthenticationService as the forth or fifth elements in my application. Now, the...
Good Design is Additive, not Subtractive
Wendy has a very good point here. Good practices, good design, are not there in order to facilitate some fad-of-the-day. The case she is giving is DI, SRP and SoC with regards to testing, and then taking the nuclear approach to testing, and deciding that since you can do just fake on the fly anything you need, you can test without DI, SRP and SoC, therefor, you don't need them any more. That is a problem. Consider for a moment the guide lines for writing good C programs. Short methods, clear naming, guard clauses, etc. Those still holds today. In...
Per Application Database
Scott Bellware is talking about application embedded databases, an application that is the sole domain of the application that is using it. I don't like the term, because embedded databases have another meaning, but I fully support the idea that the database should be an implementation detail of the applications. I had a project where we needed to use an existing database, and I want to cry when I think about the length and breath of of effort that we had to go through to get it to really work. In the last project, I have had at one time to...
Data Layer Componentization
Alex is talking about Data Layer Componentization, as it works in Base4 and apparently in the Entity Framework as well. A data-layer component simply allows for information to be shared between different applications. By supporting re-use data layer componentization allows you to break the duplication habit. You can still have silos, but the natural tendency to duplicate data disappears. Each new silo will instead contain only new types of data and where old types are required, simply pointers, or cross silo foreign keys. We have two different things here, a shared schema and a shared database. As...
Plain Old Code
When J2EE came around, the notion of a container and managed objects / entities came into the fray, but those had such a high cost associated with them, that the Java community rebelled and now mainly advocates the Plain Old Java Objects. The .NET community has taken the same approach recently, with POCO or PONO being the term used. This post is driven by my experiences with several of Microsoft's tools that tend to focus on "No Code Required". WebForms' 2.0 had a huge marketing focus on that, and MS CRM is being sold with the premise that you can extend...
Dependency Injection in a Dynamic Environment
Scott Bellware joins the DI discussion from a completely different angel, the dynamic language point of view: The level of abstraction that we work at in .NET is appropriate to explicit inversion of control and thus dependency injection. To make type mocking your main method of achieving testability in .NET development will drive your designs increasingly sap the maintainability of your code. Maintainability and agility in a static environment are mostly the effect of applying appropriate static design patterns for static development. [...snip...] The difference in my comfort level in doing this has a lot to do with the fact...
Dependency Injection: Separating the Container
In a previous post, I made the statement that I believe that using an IoC container should be transparent to the application code, and that a good IoC can and should be invisible to anything but the Main() method. I would like to expand a bit on this topic, cover the sample problem that Tobias had brought up: I never could achieve this and I'm wondering, what you mean by this statement. There a are always a lot of references to the DI/IoC container throughout the application. E.g. each IView that needs to be created to show a WinForm...
Searching for DDD code for reading
This may sound strange, but I want to read some DDD code, and I am sure where I can find some. There frameworks code is available all over the place, but DDD is often practiced inside a business app, which rarely has the source available. Preferably, it is something more than an example application.Any suggestions?
Complex problems in a dynamic environment
My last post made me think about how it is best to approach a problem in a dynamic environment. The problem that I have in mind is the one I solved with IoC and static typing here. The problem is as follows: You have several customers, each of them has its own database, with its own schema, data and behaviors, There is a common ground between the customers, but each does (different things!) differently. You need to handle each incoming user according to its associated customer. For the purpose of discussion, let us say that we want to list...
Dependency Injection: Applicability, Benefits and Mocking
Jacob Proffitt continues the DI discussion, and he opens with this: Ayende Rahien responded individually to each of my posts himself, which is more than a little bit intimidating all on its own Despite rumors to the contrary, I want to mention that I abhor violence in all its forms. It always leads to having to fill forms in triplicate, and I got sick at that when they pulled form 34-C12 from the archives, just for me. Someone should remind me (on a non durable medium only) to demonstrate how you can get whatever you want, explicitly without violence...
Dependency Injection: IAmDonQuixote ?
Jacob Proffitt has another post, called Tilting at Windmill that expound on this subject, and Nate Kohari has a good response to Jacob's first post here. In this post, Jacob starts with: I’ve been giving poor Nate Kohari a hard time over at Discord & Rhyme. He has been very patient in Defending Dependency Injection. His attitude stands in sharp contrast to Ayende Rahien's post today about testing Linq for SQL. Ayende’s snide "(Best Practices, anyone?)" is exactly the attitude I lamented in my original post on Dependency Injection Jacob, yes, I fully believe that it is a best practice to...
Dependency Injection: More than a testing seam
Jacob Proffitt has a post about Dependency Injection, which he doesn't seem to like very much. The real reason that DI has become so popular lately, however, has nothing to do with orthogonality, encapsulation, or other "purely" architectural concerns. The real reason that so many developers are using DI is to facilitate Unit Testing using mock objects. Talk around it all you want to, but this is the factor that actually convinces bright developers to prefer DI over simpler implementations. I do wish that people would admit that DI doesn’t have compelling applicability outside of Unit Testing, however. I’m reading articles...
We want to build something... beautiful!
I have currently stopped working on the UI for Rhino ETL, and it is an interesting experience. In the past, I have usually started with building the functionality and then adding the UI for it. That failed. The problem wasn't that the functionality wasn't there, it was that the UI wasn't nice enough for use, or wasn't pretty enough to attract. This time, I am doing it in reverse, I am building the UI first, at least the rough draft of it, and then I intend to go and hook everything up. This means that while the picture on the...
Business Oriented Testing: FIT vs. DSL
Jeremy Miller is talking about getting disillusioned about FIT, and I have to agree. FIT is a good idea, but the implementation leaves a lot to be desired. For myself, I am very interested in the ability to express the requirements in a way that is clear, concise and can hopefully make sense to a business user. Instead of going with the FIT route, I think that I would go with a DSL for testing the domain. For instance, let us say that I want to test scheduling work for employees. Something like this: emp = Employee("Smith", Qualifications: [...
The Visible Long-Term Costs of NOT Doing it Right
Jeff Brown has a good post about in response to my No Broken Windows post. My beef is that the absolute cost differential Ayende cites in this case is too small to be meaningful. The difference between a 1 hour task and a 4 hour task is 3 hours of sleep later in the week. Now do the same math over ten different features, what do you get? (It is a fallacy, actually, doing it ten times would give me better chance to optimize my way, with no such way in the hackish way). Quite simply, it...
The only metric that counts: Maintainability
There are a lot of code metrics that float around, measuring lines of code, code complexity, number of classes/methods, test coverage, etc. The important thing there is to remember that all of those are fairly meaningless. When measuring the quality of a code base, maintainability is the only thing that I care about in a code base. It breaks down to a lot of smaller values, but in the end, I look at the maintainability score as the sole parameter that I care about. Tests helps, and intent revealing, well factored code is manna from heaven, but it is...
No Broken Windows
I had to deal with a change request today. The customer wanted to be able to manually enter /edit data into administration tables that are supposed to be handled using batch processes only. What he wanted was really simple, and could probably be handled using a grid + edit + paging in half an hour to an hour. I refused to build it that way. The customer is probably the best I have had so far, but no one access my database but my application. That includes the admin module as well. The reasoning is a little more than extreme reluctance about...
Alas, security is a business concern...
A while ago I posted a question regarding pre-built security infrastructure that I could use. At the time, I envisioned something the like of Windows's ACLs, where you can define permissions on individual objects. At hindsight, I think that I was very naive. Here are a few requirements that I had to deal with in recent projects: you are only allowed to handle cases for customers in your assigned regions. you may only order fixes for products that you have purchased - in fact, you may not even see fixes for products that you don't own, because they...
First Class Functions: Having fun with delegates & design patterns
When writing Rhino ETL, I am getting more and more attached to the idea of first class functions, and their usage. The ability to just send a method (with context!) to be executed elsewhere is amazing. The really nice thing is the ability to express design patterns with focus on the intent vs. focus on the how. As an aside, this post should make your head hurt. Let us take a factory, for example: public delegate WaitHandle WaitHandleFactory();
public WaitHandleFactory CreateWaitHandleForCommand(ICommand command)
{
return delegate
{
ManualResetEvent resetEvent = new ManualResetEvent(false);
command.Completed += delegate { resetEvent.Set(); };
return resetEvent;
};
}
And a decorator:
public delegate void Command();
public Command DecorateWithPerformanceLogging(Command...
On Spec Reading
I feel like pushing someone through a window. I am reading a 200+ spec right now, and it is read exactly like a badly written combination of user documentation, wishful thinking and paper form specifications. This is a spec that truly demonstrate that the Blob is not just a coding anti pattern. I literally have to reverse engineer it in order to figure out what I am supposed to do.
Presentation Logic & Semi Integration Tests
Scott has an interesting view about my post regarding testing presentation logic: However, his answer to this issue involves basically executing a dynamic page (in MonoRail) and testing the HTML output. He admits that this is a "semi-integration test". I would agree with that, except for the "semi" part. If I really wanted to execute a page at the unit testing stage, I could use a tool like NUnitAsp, which operates at the same level of abstraction as ASP.NET server controls and thus insulates the test code somewhat from the actual rendered HTML markup. I call it semi integration test because...
The Correct Separation Of Concerns
I have just read this post from Hammett, talking about the difference between separating business logic and presentation logic vs. separating presentation and presentation logic. This comment has caught my eye, Nicholas Piasecki says: To me, this discussion all boils down to one thing: the foreach loop. Let’s say you want to display a table of sales reports, but after every tenth row, you want to print out an extra row that displays a running total of sales to that point. And you want negative numbers to appear in red, positive numbers to appear in green, and zeros to appear in...
Fluent Interfaces & Method Chaining
Hammett calls the term "Fluent Interface" unnecessary: And about Fluent interfaces, what about OOP? What about method chaining? Does it need, for god’s sake, a new name? This is method chaining: string user = new StringBuilder()
.Append("Name: ")
.Append(user.Name)
.AppendLine()
.Append("Email: ")
.Append(user.Email)
.AppendLine()
.ToString();
And this is a fluent interface:
return new Finder<Order>(
Where.Order.User == CurrentUser &&
(
Where.Order.TotalCost > Money.Dollars(150) ||
Where.Order.OrderLines.Count > 15
),
OrderBy.Order.CreatedAt
).List();
Anders posts about DSL building contains examples that are tied more closely to the domain.
Method chaining is something that you would certainly use in the a fluent interface, but that is like saying that you need to use interfaces when you build a plugin framework. The fact that...
What would you say if I wanted to write my own... OR/M / IoC / Web Framework / Etc ?
Occasionally I get asked this question, and the answer I give is usually along the lines of: Why? This is not a mocking question ( :-) ), it is the criteria that I use to answer this question. Are you doing it because you want to and it is fun? Are you doing that to solve your own problems, and you decided that you will get better return of investment by building your own custom thing? Are you doing it to solve a specific problem, without an off-the-shelf solution? If it is any of the above, I would tell...
Query Building In The Domain / Service Layers
Here is an interesting topic. My ideal data access pattern means that there is a single query per controller per request (a request may involve several controllers, though). That is for reading data, obviously, for writing, I will batch the calls if needed. I am making heavy use of the Multi Criteria/Query in order to make this happen. I have run into a snug with this approach, however. The problem is that some of the services also do data access*. So I may call to the authorization service to supply me with the current user and its customer, and the service...
People over Code
While there is value in the item on the right, I value the item on the left more.
This is in response to a comment by Jdn, I started to comment in reply, and then I reconsidered, this is much more important. A bit of background. Karthik has commented that "Unfortunately too often many software managers fall into the trap of thinking that developers are "plug and play" in a project and assume they can be added/removed as needed." and proceeded with some discussion on why this is and how it can be avoided.
I responded to that by...
When should you encourage code duplication?
Another story from class, today I was reviewing some of the student's code, and we run into this one: The student is one of the keenest persons I have ever seen on removing code duplication (painful experience, I understand). But in this case, I had stopped and started using clipboard inheritence to duplicate the code. No, I have no gone insane. The issue is in this case, removing code duplication has cause a violation of the Single Responsibility Principal. That is a bad idea. These two views had distinct responsabilities, they just happened to have identical...
Tools vs. Design
Just read this, Eli Lopian talks about how you can use Type Mock in order to avoid separating responsabilities: I am going to show you just how we can manage to unit test a Simple Dialog without separating concerns I have a problem with this statement. Why would I want to avoid separating concerns? The Single Responsability Principal & Separation of Concerns are critical to building maintainable systems.
What does it takes to calculate a sales tax?
Lately a few people have pointed out at sales tax calculation as a simple business logic that can be done in the database. Let us consider what is involved in such a task, shall we? Well, on the first iteration we have: The sales tax is 10% of the listed price. CalcSalesTax(price): return price * 1.1 On the second iteration, we learn that we now need to support more than a single location, so we need to keep an association of location to sale tax. CalcSalesTax(price, location): salesTax = locationsToSalesTax[location] return price * salesTax On the third...
Frameworks vs. Services: On choosing the right approach
Recently there was some discussion about the CAB during which I think we found the core difference between the CAB approach vs. some of the OSS tools that were brought up in the discussion (sorry, I can't recall who first brought up this observation). Frameworks has a much higher adoption cost associated with them vs. services. Services are more easily compassable and replaceable. The general option is that services == Often Good, Frameworks == Sometimes Good. Feel free to shift the scales every which way you want, obviously I am leaning toward the services over frameworks, but others has different approaches....
Security Considerations
Security is always an annoyance, on the one hand, you really need to do something about it, on the other hand, the moment you do, the entire application breaks. It is often useful to build the security infrastructure at the first stages of the application, but that is not always possible. For various reasons, I just now got to the point where I could deal with security, and I had a radically different set of requirements than I had at the beginning of the project. That left me wondering how to ensure that I didn't forget to add security...
Generic Entity Equality
Just got bit by reference equality vs. value equality again. Since this is the third time that I am writing this base class, I decided to put it in the blog for future reference (I wrote it first and forgot to overload the == / != operators). By the way, this is the perfect example of a Mixin. Updated: Ben Scheirman reminded me that I need to handle trasient objects as well. ...
Repository<T>, Syntactic Sugar and DDD
I am toying around with several ideas about the use of Repository<T> in DDD context. Right now it is theoretical because my current project has little need for DDD. At the moment, I am using the static access for Repository<T> as my gateway for the repositories.