Ayende @ Rahien

Refunds available at head office

Re: Versioning Issues With Abstract Base Classes and Interfaces

Phil Haack is talking about why the MS MVC team changed IHttpContext to HttpContextBase. I follow the argument, but at some point, I just lost it. This, in particular, had me scratching my head in confusion:

Adding this method doesn’t break older clients. Newer clients who might need to call this method can recompile and now call this new method if they wish. This is where we get the versioning benefits.

How on earth does adding a new method to an interface would break an existing client? How on earth does adding a new method to an interface require a compilation from client code. Now, to be clear, when I am talking about changing the interface I am talking solely about adding new methods (or overloads), not about changing a method signature or removing a method. Those are hard breaking changes no matter what approach you take. By adding a method to an interface? That is mostly harmless.

The only thing that require that is if you are implementing this interface, not if you are using it. It is possible that Phil is using the term clients in a far wider meaning than I would put on this, but this is not the general use of the term as I see it.

Prefer abstract classes to interfaces, use internals and no virtual by defaults are three of the main issues that I have with the framework design guidelines. I have major issues with them because they are actively harming the users of the framework. Sure, it might make my work as a framework developer harder, but guess what, that is my job as a framework developer, to give the best experience that I can to my users. [Framework Design Guidelines rant over]

Now, I do believe that I have some little experience in shipping large frameworks and versioning them through multiply releases and through a long period of time. I also believe that some of those frameworks are significantly larger and more complex than what the MS MVC is going to be. (Hint, if MS MVC even seems, by an illiterate drunken sailor on a rainy day, to approach NHibernate's complexity, it is time to hit the drawing board again).

And those frameworks are using frameworks to do pretty much everything. And I cannot recall off hand any breaking change that resulted from that. In some cases, where the interface is an extension point into the framework, we have gone with interface + base class with default functionality. If you use the base class, you are guaranteed no breaking change. The reasoning for an interface is that it is giving the user more choice, you aren't limiting the options that the user have when it comes to use this (by taking out the single inheritance slot, for example).

Now, if we analyze the expected usage of IHttpContext for a moment, who is going to be affected by changing this interface? Only implementers. Who is going to implement IHttpHandler? I can think of only two scenarios. Hand rolled test fakes and extending the Http Context in some interesting ways, perhaps by adding proxies or decorators to it.

In the first case, that is no something that I would worry about. The second is far rarer but also the much more interesting case, but those are generally not done by hand (I wouldn't want to type all the methods of IHttpContext, that is for sure). Even if it was, I still have no issue with it. New framework version, add a method. It is not a significant change. A significant change would require me to rework large swathes of my application.

Now, why do I care for that?

The reason is very simple. It is a pain to me, personally, when I end up running into those warts. It is annoying, frustrating and aggravating. I like to be happy, because otherwise I am not happy, so I try to minimize my exposure to the afore mentioned warts. Hopefully, I can make them go away entirely. And not just by pulling the blanket over my head.

Comments

Bob Archer
02/21/2008 06:11 PM by
Bob Archer

How on earth does adding a new method to an interface would break an existing client? How on earth does adding a new method to an interface require a compilation from client code.

Because when you add a method to an interface, all classes implementing that interface must now implement that method! So, code will break unless you add that method to your implementing classes.

BOb

Ayende Rahien
02/21/2008 06:49 PM by
Ayende Rahien

Bob,

You missed the next statement:

The only thing that require that is if you are implementing this interface, not if you are using it.

Implementing IHttpHandler is not a common scenario, and I deal with this in the post.

Kevin
02/21/2008 06:50 PM by
Kevin

As far as my understanding, once the interface is published, it's static. Create new interface with interface inheritance for changes is the way to go.

my $0.02.

hammett
02/21/2008 07:00 PM by
hammett

+1000

Now its abstract class, then they will protected internal by default, and in the end people will get the usual MS tool, while this could be a break from the old habits.

Jay R. Wren
02/21/2008 07:09 PM by
Jay R. Wren

+2000

thanks for the rant. I wanted to write the same thing, but I just couldn't find where to start. Hopefully Phil will figure it out.

Ben Scheirman
02/21/2008 07:21 PM by
Ben Scheirman

@Kevin:

That's how we have DataGrid and GridView, right? Essentially they are the same metaphor, but in order to not break existing ASP.NET sites they just introduced a completely new object.

From a discoverability aspect, this sucks because intellisense shows 2 different types of grids... which one to use? Time to read some docs.

Of course, doing something like System.Web.v3.IHttpContext sends shudders down my spine as well.

Kevin
02/21/2008 08:39 PM by
Kevin

@Ben:

Very valid point for discoverability aspect. If we are talking about Framework that we might extend on, I still prefer interface inheritance. Interface is the contract between framework vendor and user.

As you pointed out on the UI part, it actually makes sense to use System.Web.UI.V3.GridView.

Infragistics uses this versioning strategy.

Thanks for the response,

Kevin

Haacked
02/21/2008 08:43 PM by
Haacked

Oren,

As always, very insightful post. I appreciate your analysis. I'll address these questions more thoroughly in an upcoming post.

Remember, you can't change an interface. You've effectively created a new one when you add a method. So now you've changed the return type of every property/method that has a return type of that interface. Thus other classes that implement interfaces that you wrote may break simply because the reference this type in some way. I have some example code I can show to demonstrate.

Things are a bit different when you don't have strongly named GAC'd assemblies. Every assembly in the Framework are strongly named and GAC'd. The rules change a bit.

As for adding a method, you can add a non-abstract method to an abstract base class and not have breaks for existing clients to that class because they aren't calling that method.

mkuczara
02/21/2008 09:00 PM by
mkuczara

Ayende has right. No one wanted to implement IHttpContext (and thus there is no fear about recompilation) , it was meant to use aas a Type.

Ayende Rahien
02/21/2008 09:18 PM by
Ayende Rahien

If you have two versions of the same assembly with the same appdomain, you are doing things wrong. Majorly wrong.

Furthermore, since you are going to ship as part of the BCL, that is not any of your concern anyway. Inside any ASP.Net app there could be only one type of IHttpContext that will be usable at any rate, so I don't get the point.

Yes, assembly binding can be a bitch, but assembly redirect solve them and you would have the issue anyway.

Since IHttpHandler is not a candidate for mass implementation, this is something that goes even further than that.

And excuse me, but I fail to understand the difference between interfaces coupled to strongly typed assembly versions while base classes are not.

From experience, I can tell you that the runtime treat missing method just the same, and can handle itself just fine in those scenarios.

Mats Helander
02/21/2008 11:08 PM by
Mats Helander

"Prefer abstract classes to interfaces, use internals and no virtual by defaults are three of the main issues that I have with the framework design guidelines."

Right on!

/Mats

Andrey Shchekin
02/21/2008 11:20 PM by
Andrey Shchekin

I think the implementers are obviously a part of the clients and it is understood this way for a long time now. While we can assume no one implements this particular interface, such assumptions are generally costly to MS developers.

Andrey Shchekin
02/21/2008 11:20 PM by
Andrey Shchekin

I think the implementers are obviously a part of the clients and it is understood this way for a long time now. While we can assume no one implements this particular interface, such assumptions are generally costly to MS developers.

Krzysztof Cwalina
02/22/2008 07:05 PM by
Krzysztof Cwalina

"Prefer abstract classes to interfaces, use internals and no virtual by

defaults are three of the main issues that I have with the framework

design guidelines. I have major issues with them because they are

actively harming the users of the framework. Sure, it might make my

work as a framework developer harder, but guess what, that is my

job as a framework developer, to give the best experience that I can

to my users. [Framework Design Guidelines rant over]"

Oren and Mats, the main reason we don’t add members to interfaces is not that it will break us (.NET Framework). Our customers also implement the abstractions we ship and they don’t want to be broken. If they were not implementing them, then why would they demand abstractions?

Secondly, we do want to give you (all users of the Framework) the best experience using the framework. For many people it means compatibility, usability, etc. For many it [also] means extensibility. We can give you extensibility with abstract classes! We want to do it and I think we are doing it. If we are on the wrong track (i.e. the ABC does not work), I would love to understand why.

BTW, I also posted a very long comment on Phil’s blog that touches some of the issues.

Ayende Rahien
02/23/2008 12:07 AM by
Ayende Rahien

Krzysztof ,

See the detailed response to that exact scenario in the post.

Yes, adding memebers to interfaces that you expect the user to implement is a breaking change (imagine what making a change to IDisposable would do).

There are ways around that, either using derived interfaces (IDisposable2) base class that allows point by point extensibility (DefaultDisposable), etc.

I want interfaces instead of ABC because it make my life easier. Yes, it make the framework design harder, but that is a framework challenge, not something that should impose a limitation on the user.

I like interfaces.

They are very explicit in what they do. You can't add a non virtual method there, you can't add an internal method, you can't add behavir.

I want to ensure all of that in any of the things that I have to end up referencing. Because if I would have to write AbstractHttpContextAdapter, I have lost.

Mats Helander
02/23/2008 02:39 AM by
Mats Helander

Hi Krzysztof,

I have left a comment to you in my blog:

http://www.matshelander.com/wordpress/?p=86

/Mats

Krzysztof Cwalina
02/23/2008 03:42 AM by
Krzysztof Cwalina

Oren, IDisposable2 has the problem I tried to explain in the API design talk: http://www.researchchannel.org/prog/displayevent.aspx?rID=11087&fID=2740. The part about interfaces starts at 2:54.40.

The problem is not that we cannot add IFoo2. It's that it would have a ripple effect on other APIs that expose the interface or will force users to do dynamic cast which are not self-documenting (a good principle of API design).

For those who say that dynamic casts are fine, imagine if all APIs in the Framework returned System.Object and you had to down cast it to the actual runtime type to use its useful methods. Would that be a good place to be?

Lastly, by saying that you cannot design good extensibility using abstract classes, you are implying that all the languages that don't support interfaces (C++ for example) are fundamentally flawed from the extensibility perspective. IMHO, I would actually prefer if out type system did not support interfaces at all. We only added them to support pseudo-multiple-inheritance without having to solve the diamond issue.

And so I think the issue is in how we design the abstractions (I am with you that we often make them too ridgid), not that we don’t use interfaces (unless you need MI, but in this case it’s not the case).

Mats Helander
02/23/2008 05:57 AM by
Mats Helander

@Krzysztof

"For those who say that dynamic casts are fine, imagine if all APIs in the Framework returned System.Object and you had to down cast it to the actual runtime type to use its useful methods. Would that be a good place to be? "

No, that's why we have interfaces.

"Lastly, by saying that you cannot design good extensibility using abstract classes, you are implying that all the languages that don't support interfaces (C++ for example) are fundamentally flawed from the extensibility perspective."

No, because C++ supports multiple implementation inheritance and thus doesn't really need interfaces - when you have multiple inheritance an abstract class with only abstract members is as good as an interface.

"IMHO, I would actually prefer if out type system did not support interfaces at all. We only added them to support pseudo-multiple-inheritance without having to solve the diamond issue. "

Interfaces with explicit implementations do solve the diamond problem, in a fairly elegant way.

"And so I think the issue is in how we design the abstractions (I am with you that we often make them too ridgid), not that we don’t use interfaces (unless you need MI, but in this case it’s not the case)."

The fact that you don't use interfaces is the issue with how you design your abstractions. That is what is too rigid about them. And yes, MI (or the lack thereof) is at the core of this issue.

As I try to explain in my blog post, linked above, it seems to me you are rejecting well known OO principles on the basis that "developers don't understand dynamic casting"?

/Mats

Krzysztof Cwalina
02/23/2008 07:01 AM by
Krzysztof Cwalina

Great! I think we are now in agreement:

  1. You agreed that dynamic casts have drawbacks. Otherwise object based (untyped) APIs wouldbe fine. I am glad you agree that they are not fine and we need "interfaces" instead of casting all over the place.

  2. I am glad we now agree that there is no fundamental problem with not using interfaces (C++ does not have them). The problem is that CLR classes don't support MI (as you said "MI (of lack thereof) is at the core of the issue"). I am with you on this. If you need MI, by all means use interfaces. But I thought you guys agreed that HttpContext does not need MI?

  3. We agree that interfaces solve the diamond problem in an elegant way. I just think they solve one set of problems and introduce others (the evolvability problems). I think MI diamond problem can be solved without the need for interfaces.

I will try to respond to your post, but I got to run now and it's pretty long.

Ayende Rahien
02/23/2008 01:37 PM by
Ayende Rahien

I am not sure that IDisposable2 would have a ripple effect, actually.

You would need to evaluate if some things require the new version or no, and obviously you would need to have the Compiler suport it, but beyond that?

That is a non breaking change by defination, so I don't see that.

IFoo2 is an interesting problem. If this is the default approach, then yes, you want to change everything that uses that.

If this is a way to selectively add features (Streams with timeout, for example), I see this as a great way to do this.

I am very concerned by the limitations of the platform that are spawned from "devs can't understand dynamic casts". If the usability study would have shown issues with the while loop, would you remove that as well?

The "everything return object" is a strawman, sorry. There is a huge different between interface and object. Interface is typed. Object is not.

About C++, it has interfaces, it just doesn't call them this way. Purely abstract classes are interfaces.

In C++ you also have multiply inheritence, which allows for much more interesting solutions.

Yes, I would take MI over interfaces in a heartbeat.

Since I can't have that, a base class is a huge problem, it is a tight dependency, it doesn't let me use my own bas class.

Mats Helander
02/23/2008 04:32 PM by
Mats Helander

@Krzysztof Cwalina,

We are not yet in agreement about the point under discussion, but we are making headway towards not talking past each other. Indeed, it is helpful to note that we are more in agreement over some of the terminology than it initially seemed, and that will make fruitful discussion more realistic.

I will address your points in inverse order, starting with what I percieve as the low hanging fruit.

3) I'm really happy we agree here that it makes sense to say that interfaces (with explicit implementation) actually do solve the diamond problem rather than to say that they are a way to avoid having to solve the diamond problem.

However, I'm not sure what you mean when you say you think the diamond problem can be solved in a good way without interfaces....links? If you are referring to existing approaches in for example C++ I would not agree that they are equally satisfying.

(@Ayende, I don't completely agree with you when you say you would take MI over interfaces in a heartbeat...are you assuming an MI implementation that didn't suffer from the diamond problem? Do you have any links to such implementations? What am I missing? Has this been solved in a good way without interfaces?? :-P)

I also don't agree completely that interfaces introduce any so-called "evolvability problems" - (published) interfaces (from real companies like MS, anyway :-P It is not like the interfaces in NPersist have always been rock solid) don't evolve - they are superceeded and become obsolete.

/Mats

Mats Helander
02/23/2008 04:37 PM by
Mats Helander

More for Krzysztof,

2) Well, on one hand I haven't actually taken a lot of interest in the HttpContext example, but on the other hand I don't want you to feel you're fighting a 50-front battle and as soon as you have a good argument your opposition goes "oh, well now, that was never really my argument that you just destroyed there, as such" ;-)

So while I can't really argue about the specifics of the HttpContext case, if I may be allowed the slightly more general stance I would answer: MI is always the issue - even with HttpContext, regardless of whether you see any obvious use cases where someone would want to implement IHttpContext but not inherit HttpContextBase.

Here's the thing: As Jeffrey Richter pointed out in your chapter, inheritance represents an IS-A relationship between two types whereas interface implementation represents more of a CAN-DO relationship. Even when you have MI, stating that A IS-A B is a very strong statement that should not be made when it doesn't actually make sense from the type hierarchy perspective - when it doesn't actually carry some kind of domain-related "truth", such that given the domain to be modelled, an A really IS-A B, not just that they should be able to show up at some of the same functions and galas.

A language such as C++ still offers the possibility to capture CAN-DO relationships via purely abstract classes even if it lacks direct concepts for modelling them. But in a language with single implementation inheritance (SI) and interfaces, this distincion becomes central. In such a language, distinguishing between IS-A and CAN-DO is taken very seriously, to such an extent that true IS-A relationships are restricted to one such relationship per class.

Thus the best way to explain why you should provide interfaces in addition to base classes might be: you should leave it up to me as the developer to decide if it makes sense in my domain that my class which happens to be able to CAN-DO IHttpContext also should be considered to BE-A (IS-A) HttpContext(Base). You shouldn't force me to state that MyClass IS-A HttpContext just because it CAN-DO IHttpContext, because looking at my domain, such a statement might simply very well not be true.

/Mats

Ayende Rahien
02/23/2008 06:42 PM by
Ayende Rahien

Mats,

Given the choice, I would solve the diamon issue with:

public abstract class Foo

{

public asbtract Execute();

}

public abstract class Bar1

{

public void Execute() { // do a }

}

public abstract class Bar2

{

public void Execute() { // do b }

}

public class Fubar : Bar1, Bar2

{

}

new Fubar().Execute();

That is the problem, in short terms.

I would have solved it using the following approach.

new Fubar().Exeucte();// compiler error, ambigious method name Execute on Bar1 and Bar2

((Bar1)fubar).Execute();//works

((Bar2)fubar).Execute();//works

And I would allow to do this as well:

public class Fubar : Bar1, Bar2

{

public void Execute() { ((Bar1)this).Execute(); }

}

This remove the ambiguity and more or less solve the issue, I think

Mats Helander
02/23/2008 09:55 PM by
Mats Helander

Ayende,

Your suggestion does make sense...I like it - know of any platform that does it like that? I can see some reasons why OO purists might not love it and I would personally prefer the interface based approach, but lacking interfaces I could definitely live with what you suggest.

/Mats

Ayende Rahien
02/23/2008 10:39 PM by
Ayende Rahien

Mats,

I just made that up.

Diamond problem is an edge case, and it deserves to be treated as such.

I don't think that this is different much from the interface based approach.

Something else you can do is to use single inheritence, but introduce mixin inheritence.

That should give you a pretty good usage scenario without introducing the problem

Mixin inheritence is something that you can probably deal with on the compiler level, without much issue.

Mats Helander
02/24/2008 03:23 PM by
Mats Helander

Ayende,

I don't see the diamond problem as an edge case, these things have to be tight.

I certainly also agree with your proposal to use mixins, just that I think interface based mixins are the best way to go because, well, the diamond problem ;-) (But yes using your made up syntax would work for that too!)

/Mats

silky
02/26/2008 03:53 AM by
silky

Must say, I think your example of how to solve the Diamond problem is quite stupid.

object and ArrayList both implement .ToString()

class Foo : ArrayList { }

new Foo().ToString()

does it compile? why? why not? do I need to cast it?

Ayende Rahien
02/26/2008 04:00 AM by
Ayende Rahien

Silky,

Since they are both in a single inheritence chain, no, you don't.

If you wanted to do something like:

Foo : ArrayList, Dictionary

new Foo().ToString()

You should get a compiler error, this call is ambigious.

silky
02/26/2008 04:24 AM by
silky

what do you mean "single inheritence chain"?

Foo, in my example, extends both object and ArrayList. Why doesn't it?

what happens if I write

class X : ArrayList, Object

under your system? am i allowed?

introducting casting requirements to use an object orientated language seems bad, imho ...

Ayende Rahien
02/26/2008 04:31 AM by
Ayende Rahien

Yes, you would need to cast under this situation.

Single inheritence chain means that object is parent of ArrayList.

In the latest code, object is the parent of ArrayList but is also a second parent of Foo.

Silky,

foo.ToString();

Which method is getting called? ArrayList.ToString()? object.ToString()?

silky
02/26/2008 04:41 AM by
silky

so if i specifically write object I must cast? that's crazy.

in c# today in foo.ToString() arrayList is called.

but in your proposed c# where objects can have multiple solid parents, how do you determine which is what at runtime?

if you have object a, who is his direct parent? who is is 'other' parent? his 'other other' parent?

Ayende Rahien
02/26/2008 04:48 AM by
Ayende Rahien

Silky,

My proposal have remove the idea of a single parent.

And in my proposal, so it would be.

I think that are confusing inheritance chains with MI.

I would tend to not determain that at runtime, just like I rarely do now.

When you emit a method call, you also emit who the class this methd is on.

silky
02/26/2008 04:54 AM by
silky

Ayende,

As an aside it seems I can't see any of these recent posts in firefox. It's strange.

Anyway, I think you are missing the point of what I am saying. Under your scheme the compiler won't know if it's in the so-called inheritence chain or if it's MI'd. Unless there is some facility to detect that ...

Ayende Rahien
02/26/2008 05:02 AM by
Ayende Rahien

Ha, of course it would.

Foo : ArrayList, object

foo.Count;

You want each parent recursively trying to find a matching method.

If you have more than a single one, you error.

Very simple.

silky
02/26/2008 06:02 AM by
silky

Exactly!!

under

Foo.ToString() there is more then one matching, isn't there.

Ayende Rahien
02/26/2008 09:41 AM by
Ayende Rahien

Silky,

for Foo : ArrayList, Object, yes.

for Foo : ArrayList, no.

silky
02/27/2008 05:36 AM by
silky

how are the different at runtime?

Ayende Rahien
02/27/2008 01:33 PM by
Ayende Rahien

Silky,

On the CLR, method resolution is performed at compile time, not on runtime.

Comments have been closed on this topic.