+972 52-548-6969
The missing Linq: Stateful extentions methods
It seems to me that right now there is a lot of exploring going on with extention methods. People are doing some really cool stuff with them.
What I am afraid of is that this exploration is going to hit glass wall the moment people are going to try to do the interesting stuff with them. The reason for this is that beyond the simplest cases, I would really want to have some sort of a state between method calls.
Here is a code sample that demonstrate the issue:
myRandomObject.Tag("Linq ExtentionMethods");
myRandomObject.GetTags();
Tag() and GetTags() are extention methods. The problem is that as it stands now, there isn't really a way to implement this functionality using Linq. You need to keep the tags associated with the object, which usually means that you will use a hash table. This seems like a solution, but it runs into the issue of breaking the GC, since I would like to have the state freed by the GC when the object goes out of scope.
Hashtable based on WeakReference are one solution to this, but this is quite a bit more complex than it sounds, and you needs to scavengar that table every now and then, which means that you can either suffer wasted memory(bad) or have a separate thread that will clean the table (bad).
I can see two solutions to this problem. Either adding another field to System.Object "IDictionary ExtendedProperties" or supplying a weak hashtable implementation that ties into the GC in the same manner that WeakReference is there. The first solution is the simplest, but it is tying object into IDictionary, as well as increasing the size of System.Object significantly. The second solution is more complex, but it is much prefered, since it doesn't affect anything else.
The problem is that both of those requires support at the framework level, it is not something that can be done externally.
Comments
I don't think extension methods are really for this particular case, as you then could argue: write a normal method.
Extension methods aren't usable as methods weaved in with AOP, they're just a compiler trick.
Agreed, they should have simply accepted that AOP is great and that they needed to add support for that at the core, CLR level. Now they avoid doing any AOP but stuff elements of it into the language here and there, just to support Linq, however they forget to look a bit further and take that extra step so other features also could be supported.
Heijlsberg doesn't really like AOP, and IMHO that's also the reason why we're stuck with silly constructs like this.
Another one is cramming Linq, which is a DSL into a language, C# which isn't compatible. You then have to butcher the OO language with non-OO constructs to make the non OO language, Linq, fit into the host language. That won't work unless you want to close your eyes for the mess that is needed to make it all happen. A DSL awareness inside C# would have been much better and Linq would have been possible without problems, AND on top of that, other languages would be able to be added to C# as well, for example a simple rule language for BL rules.
I'm not sure this is a problem best suited for extension methods.
A simpler solution would be to create an AttachedState class, and use that to bind the values to the objects in question. You could even write an extension on object that took the value and the AttachedState instance, if you wanted.
But it sounds like what you want more would be something closer to extension properties.
@Frans,
I know that Extension methods are not for this.
But as they are the only extension mode that I have for classes short of deriving (not practical in many cases), I am very interested in this scenario.
I don't think that AOP is relevant here. What you are talking about is open compiler vs. closed compiler. And open/closed here are not about Open Source or not, it is about whatever you can modify the way the compiler is dealing with your code.
Boo's compiler is open, and the using statement is actually a user-mode extension, not a compiler hack.
An open compiler can be used for AOP, certainly, and it can do some fascinating stuff, but having the ability to handle DSL is a killer here.
You can argue that set based operations aren't natural to imperative language like C# are wrong, and complex Linq queries certainly feel hacked together.
Can you clarify what you call DSL awareness? It is similar to what I consider an open compiler?
@Jacob,
Extension properties and events would be _wonderful_, yes, but that is not what I am talking about.
About the AttachedState class, how do you associate the state with the object in such a way that it is available as long as the object is not GCed, but freed when the object is GCed?
About extension methods: I agree that this is about the only thing which looks like it could have been great and really useful for extending classes with code without placing that code inside the class. I also find it sad that they're too limited, that the sole purpose of these things being there isn't because of the necessity in general for extension methods (otherwise why did they cancel the extension properties feature?) but because linq needed it. Which is IMHO a stupid reason.
About DSL awareness: when you go from one language to another in a single scope, you have to transfer state from one compiler to another, so the second language is aware of its surroundings, or better: has a realisation of the scope its in. This also is reversable: when you move from second to first language, information necessary for the scope are transfered back. This way you can compile both languages with 2 compilers and result in working code without butchering the first language.
So they could have made C# DSL aware in such a way that when a foreign language block was entered, the compiler for that language would take over and know it was in a given scope, which symbols were reachable and usable and for example at runtime which elements are changable / passable between languages.
It then could be possible to use native SQL as DSL inside C# instead of Linq and still be type safe and have compile time checked code.
It's complex to do this, and it would have been a real challenge but it would have been IMHO the only way to implement Linq properly, because now with whatever extension they come up with, it will always be affecting C#, which is precisely what you DON'T want, as it will introduce constructs to C# which will be misused in other areas where the second language isn't used. (like with C# 3.5 for example the 'var' keyword)
What you are basically saying is that you would like some sort of a way to tell the C# compiler, give this chunk of code to another compiler to handle, which would process it (and probably the C# AST as well) and then return to the compiler. I am trying to come up with a reasonable syntax that would still allow you to do assignment of results, and I can't really think of anything off hand.
What this really sparks is the idea of forking to a C#-like compiler...
I agree that focusing solely on Linq's needs to extend the language is bad, but it is bad because it is not taken far enough, not because of the implementation corrupts the language.
var is quite useful outside of queries, but again it is very limited to local scope, etc.
I'm going to have to start reading this blog more often, lots of great topics that really make me think.
I agree with the cool vs. interesting or useful assessment. I'm still trying to wrap my head around extension methods, conceptually I think of them as decorator pattern methods, but I know I'm missing nuance in that categorization.
Comment preview