+972 52-548-6969
The Law of Conservation of Tradeoffs
Every so often I see a group of people or a company come up with a new Thing. That new Thing is supposed to solve a set of problems. The common set of problems that people keep trying to solve are:
- Data access with relational databases
- Building applications without needing developers
And every single time that I see this, I know that there is going to be a catch involved. For the most part, I can usually even tell you what the catches involved going to be.
It isn’t because I am smart, and it is certain that I am not omnificent. It is an issue of knowing the problem set that is being set out to solve.
If we will take data access as a good example, there aren’t that many ways tat you can approach it, when all is said and done. There is a set of competing tradeoffs that you have to make. Simplicity vs. usability would probably be the best way to describe it. For example, you can create a very simple data access layer, but you’ll give up on doing automatic change tracking. If you want change tracking, then you need to have Identity Map (even data sets had that, in the sense that every row represented a single row :-) )
When I need to evaluate a new data access tool, I don’t really need to go too deeply into how it does things, all I need to do is to look at the set of tradeoffs that this tool made. Because you have to make those tradeoffs, and because I know the play field, it is very easy for me to tell what is actually going on.
It is pretty much the same thing when we start talking about the options for building applications without developers (a dream that the industry had chased for the last 30 – 40 years or so, unsuccessfully). The problem isn’t in lack of trying, the amount of resources that were invested in the matter are staggering. But again you come into the realm of tradeoffs.
The best that a system for non developers can give you is CRUD. Which is important, certainly, but for developers, CRUD is mostly a solved problem. If we want plain CRUD screens, we can utilize a whole host of tools and approaches to do them, but beyond the simplest departmental apps, the parts of the application that really matter aren’t really CRUD. For one application, the major point was being able to assign people to their proper slot, a task with significant algorithmic complexity. In another, it was fine tuning the user experience so they would have a seamless journey into the annals of the organization decision making processes.
And here we get to the same tradeoffs that you have to makes. Developer friendly CRUD system exists in abundance, ASP.Net MVC support for Editor.For(model) is one such example. And they are developer friendly because they give you he bare bones of functionality you need, allow you to define broad swaths. of the application in general terms, but allow you to fine tune the system easily where you need it. They are also totally incomprehensible if you aren’t a developer.
A system that is aimed at paradevelopers focus a lot more of visual tooling to aid the paradeveloper achieve their goal. The problem is that in order to do that, we give up the ability to do things in broad strokes, and have to pretty much do anything from scratch for everything that we do. That is acceptable for a paradeveloper, without the concepts of reuse and DRY, but those same features that make it so good for a paradeveloper would be a thorn in a developer’s side. Because they would mean having to do the same thing over & over & over again.
Tradeoffs, remember?
And you can’t really create a system that satisfy both. Oh, you can try, but you are going to fail. And you are going to fail because the requirement set of a developer and the requirement set of a paradeveloper are so different as to be totally opposed to one another. For example, one of the things that developers absolutely require is good version control support. And by good version control support i mean that you can diff between two versions of the application and get a meaningful result from the diff.
A system for paradeveloper, however, is going to be so choke full of metadata describing what is going on that even if the metadata is in a format that is possible to diff (and all too often it is located in some database, in a format that make it utterly impossible to work with using source control tools).
Paradeveloper systems encourage you to write what amounts to Bottun1_Click handlers, if they give you even that. Because the paradevelopers that they are meant for have no notion about things like architecture. The problem with that approach when developers do that is that it is obviously one that is unmaintainable.
And so on, and so on.
Whenever I see a new system cropping up in a field that I am familiar with, I evaluate it based on the tradeoffs that it must have made. And that is why I tend to be suspicious of the claims made about the new tool around the block, whatever that tool is at any given week.
Comments
Reinventing wheels of any kind is extremely common. There is some kind of religious movement in developers' world making us think, that there is a Holy Grail which allows us to build the ultimate solution. I'd add that the same movement creates false truths as 'ORM are horribly slow', 'I can do it better ' (in terms of log4net) and so on
Sounds like my team lead, I wrote to Oren a few weeks ago...seems his solution is always better than what is already is there... very frustrating!
I agree with points above for sure!
Sorry a little unclear with my above post... my team lead thinks his idea is always better and is scared to use anything ORM like...
I guess you are talking about lightswitch?
I personally believe we can do better than the current state of art in programming. There will be a new system, which is better for building business / data centric applications. There is currently way too much repitious work for developers. Of course there will be tradeoffs. Most probably losing the ability as a developer to do everything with 'designers'.
@ scooletz - on the flip side, if we didn't reinvent wheels, we'd be driving on stones.
I'm okay with letting a free market decide what tools make it and what tools don't. Of course, as developers, we're all very opinionated about what tools should make it and which ones should die a horrible death. Tools and frameworks like Webforms, Zope, and Rails, and Naked Objects all have their grand promises and very real drawbacks. The key to being a savvy developer is recognizing them quickly and accepting or rejecting them based on the right tradeoffs for the current problem domain. Sometimes, you go with a technology not because it's the best, but because there's a huge job pool or technical resources available. Sometimes you hold off on a really promising technology because it's not mature enough for your risk tolerance yet. And sometimes, you stick with what you know because you know it well because as Phil Haack claims, "We're not here to write software, we're here to ship products and deliver value. Writing code is just a fulfilling means to that end".
Para-developer... I think better term would be Quasi-developer :)
This is due to people watching too much Star Trek where they say stuff like "computer, simulate a space shuttle in the holodeck" and boom, there it is. A fully functional space shuttle.
Rene,
It's not just Lightswitch, I believe Oren is referring to the fact that the "easy" stuff is easy, even for the most junior developers as long as they have even average (or barely below average) skill.
The problem is, nothing is as easy as people think once you get beyond the simplest of applications. So these tools (be it data access or coding tools used by Quasi-Developers) rarely have much value since they, in themselves, are only doing the "easy" work.
Once you get a semi-complex scenario, toys that do data access or generate code mostly end up causing more problems then they solve.
@Steve,
I cannot agree more with the fact that you cannot solve complex problems with easy tools. And that even trying to use those tools essentially set you off in the wrong direction making it even more complex and hard to maintain.
I am currently working (with a team) on a framework in C# for building complex, datacentric business applications. The framework lets one define a UI in an abstract sence, so no designers (yet :)). Also data and functions are abstracted and can be run on every tier. There is some resemblance with lightswitch, but I think we are more flexible and open. The tradeoff is that is more difficult to develop a system in this way. But so far we have come a long way.
Alex, how about "undeveloper"? ;-)
The issue is complex systems are just that, and while you can shift the burden of that complexity around, it cannot be eliminated. The problem in attempting to 'hide' complexity to shield a large percentage of developers (para-developers) from it is it means you are making the system way more complex for some other percentage of developers (framework / product developers).
The other effect is the process of 'hiding' complexity invariably involves limiting a system's exposed capabilities - you see this in discussions such as Oren's [ORM] comments about using repositories versus just using the session directly, or in architectural decisions driven by the question of how best to utilize a given mix of junior and senior developers.
Once you understand you can't eliminate complexity from systems then it's more a matter of determining who really needs to be shielded from it and onto whom the burden of complexity is going to shift - and at what cost. For me it's a more a matter of resource utilization / optimization, and one in which the business case needs to be examined very carefully before embarking on grand efforts to shield the masses from inherent complexity.
It could be called "The Law of Conservation of Essential Complexity" as well.
As an optomist, I'd say the issue is that no-one has found the answer yet, not that it doesn't exist !!!
OO to Relational databases is just ONE of the impedendance mismatches we face on a daily basis...
We have the whole language mismatch mess of T-SQL -> C# -> XML -> XSL -> HTML-> CSS -> JS Each language interface is an impedence (I'll exclude Linq here, as IMO that is yet another sublanguage thrown into the mix :( )
While we are on the subject of language impedence, probably more important is the interface between "operational person" to developer.
Dig further and let's not forget classic OO doesn't solve the impedence mismatch of real world vs computer models of objects over time (real world things change over time, without being versioned controlled, IMHO computer science OO works for things that have lifecycles that are measured in the lifetime the program.)
Then when simple things like why the heck when using new() {something = "fred", somethingElse = "bob" } the seperate parts are comma seperated, not semi comma seperated is completely beyond me. (Whilst we are on the subject... why require ";" FFS that was required when compilers needed every line terminated to gain the last ounce of a z80 chip... )
It feels to me at least we are operating in Ptolemy's realm... one day a modernday CS Copernicus will come along....
....well as long as people keep searching for it!!
Oh one more impedAnce is my lack of spelling ability...
still optOmistic though, that Oren's rewritten blog software will have spellchecker :)
I think the software will evolve to something like living creatures. We can see too much similarity, too much similar issues and solutions. Encapsulation (cells), Inheritance etc. Denormalization (actually brain stores data denormalized too)
It's bad we do not have enough time to study biology. I believe we could learn a lot from that.
@Daniel
Interesting thought... Biology values heterogeneity, the other sciences (esp. math) like homogenius solutions...
Yet another impedance :)
Si
It's just natural evolution. Read the history of Computer Science, we started with machine language. Abstracted that into a more usable set that a larger amount of people could understand. Rinse and repeat.
The key here is the free market knows that there aren't enough great technologists to serve the demand for solutions that exists. We are going through a revolution equivalent to the Industrial Revolution right now. Developers must think differently and build tools that let less talented and skilled people accomplish something of value. Why? Because there aren't enough smart people to build all of the applications the world needs.
I very much agree with this post, having the experiance of working with a tool like you're describing (PNM Sequence if anybody is interested).
One fact i think you didn't mention, is that sometimes those tools have very poor performance, or cause a lot of security risks for the application.
The even bigger problem, is that because real developers are not always invloved with the development, it could be that no one will realize the security threats.
The software i worked with, is a BPM tool that is actually very successful, and it gives you a designer to design a buisiness proccess, including desiging screens. It also lets you write javascript for those screens (in an ugly and anoyying notepad like window). The big security problem, is that the "para-programers" as you call them, had to do validation on the input data!!!!
of course they can only write javascript!!!
i guess every developer can see the big security issue with with validating only at the cllient side, but because no real developer had ever looked at whats going on there, no body even realized there is a problem. everyone just saw how the input is validated correcty...
Comment preview