http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Ayende, Thanks for saying what some us were thinking even in the face of popular opinion and potential criticism. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment21http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment21Fri, 23 Jan 2009 09:42:47 GMTI'm glad, and pleasantly surprised, to hear you say this. I think you created some waves and thought process in some who, may be, a bit to 'rigid' in their approach and hopefully this can help many see the 'bell curve' in 'methodologies'. This is the type of 'master of the art' blog post I prefer seeing from 'popular names' out there - such as yourself; where we demonstrate that there is no magic formula, that it is not black and white, and success is not measured by things done or not done, but by final results - and how you obtained those results is irrelevant. But in order to 'get' this concept, you have to be able to see the forest through the trees, and that I'm afraid too few people are capable (or willing in several cases) of doing that. I can see this by responses implying that you were only successful in that project with tests because of the 'process' you followed where 'change was not allowed' per se. I myself was very successful on a larger waterfall project (nearly 3 years) that had 0 unit tests but a crap load of refactoring along the way; was it successful because our major chunk of refactoring (which was done by cut and paste btw...not ReSharper or similar tool) was 'collapsing similar child methods to parent'? or was it the fact that we did all our design up front before starting code? Or maybe it was just the people? Or maybe it was combination? Some on the team knew, and were advocates of TDD, but we didn't go that route and still succeeded. And we've all gone on to have successful projects with a variety of processes, and spectacular failures using the same process as the ones that were success. Some had tests, some did not on both ends of the spectrum, and in the end, the projects that were complete failures - tests and whether we used TDD, XP, Waterfall, or PP made little difference. It was the people who did. But then myself, and the people I know are 'nobodys' in this industry - so who would listen to us? Thanks for being our voice on this one Ayende! http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment20http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment20Wed, 24 Dec 2008 04:00:18 GMTAh OK ! I just read your post here: [ayende.com/.../...ose-principle-as-applied-to.aspx](http://ayende.com/Blog/archive/2008/12/23/composite-architecture-the-open-close-principle-as-applied-to.aspx) Growing a project like a tree, by adding layers without changing the core... Zero tests because of zero refactoring because of uncompromising Open-Close Principle... That's an interesting kind of extreme programing. And by the way, I'm rather happy to be the one that triggered the unveiling of the solution of "the-less-you-write-tests-the-more-successful-your-projects-are" enigma. Billions have beed saved to the whole industry :) http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment19http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment19Tue, 23 Dec 2008 09:45:17 GMTDenis, "you won't be able to ship periodically at sustainable costs and quality without relevant engineering practices like unit tests, refactoring and continuous integration" Yes, I would. I am not talking in the air, I have the practical experience and two years of a running project to back me up on that. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment18http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment18Tue, 23 Dec 2008 00:57:48 GMTI completely agree with your post. The FACT is that lots of projects are successful without test. A lot of people prefer to test because we feel it makes our software better. Not all people would agree. I’m quite certain I’ve read this somewhere… “Individuals and interactions over processes and tools” If you want to cram testing down people’s throats, you will get shit test. That’s just the way it is. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment17http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment17Mon, 22 Dec 2008 20:09:16 GMTAre you implying that the customer is solely responsible for verification that the software product works? http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment16http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment16Mon, 22 Dec 2008 17:11:58 GMTYour post is going to be interpreted as "All you have to do to succeed in being agile is to get customer feedback and ship often", or in other words, "willing to be agile is sufficient to be agile". Unfortunately this is simply wrong because you won't be able to ship periodically at sustainable costs and quality without relevant engineering practices like unit tests, refactoring and continuous integration. I tried the two ways, the one with relevant tools is clearly superior in terms of customer satisfaction. But it's far from being the more common one. And this is rather disapointing because you are read by a lot of developers... http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment15http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment15Mon, 22 Dec 2008 16:22:34 GMTLong time here is 2+ years. And no, off the top of my head I can't think of an OSS example. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment14http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment14Mon, 22 Dec 2008 12:10:50 GMT"And it was big, extremely non trivial, had a reasonably sized team and went on for a long time" What is "long time" for you ? Can you give us an open-source example ? The example you give, Rhino Igloo, as far as I can see by checking it out, is a very small project that doesn't fall in this category. In my experience, what prevent a software from being shipped often (with few bugs and without increasing dev costs) is the combinatorial explosion of sub-components features. When new developpers enter the team or when tired developpers change such sub-components in order to implement new features, bugs occur. Some of these bugs may be detected early by unit tests on sub-components. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment13http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment13Mon, 22 Dec 2008 10:43:45 GMTJoakim, Bare necessities of agile? Ship, often, and get customer feedback. That is it. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment12http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment12Sun, 21 Dec 2008 20:48:51 GMTInteresting you say this - all good. I heard a comment recently 'the whole point of doing asp.net mvc is to unit test - where are the tests...' To me, that isn't the 'whole point' - mvc is an architecture. Sorta in the same light as people saying 'I use IoC to help in testing' I use mvc, ioc, etc.. because I consider them to be good proven architectual approaches to building my code, not just because it's easier to test. I do like tests, but not in the approach of 'I must have 99% coverage' - but more as a tool to help in design and prototyping. Especially with domain/service logic where I want to run it outside of the UI. - Firing up the web page, clicking 3 times to execute Service 'xyz' ends up taking alot of time. Having a test the executes the service is much easier to test and debug. In that regards, it quickly throws out the 'it takes too long to build tests', in those situations I end up saving alot of time and the side benefit is others can see what the service does by looking at the test. So those are the benefits of testing I think. Not a 'every single function in your app needs a test' mentality. Sure that is good if you have resources, I think - but not always possible Good to see your post on this with that attitude, to me - agile is about the customer and iterative development where they are making decisions, seeing the progress - with many releases. An attitude of 'this is their software, not mine'. This is much better than a waterfall, 'we'll show you when it's done in 6 months, hope it is what you wanted' . I've seen that - it never works. Currently I'm a project that meets regularly with the customer, many reviews/feedback and we have built and deliver every week what they want. Seems to be working to their benefit - so ... it's about delivering what they want, and at the same time, they learn that it takes time to develop good quality stuff, and they appreciate it and are able to help make good choices along the way. Last place I worked, wasted 6 months without writing any code just talking about what the customer wanted - it was a mess. Over analyzed, over thought, never delivered http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment11http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment11Sun, 21 Dec 2008 19:41:22 GMTI am using TDD for the last six months and I could see that my softwares are getting much better and lots of problems I am getting on the tests. Another interesting thing I can see is that when a new developer is allocated in the project, he can understand it better when looking to the tests. About time, I am not sure if TDD spends more time. All the time you spent creating the tests is saved during the maintenance phase. I agree with you about your metric. Shipping the software very often is the best way to detect defects. As people above me said, "there's no magic formula", and TDD WILL NOT save all your projects! http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment10http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment10Sun, 21 Dec 2008 14:11:47 GMTWell, sometimes you're just lucky I guess... :-) Of course you have a point, but what I also was trying to point out was that it still does not say much about what "agile" is or is not. You will always be able to find exceptions to the rule, but what are the main criterias for deciding if a project is agile or not, "the bare minimum aspects of Agile project" to quote your introduction? Wether you choose to interview prominent "agile gurus" or survey (succesful) agile projects (like Scott Ambler does, who coincidentally mentions testing as one of perhaps four core tenets of agile) I think you will find that testing is a core practice. Agile schmagile - why does it matter? I think it matters because _most_ agile teams will probably find it difficult to succeed without focusing on testing. Agreed? :-) http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment9http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment9Sun, 21 Dec 2008 14:05:43 GMTOh, And it was big, extremely non trivial, had a reasonably sized team and went on for a long time http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment8http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment8Sun, 21 Dec 2008 13:53:16 GMTJoakim, I wasn't clear. That WAS an agile project. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment7http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment7Sun, 21 Dec 2008 13:50:52 GMTI actually think you are confusing two things here - agile and "succesful" is not necessarily the same thing. There are lots of succesful software projects with no automated [unit] testing (I guess that's what we're talking about?) what so ever and I guess a lot of people now doing and evengelizing TDD have managed to deliver good software before they discovered TDD as well. If you do agile software development in a non-trivial domain, with more than a few developers and over a not so short time period, I think that some level of automated testing will be the most cost efficient way to go as it is either that or a lot of manual downstream regression testing that ensures that features checked off in earlier iterations doesn't break. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment6http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment6Sun, 21 Dec 2008 13:49:29 GMTI am a fan of releasing often. Theer is in average, more than one version of NDepend per month: [http://www.ndepend.com/ReleaseNotes.aspx](http://www.ndepend.com/ReleaseNotes.aspx) One secrets is code reviewing of changes since last version: If you release often changes are epsilon and easy to review. Moreover you can count on the fact that code not touced is working in production, else users would complain.I wrote about this: How to avoid regression bugs while adding new features [searchwindevelopment.techtarget.com/.../...00.html](http://searchwindevelopment.techtarget.com/tip/0,289483,sid8_gci1285059,00.html#) Another secret: make sure that changes are well covered by automatic tests. I wrote about this too: Are you sure added and refactored code is covered by tests? [codebetter.com/.../...ode-is-covered-by-tests.aspx](http://codebetter.com/blogs/patricksmacchia/archive/2008/07/10/are-you-sure-added-and-refactored-code-is-covered-by-tests.aspx) These features were developed primarily with the dog-fooding use in-mind. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment5http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment5Sun, 21 Dec 2008 11:37:52 GMT@Klaus, I disagree ... TDD development doesn't magically make "inexperienced" developers "experienced". If you are worried about needing tests because your developers are "inexperienced" ... shouldn't you be worried that the tests those developers would have to write could be wrong? Tests are simply code and as with any code they can have defects too. My thoughts on the topic at hand is; If you want tests, write tests, if you don't, don't. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment4http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment4Sun, 21 Dec 2008 10:26:14 GMTI think in many ways you are right about using testing when needed. But I think that we need to remember that you are an expert software developer, with much experience. This aspect is good to remember, since I hav earlier met experienced software developers who could skip testing and make it OK. As i see it two import aspects of testing are "built in" to expert software developers: 1. Designing APIs, when you have developed many APis before - you know how to do it. 2. Affirmation in developing. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment3http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment3Sun, 21 Dec 2008 09:48:04 GMT@Jon, I think you hit it right on the head when you said "There is no magic formula in software development...". http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment2http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment2Sun, 21 Dec 2008 05:11:24 GMTI am a huge proponent of TDD and writing tests. I talk about it all the time. But at the same time, I still think you need to evaluate whether it is worth it to write tests in certain cases. If I have some code that is really simple, is not going to change much, and does not have a lot of other code dependent on it, I don't know that it's worth taking the time to write tests. Like you said, a lot of UI code falls into this category (that is, the UI code that you are actually able to test). Like anything else, each situation needs to be reviewed individually. There is no magic formula in software development that is the best choice in every situation. http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment1http://ayende.com/3756/the-tests-has-no-value-by-themselves-my-most-successful-project-didnt-have-any-tests#comment1Sun, 21 Dec 2008 03:46:45 GMT