http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Ayende, Just one brief point. There is a huge difference between Steve Yegge's scenario and yours. Namely that Steve wrote and maintains every one of those half-million of his lines of code all by himself. If I'm understanding you correctly, this is definitely not the case for you with Castle, NHibernate, Boo, and Rhino Tools. You can argue that if Steve was a better programmer it wouldn't have taken him half a million lines of code to write his app the first time, regardless of language choice. But without an intimate familiarity with the source code, I think that's an unfair, maybe even naive, criticism for you to make. http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment23http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment23Mon, 07 Jan 2008 21:53:24 GMTEven when I think that LOC can indicate complexity I'm afraid to put a number of when the code is reaching some type of maintainability barrier. There is countless examples of code (small code) that is totally screw and work on it can be your worst nightmare. At the end architecture of the solution is what will make your code scalable and maintainable at the same time. The concepts are the same and have been the same for a long time, make small methods that do one thing and one thing only, separate concerns, divide your code in namescape/classes/libraries in logical ways. (ex: don't put a file handling class in the same namespace of your business rules for creating a member). Avoid circular references,etc. Some of this practices will make your code smaller, but also some of them may create more lines of code. Good post. http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment22http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment22Sun, 30 Dec 2007 08:38:22 GMTLong time has passed since LOC is not considered a metric for code complexity anyway. One way to handle complexity is to break it in smaller pieces by using packages or namespaces ( ir fyou preffer ) to organize your classes according to the role they assume in your system. http://hcmarchezi.blogspot.com/2007/08/packages-tool-to-organize-classes.html http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment21http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment21Fri, 28 Dec 2007 19:47:31 GMTRegarding dealing with all the lines of code at once, We have to be careful to distinguish between when the _IDE_ would need to deal with all the LoC at once - such as during a project wide refactoring - and when the _developer_ has to (which should be, hopefully, never) Just to be clear, I think your discussion - architecture is more important than language choice - is a better one than Steve's. Much more interesting and better potential to yield useful results, if you ask me. So I'm all for it. I just don't think it is necessarily the same discussion that Steve is holding. /Mats http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment20http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment20Wed, 26 Dec 2007 17:02:58 GMTYes I agree with your point that good structure makes the difference, not the language. But, "What Steve is saying, repeatedly, is that the _existence_ of those lines of code bothers him. What I am saying is that if you need to deal with all of them at once,..." Exactly - and Steve never talks about how all these lines of code would be bad only if you had to deal with them all at once. He never assumes anyone would have to do that. Just as you say, it is the _existence_ of all those lines of code that bothers him, even after they have been properly architected along your suggestions so that you never have to deal with it all at once. The argument is about whether a dynamic language will really dramatically reduce your LoC (I doubt that) and whether giving up compile time type checking to potentially reduce LoC is a good idea (I seriously doubt that). http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment19http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment19Wed, 26 Dec 2007 16:53:22 GMTMats, What Steve is saying, repeatedly, is that the _existence_ of those lines of code bothers him. What I am saying is that if you need to deal with all of them at once, you are in a bad shape not because of the language, but because of your project structure. If you need to move mountains of code, you lost. Trying to run to a dynamic language to reduce your line count is just postponing the inevitable. Good structure makes the difference, not the language. http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment18http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment18Wed, 26 Dec 2007 16:45:35 GMTI agree with your points but I don't agree they address Steve's concern. Steve spends a large part of his post arguing that if you have a mountain of dirt (code), big bulldozers and other machines (architecture and IDEs) is what will allow you to work with this mountain at all. But Steve's suggestion is to try to avoid having a mountain of dirt, thus avoiding (to some extent) the need for heavy machinery (architecture/IDEs) to move it around. When you (Ayende) describe how you relatively effortlessly jump around in your fairly massive code base, looking at cohesive modules which are easily grasped in isolation, you are not describing the scenario that Steve forgot about - you're describing the same scenario he is, only he doesn't consider it a rosy picture, he thinks you and I are in denial about how hard it really is. In Steve's post, he expresses the point of view that a large code base is _not_ the inevitable result of a large set of requirements. It seems he thinks dynamic languages offer a way out. I don't agree with Steve at all, but the points he invites to discussion around is whether static typing blows up your code base with additional lines of code that can't be motivated by the added safety offered by compile time type checking, given that they also (hypothetically) add substantially to the " mountain of dirt". While he doesn't make the point explicitly, I suspect that he leans to some degree on the idea that unit tests relieves the compiler of a lot of its type checking duties. Personally I can't write three lines of code without the compiler letting me know about some mistakes I've made, and realizing how that holds just as true when I'm writing my unit tests, I'm all for static type checking. /Mats http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment17http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment17Wed, 26 Dec 2007 16:28:11 GMTWithout NDepend, and at least for your "framework" code, you can see what Ohloh thinks of your codebases. Just a random example - you said Boo was 212KLOC. Ohloh says between 100K and 120K, depending on comments and whitespace: http://www.ohloh.net/projects/3515/analyses/latest http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment16http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment16Wed, 26 Dec 2007 11:29:41 GMTThat is a beautiful diagram. http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment15http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment15Wed, 26 Dec 2007 00:42:08 GMT"The key hole is that you only need to understand a single part of the system at a time. You will probably need to know some of the infrastructure, obviously, but you don't have to deal with it." I call this partitioned or compartmentalized complexity. It's not new with you or I. I could find a few references I could quote. It's hard concept to grasp until you've either experienced it (and noticed) or studied design or architecture (ie.. book knowledge). A lot of people have neither. http://tech.groups.yahoo.com/group/altnetconf/message/6490 Roger Sessions has some interesting work in the EA space on this same topic (where he approaches it mathematically). http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment14http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment14Tue, 25 Dec 2007 23:57:59 GMTI liked Frans's response, but I think yours nails it even better. I think Steve and Jeff are confusing capacity with scalability [1]. As you note, scalability has to do with the architecture, not the language. The architecture impacts whether the scalability curve goes up, down, or is linear. The benefits of a dynamic language only help with the steepness of the line, or the "capacity" of the language to express things concisely. [1] http://www.pervasivecode.com/blog/2007/11/13/capacity-vs-scalability/ http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment13http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment13Tue, 25 Dec 2007 22:07:12 GMTGreat post, Oren. I like hearing from you and Frans because you both regularly work on huge code bases and can speak from first-hand experience. http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment12http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment12Tue, 25 Dec 2007 20:59:06 GMTJohn, I am aware of the accuracy issue with date.Now The timespan that I am testing right now is 10 seconds, which is long enough to make accuracy pretty much non relevant. About the line count, I am trying to hit ballpark numbers here, obviously comments, linebreaks etc are adding to the score, but it is also the simplest way to get it checked. http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment11http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment11Tue, 25 Dec 2007 19:44:53 GMTI'm glad I'm not the only one with this reaction, great post Ayende. The thing that gets me is this whole "no code is the best code" school of thought. Because "no code" doesn't mean "no code, it means "someone else's code". And while that may get you to 80% or even out the door faster, it is going to make customization and maintenance much more difficult. And the notion of quoting those codebase sizes blew my mind as well. One of the central tenets of coding in particular and life in general is decomposing the problem into smaller digestable chunks. Merry Christmas! http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment10http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment10Tue, 25 Dec 2007 19:25:24 GMTI built revision 3192 of NHibernate's trunk, and let NDepend analyze it (NHibernate.**.dll and Iesi.Collections.Test.dll). The result* was as follows: Application Metrics Number of IL instructions: 474318 Number of lines of code: 73035 Number of lines of comment: 41303 Percentage comment: 36% Number of assemblies: 26 Number of classes: 2192 Number of types: 2489 Number of abstract classes: 106 Number of interfaces: 211 Number of value types: 86 Number of exception classes: 42 Number of attribute classes: 96 Number of delegate classes: 15 Number of enumerations classes: 76 Number of generic type definitions: 35 Number of generic method definitions: 90 Percentage of public types: 93.61% Percentage of public methods: 85.28% Percentage of classes with at least one public field: 5.79% *) For those who don't know about it already, NDepend is an incredibly useful tool that does a lot more than provide you with statistics. Give it a try, you won't regret it! http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment9http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment9Tue, 25 Dec 2007 19:21:40 GMTAyende, A couple points. First, I think they are discussing lines of code as it relates to operations, not really the comments and other fill which isn't code. Counting lines of code the way you provided will obviously lead to inflated numbers. Without a clear understanding of what each person is counting it becomes very difficult to compare. Second, I think the original articles were talking about one person maintaining a project of that size. Jeff alluded to the fact that developers working on a project of this size usually work on a team or as he put it open source it to receive help from the community. More people certainly makes the task more manageable. Lastly, this is just a side point, but I would recommend against using the DateTime class for your max duration testing. If you are testing for long durations that should be fine, but DateTime is extremely inaccurate. I disccovered this myself earlier and wrote the following blog post about it: http://jaychapman.blogspot.com/2007/11/datetimenow-precision.html I then found the StopWatch class which solved my timing issues. If you want accurate time testing use that class instead. http://jaychapman.blogspot.com/2007/12/datetimenow-precision-issues-enter.html I realize that you are probably just showing an example here, but it may be helpful for other people who do not know of the StopWatch class. I know I didn't when I started my duration testing. http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment8http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment8Tue, 25 Dec 2007 19:08:31 GMTPatrick, I measured line count the simplest way: for file in GetFiles(): count += File.ReadAllLines(file).Length Not really nice, but it is quick & dirty solution http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment7http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment7Tue, 25 Dec 2007 13:56:26 GMT * Castle: 386,754 * NHibernate: 245,749 * Boo: 212,425 * Rhino Tools: 142,679 * LLBLgen : > 300K Ayende, I am surprised by these numbers How do you count your number of LOC? As I commented on the Jeff and Frans blog: "Developpers always claim that they are maintaining much more lines of code than they actually do. Recently I was consulting on a large project, they said 4M LOC, but it was actually 700K LOC (measured with NDepend). The difference comes from the way you're measuring LOC and I detailled here how you should do in the .NET world:" codebetter.com/.../how-do-you-count-your-number-of-lines-of-code-loc.aspx When NDepend measure the number of LOC of NDepend it says 53K LOC by now and it contains a CQL compiler and editor, an IL analyzer, and a lot of WinForm GDI stuff. It might sounds ridiculously small, but still, I consider it as a huge and complex project! . http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment6http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment6Tue, 25 Dec 2007 13:53:41 GMTThe 500kloc project is a game that he was working on singlehandedly, the projects you mentioned are for the most part are open source, which i think is a totally different thing. I think the point he's trying to make is that for one person handling a project alone code size does matter because you can keep track of more at a single time. Especially when it is feature complete and all the work left to do is bug fixing, it can make solving these issues easier and quicker. http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment5http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment5Tue, 25 Dec 2007 12:00:27 GMTWell, I do the same for the project itself, forcing separation until I can deal with each piece in isolation http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment4http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment4Tue, 25 Dec 2007 11:04:28 GMTWhat I meant is that they are "separated" from the actual (business) projects. Although they have a direct impact on the projects (+ may require development time), translating that into lines of code is a subjective topic without any context. http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment3http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment3Tue, 25 Dec 2007 10:49:45 GMTPierre Henri, I am including the libraries because I am not treating them just as a user, I am also a developer for all of them. http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment2http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment2Tue, 25 Dec 2007 08:37:29 GMTAs you said, the key here is separation of concerns. However, you shouldn't include the LOCs of the libraries you use because they are well-enough separated to not be a concern of your project. If you really want to go that way, you may as well add the LOCs of the BCL. :) By the way, do you know where I could read some details about one of these beasts with millions of LOCs? http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment1http://ayende.com/3070/code-base-size-complexity-and-language-choice#comment1Tue, 25 Dec 2007 07:55:58 GMT