Ayende @ Rahien

It's a girl

NHProf.com downtime, and using windows as a server system

Yesterday nhprof.com was down, there was a YSOD and when I logged into the server to try to figure out what the problem was, I got this:

image

The server has been running with no issues for over a year, so it is not an issue with not activating after installation. This is a Windows 2008 Server, and I find it incovievable that a server would go down because of activation. To spread more salt on the wound, it wasn’t a problem that could be fixed remotely, I had to call my host and ask them to fix that (they did in a remarkably short time).

Annoying!

Comments

Aleksander
11/30/2009 06:52 PM by
Aleksander

Use an operating system that doesn't require activation. Simple ;)

Arne Claassen
11/30/2009 07:03 PM by
Arne Claassen

Reason why i develop in VS and deploy on Centos+mono

Robert
11/30/2009 07:41 PM by
Robert

@Arne Claassen: How is the performance compared with a Windows server?

ng
11/30/2009 08:04 PM by
ng

I've been working on an app and configuring hosting solutions in both rackspace and amazon clouds for the last few months. First time working with nix servers. I can't describe what a better experience it is than working with windows servers. It's really night and day.

Demis Bellot
11/30/2009 08:05 PM by
Demis Bellot

Yeah I do the same, deploy on VS.Net and deploy on Nginx + FastCGI Mono on CentOS.

Haven't done any benchmarks but it runs pretty quick - this is a sample ajax website powered by mono webservices:

http://www.freelamusica.com/ (links to spotify so best viewed in europe)

Pretty stable as well, haven't had any crashes yet.

Ramon Smits
11/30/2009 09:25 PM by
Ramon Smits

At least put some stuff about the visitor stats on your site before you write such stuff. Ofcourse it works but what about performance.

How many views per second do you achieve and how much memory does it require compared to a Windows 2008 webserver.

Demis Bellot
11/30/2009 10:27 PM by
Demis Bellot

At least put some stuff about the visitor stats on your site before you write such stuff.

What stuff did I write? that it runs pretty quick? Should I have written that its quick enough for my needs?

I haven't benchmarked it because it doesn't affect my decision, I only have a linux dedicated host so the only way I could host it was on Mono. Nginx appears to be the fastest web server so I've configured it to use that. Even if I did care enough to benchmark it, it would be meaningless as my development pc has very different specs to my dedicated server.

Code-architecture is much more important than server platform anyway. You can swap out and deploy your software on a different host much easier than you re-write your application.

I actually do care about performance, which is why my hobby website works on top of a sharded architecture that employs a full-text index, and intelligent compressed front-line caching. I also use JSON web services which works inside a static Ajax website that uses compressed javascript, html and css (like gmail and google maps) for the best end-to-end performance.

If at anytime it ever gets popular to the effect that mono poses a problem than I can easily setup a windows server and change the url on my deployment script and have it hosted on windows.

Patrick McEvoy
12/01/2009 07:42 AM by
Patrick McEvoy

We develop and deploy on linux and have been doing for over 2 years. We only use windows for IE testing. Windows servers are a joke compared to linux rock solid stability particularly from distros like Debian. This is why I think it's such a shame you've picked a windows only front end for nhprof.

@Ramon we do 40k+ page views a month with Clubhouse ( http://www.clubhouseapp.com) with apache/mod_mono. I can't give you direct performance comparisons but I'm sure you know there's more to picking a platform than performance afterall that's the point of this post. And of course being open source anyone can improve things themselves. Big missing things are garbage collection, but seriously should we be relying on GC?

Frans Bouma
12/01/2009 08:49 AM by
Frans Bouma

@Patrick: any OS can be ruined by bad administration. calling windows 'a joke' shows you don't really have used windows server OS-es much. It's easy to get a lot of uptime on a windows box, just use a good firewall so you don't run the risk of having an open box with every security flaw, but that's the same with Linux which has many security flaws per month as well.

There are some areas where some OS is better than others. Personally I really liked windows 2000 server, compared to for example win2k8, as a lot of the management UIs are .net, but can break easily when something is installed wrong.

The best server OS IMHO is still FreeBSD, and definitely not linux. Why FreeBSD? Because of the technology they have with respect to scheduling, memory management and process management. That doesn't say other Os-es are 'jokes', they just lack in some areas, and a user has to pay attention to those areas, and Linux isn't an exception to that rule, on the contrary, so please keep things in perspective, OK? Also, that things are 'open source' is absolutely no argument either as using a program shouldn't imply updating the code yourself.

GC is required in VM based interpreters, I don't really see what you mean with a missing GC?

Finally, the licensing issue suggests that some new instance was spinned up which wasn't registered in time, not sure why that didn't happen, but it's not something that's 'not reliable', it's more a thing about GoGrid not being able to register OS copies in time.

Patrick McEvoy
12/01/2009 09:35 AM by
Patrick McEvoy

@Frans I completely agree that bad administration can cause problems, but that's just obvious and goes without saying. You are right in saying I don't use Windows much and that's because we were sick of 2000 screwing up all the time. I use 'a joke' from personal experience it's not a naive throw away comment and therefore I realise your experiences are different and won't feel the same.

I find FreeBSD great for servers you set up and don't touch again, although that's not such an issue now with binary packages and faster hardware. A while ago support for mono on FreeBSD wasn't that great, although looking now things might have improved a lot.

"Also, that things are 'open source' is absolutely no argument either as using a program shouldn't imply updating the code yourself."

Again this is obvious and of course I agree. Do you see merit in being able to view and adapt source code your applications and platforms run on? I do, and feel strongly this should be a decision making factor, that's all I said above in relation to performance. No need to start a open vs closed source argument.

"GC is required in VM based interpreters"

Yep sorry, I missed out the word 'compacting' :)

firefly
12/01/2009 10:17 AM by
firefly

This is an area that MS need to fix quickly. I personally think somebody deserve to be fire over this.

It's always bad business practice to prosecute the good guy by accident... Especially this is a server OS that has already been activated. A server is something that should be relatively maintenance free. I am running Windows Server myself and for the most part they are maintenance free... It's just appalling to know that a server that has already been activated goes down because of activation issue.

Rafal
12/01/2009 10:21 AM by
Rafal

Maybe the hosting provider has changed VM configuration and windows thought it has been transplanted to another machine?

Stephen
12/01/2009 10:47 AM by
Stephen

Wow, check out all this windows bashing :P, personally whilst windows caused the issue because of activation issues, activation is sadly the way the world has forced microsoft to go, and I would say the original issue which caused the activation issue was caused be your hosts.. or yourself.

Changing certain hardware (even installing certain new drivers) will cause the hardware fingerprint to change and cause an activation problem.

Jozef Sevcik
12/01/2009 12:45 PM by
Jozef Sevcik

@Stephen

World has forced Microsoft to design activation in a way that change of hardware or installing new drivers may cause server to go down ?

I don't think so. This is the case illustrated in the article.

I don't care about how activation process works, if it's good or bad design in general.

But if SERVER OS goes DOWN for this, then it's just not acceptable.

Stephen
12/01/2009 10:12 PM by
Stephen

Jozef, yes the world has, without hardware identification the activation wouldn't work.

Also, strangely if you unplug a server, or hit shutdown from the start menu, or hell.. tons of things then yea the server will go down.. my point was this error is highly unlikely to have happened randomly, there is user error involved most likely from the hosts.

So don't get all 'crazy serious' about server up time, when its THAT easy for a single server to go down, if you are worried that much about up time you wouldn't have a single server.

Constraints of REALITY and all that..

Jozef Sevcik
12/02/2009 12:36 AM by
Jozef Sevcik

Stephen,

I agree with you, bad things always happen and for critical stuff, you should not rely on one server - but again that is not the point.

This case is very concrete - server goes down or gets unreachable, because of activation issue.

This is very different from invoking shutdown from start menu, unplugging server by user error or tons of other things where people/hardware/services fails.

Stephen
12/02/2009 10:40 AM by
Stephen

How is it any different? that's MY point, the activation issue is highly likely caused by something a user has done, I would say the hosts most likely.

firefly
12/02/2009 11:27 AM by
firefly

The semantic is different. Just because some arbitrary bad thing might happen doesn't mean it's okay to introduce another point of failure. Say you might trip and fall tomorrow doesn't make it okay for me to punch you in the face.

Again this is nothing against MS, but the heavy handed approach is crying out for a foul coming from any company. It's just bad customer service. I think it's better for a company to lose a few sales that they probably would never have than to persecute a good guy by accident...

Andrew
12/02/2009 04:21 PM by
Andrew

@firefly and Jozef,

I think you're both missing Stephen's point, in all likelihood this is the fault of the host, yet since we're dealing with Microsoft its instantly Microsofts fault. I'm just surprised we haven't see a "Micro$oft" spelling in any of these comments yet.

Microsoft sells a product that requires activation in certain scenarios, someone at the hosting company (that gets paid to be experts in this product) more than likely voluntarily did something to cause the activation to come up, blame the hosting company.

Chris Smith
12/02/2009 04:21 PM by
Chris Smith

That is usually resolved by re-entering the product key, activating it again and rebooting the box.

It's almost as if windows has an insecurity complex.

We've never had it on live kit even though we use VMware.

I'm beginning to think that software that costs money isn't worth the hassle. I've got 2 production apps on Django on Python on a Debian Linux virtual machine and there have been NO problems at all so far, not with the software environment, deployment or host.

I'm slowly being driven away from .Net/SQL Server and Windows for the complexity of managing licensing, market-segmented memory and cpu restrictions, vicious activation deaths, stuff that cocks up that you don't have the source for and just the general complexity and depth of the stack. I just want to use my kit and go home.

Jozef Sevcik
12/02/2009 08:56 PM by
Jozef Sevcik

Andrew,

as I already said before: I'm not going to judge if the Activation process or its design - I know nearly nothing about it (and thanks god we had no problems with w2k8 activation on servers so far).

I'm trying to say (once again), that letting server go down because of licencing issue (which is not hardware/OS failure) is maybe acceptable for desktop OS or product installed in the OS.

But it's NOT for the Server system - that's all I want to say, nothing else.

Getting server down because of licensing issue is just a joke.

Rafal
12/02/2009 09:32 PM by
Rafal

@All

If all everyday technical problems were so 'hard' and 'serious' like this one no one would pay us a dime to solve them. So stop whining and do some real work instead to show your bosses you're worth your salary

Driesie
12/03/2009 10:13 AM by
Driesie

+1 on Linux (Debian in my case) & mono for server .NET

I was recently asked by Amazon to move one of my EC2 instances in order for them to perform some hardware maintanance. Uptime had been 324 days.

Before I moved to Amazon, we had a dedicated linux box. When I discontinued that, uptime had been more than double that figure (embaressingly long actually).

Windows on the client ... yes (although I use Ubuntu and run VS in a VirtualBox environment), on the server ... good luck.

I'm surprised Mono doesn't get more time on open source .net groups/conferences/discussions etc. It's amazing what's been achieved there. Give it a try and be surprised!

Matt Freeman
12/08/2009 08:20 AM by
Matt Freeman

Has a post mortem been carried out to find out why a server that's been running for a year has activation issues? If the provider has a MS volume license or host license they should have a ticket allowance that they can take up the issue directly with MS. Would be interested in finding out the circumstances that lead to this.

Comments have been closed on this topic.