http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Our stuff automatically sets all loggers to ERROR on deployment after precisely that cock up. It's even worse with the verbose logging in Spring.Net http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment9http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment9Sun, 25 Oct 2009 10:05:08 GMTI recently converted a bunch of logging in an app from Debug.Trace to log4net since we actually needed the logs. Long story short, turning logging from DEBUG to INFO took a reporting process down from about half an hour to about a minute and a half. http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment8http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment8Sun, 25 Oct 2009 05:55:37 GMT@Bryan I don't know if you have used NLog for logging in your application, but NLog has some nice features that can help to keep both log messages and performance. For example, there's an async log target that writes messages to a log in a background thread, without slowing down threads that do the real work. There's also a caching target (I don't really remember the details) that collects thread's debug messages in memory and writes them all to log file if an exception occurs. If there's no exception thrown no debug messages are written - this way you can have debug information only for requests ending with an exception. And it has minimum overhead for non-logged messages. http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment7http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment7Sat, 24 Oct 2009 18:58:13 GMTThe Rhino Service Bus link is broken (404). http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment6http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment6Sat, 24 Oct 2009 17:49:49 GMT"visibility into what the application is doing is key" That is one of the things I found interesting from your NDC talk where you mention SLA's and queues. Not necessarily logging, but still a glimpse inside your applications state. http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment5http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment5Sat, 24 Oct 2009 14:48:58 GMTSmartInspect by Gurock Software. Logs all, very - very -very fast, viewer filters by threads, too many features to list. Nuff said. http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment4http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment4Sat, 24 Oct 2009 14:44:02 GMTA solution that has worked well for me is to aggregate logging messages and periodically flush them to the log. Of course- and you've indicated this above- the messages being batched in production are much less verbose than the development / test environments. http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment3http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment3Sat, 24 Oct 2009 12:50:39 GMTI am going to touch on that in a separate blog post http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment2http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment2Sat, 24 Oct 2009 12:06:19 GMTI strongly agree, but I have a question: as a person who's never had the pleasure of maintaining a big enough server system while it was in production (the only ones I've written had very low usage) - what *do* you log on production? I suppose you don't turn off logging entirely, do you? http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment1http://ayende.com/4263/how-to-mis-configure-your-production-to-dead-crawl#comment1Sat, 24 Oct 2009 12:03:04 GMT