http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660id use stamping, but not with time. use a logical clock instead (stamp with int++) that way ordering is supported by thee messsag logically, instead of assuming syynched cpus or just that ordering is more thann x wrong messagges in a timespan y. sorry for typos, answering on aa pdaa. http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment8http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment8Fri, 24 Oct 2008 18:48:17 GMTI am listening to an event stream generated by a single app instance on a single machine. I can rely on timestamps for orders, yes http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment7http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment7Fri, 24 Oct 2008 01:47:05 GMTJust realize that the timestamp-based solution only works if the event stream is coming from a single server (i.e. relying on the accuracy of clocks across multiple machines is not a good solution).. If you plan on ordering streams from multiple servers, you'd need to look at something like vectors instead of timestamps.. One other thing to realize, timestamps give you ordering. Vectors give you absolute ordering (or causality for other non-sql circumstances). With the vectors, you know when receiving B that A directly preceded it. That solves other crazy scenarios like, "do i have all the messages that preceeded the one i'm about to display" Although you may not care. And it may not be relevant to NH Prof :-) http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment6http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment6Fri, 24 Oct 2008 01:42:01 GMTCollection and processing != display. You could process the messages as they arrived within a context and reorder them according to the timestamp, updating the context as required. However associating messages with an appropriate session is not a constraint I had forseen, now your choice makes a lot more sense http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment5http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment5Thu, 23 Oct 2008 19:27:19 GMTNeal, You really want to be able to see the statements for the profiler in the same order that the app executed them, and you _really_ want to be able to associate the statements to the proper session. I solved the issue by relying on the timestamp, and ensuring that I can handle out of order (to some level) messages. http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment4http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment4Thu, 23 Oct 2008 19:12:48 GMTIn my experience with async messaging, although more with messaging systems like ActiveMQ, assuming order can be a bad assumption. It shouldn't be a problem that can't be overcome though. In my case, a simple integration test that spit out a few hundred thousand messages and checked for order was enough to know whether or not it is a good assumption. If this doesn't suit you, then I would lean on the side of figuring out a solution that doesn't assume order. Bad assumptions suck in the long run. Much worse than most bugs in my opinion. http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment3http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment3Wed, 22 Oct 2008 21:42:12 GMTThe nastiest issue I can see with relying on the timestamp is, at what point do you "know" you have all of the messages between two points in time such that you can begin to order them? Or do you deal with this at the last possible moment such that each message is processed and your visualisation of the process is no longer a queue but a list where any message may be inserted at any point (according to the timestamp) therefore forcing an update of any dependant metrics? Why is order important? What aspect of the application requires that messages must be processed in-order? Could you not argue that the real-time nature of the profiling is such that it is more likely capture and review within a small timespan (10 - 15s) and that some out of order processing (so long as the order is eventually corrected) will not cause a significant distortion of the perception provided by the profiling? Interesting problem. http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment2http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment2Wed, 22 Oct 2008 19:28:04 GMTBut where's the problem? You are monitoring a sequential stream of events so the order is known and never changes. It gets mixed up because of the data transmission method you chose (async loggers). Fix the data transmission and the problem will disappear. http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment1http://ayende.com/3656/a-messaging-problem-order-in-the-bus#comment1Wed, 22 Oct 2008 19:15:33 GMT