http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Ah, I misinterpreted. This is for the billing party, not the payment processor. http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment7http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment7Fri, 22 Apr 2011 17:53:21 GMTSince we want it to be really fast, I would proceed in the following manner: - before running the service that charges clients, I would create tables with the same structure of the payments in the database, the number of tables would be equal to the number of parallel processes I would use - have the given process run in parallel, the more the better (equal with the number of tables created), and have them store the results in their tables - when the code finishes, it should do a "merge" from the new tables to the base table - drop the newly created tables http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment6http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment6Thu, 21 Apr 2011 18:25:53 GMTI assume batching would mean some kind of a script, where actions executed one-by-one in a chain. And we cannot do that, therefore I would vote for a parallel execution of the task. In a simple scenario it can be multithreading, where each thread performs the job for one account in parallel to other threads. http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment5http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment5Thu, 21 Apr 2011 16:42:23 GMTYou need the user's current balance. Storing that as a fixed number is prone to race conditions that cause you to overpay *and* record the wrong balance for the customer account. You could store a series of operations, but eventually you need to reconcile them. This way is prone to overpaying, but you can report an accurate balance. So, you use database level locks and store the actual balance. Don't read before acquiring the write lock for that customer account. Record the account transactions in an append-only table and periodically ensure consistency. Since no customer account needs to make reference to any other account, this partitions extremely well. http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment4http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment4Thu, 21 Apr 2011 16:26:04 GMTI'd prepare as much as possible before the billing process starts. E.g. retrieve all necessary biling information from the relatively slow DB to memory. Hardware and network optimization will also help. http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment3http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment3Thu, 21 Apr 2011 12:58:18 GMT"At that point, it actually becomes a race, who will be able to charge the accounts first, and get the money|" Pretty interesting business model ;-) http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment2http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment2Thu, 21 Apr 2011 10:29:43 GMTHaving information about 'when' it will happen, you quantize time with, for instance 5 minute periods and get all the users within one period? http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment1http://ayende.com/4818/when-race-conditions-unbounded-result-sets-are-actually-the-solution#comment1Thu, 21 Apr 2011 09:22:38 GMT