http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Have you full source code for this perf test?http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment19http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment19Thu, 06 Dec 2012 13:07:19 GMTSo, how did the older version do on this? What's the improvement (if any) does 2.0 bring?http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment18http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment18Mon, 26 Nov 2012 17:18:08 GMTVery nice! What do the indexes look like?http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment17http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment17Fri, 23 Nov 2012 16:24:03 GMT@Daniel Bulk loading should include indexing. In my earlier example, indexing during bulk load is enabled. http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment16http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment16Fri, 23 Nov 2012 11:36:40 GMTAndersM: My point is - the only metric I care about is the time it takes to do both, writing _and_ indexing. No, I don't mean bulk import of data-sets because this is something you don't do frequently and when you do it, it's generally not time sensitive (like migrate from another database).http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment15http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment15Fri, 23 Nov 2012 10:43:54 GMT>AndersM, Not really, just loading the data and waiting for indexing, and loading the data with indexing would result in about the same time frame Maybe in the Ravendb world... As Will Hughes suggested, it would be more interesting to see the difference with the previous release, right now it's just some random numbers. http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment14http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment14Fri, 23 Nov 2012 09:51:08 GMTOk, i did not know how Raven would handle this, but answered based on Daniels numbers :)http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment13http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment13Fri, 23 Nov 2012 09:45:54 GMTAndersM, Not really, just loading the data and waiting for indexing, and loading the data with indexing would result in about the same time framehttp://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment12http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment12Fri, 23 Nov 2012 09:12:20 GMTDaniel: Of course it matters, Jamie clearly stated why. If you need to store large amounts of data quickly, and only need indexes later, bulking makes sense.http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment11http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment11Fri, 23 Nov 2012 09:10:59 GMTI really don't understand why people care so much about 'bulk load' performance. I mean really, what's the difference between writing 1.000 or 5.000 documents per second WITHOUT indexing? The whole point about raven is that is has indexes for you to do calculation or queries. If you don't need that, you have a key/value store for which you don't need raven in the first place. Perf metrics without indexing are useless.http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment10http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment10Fri, 23 Nov 2012 08:29:42 GMTThe "ETL code" is just the smuggler.http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment9http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment9Thu, 22 Nov 2012 23:41:55 GMTJamie, We will have bulk load work done after the release. It is a bit involved, as you might imagine.http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment8http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment8Thu, 22 Nov 2012 23:41:44 GMTIs it possible to get a comparison with Raven 1.x's performance using the same dataset and hardware? http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment7http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment7Thu, 22 Nov 2012 22:13:32 GMTWould be great if you could direct us to your ETL process. I noticed the old ETL project in the raven source is no longer there?http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment6http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment6Thu, 22 Nov 2012 20:51:52 GMT@Jeús http://www.imdb.com/interfaceshttp://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment5http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment5Thu, 22 Nov 2012 19:23:15 GMTMay I ask where you downloaded the IMDB dataset from?http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment4http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment4Thu, 22 Nov 2012 16:43:29 GMTWhat options do you have for doing an actual bulk load? Say if we wanted to load 250m moderately complex documents - is there some kind of bulk load option which can do batch indexing after?http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment3http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment3Thu, 22 Nov 2012 16:42:41 GMTPop, This is meant to show indexing performance more than anything else. Bulk load is doing something quite different.http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment2http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment2Thu, 22 Nov 2012 16:32:54 GMT"Loading 4.66 millions records in 44 minutes" This means 1756 documents / second. Is the I/O channel saturated from this? Disk write speed maxed out? I don't know about the complexity of those documents, however an ETL process can reach over 50k "rows" per second on my modest machine using bulk load. Therefore I think it would be interesting to see some benchmarks for small documents (1 property), medium (10-100 properties) large (1000+ properties) and the I/O caracteristics of Raven DB during such operations. http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment1http://ayende.com/160033/what-is-up-with-ravendb-2-0-performance#comment1Thu, 22 Nov 2012 16:23:23 GMT