Oren Eini

aka Ayende Rahien

Oren Eini

CEO of RavenDB

a NoSQL Open Source Document Database

Get in touch with me:

oren@ravendb.net +972 52-548-6969

Posts: 7,578
|
Comments: 51,203

Copyright ©️ Ayende Rahien 2004 — 2025

Privacy Policy · Terms
filter by tags archive
stack view grid view
  • architecture (608) rss
  • bugs (450) rss
  • challanges (123) rss
  • community (378) rss
  • databases (481) rss
  • design (894) rss
  • development (640) rss
  • hibernating-practices (71) rss
  • miscellaneous (592) rss
  • performance (397) rss
  • programming (1085) rss
  • raven (1445) rss
  • ravendb.net (529) rss
  • reviews (184) rss
  • 2025
    • May (10)
    • April (10)
    • March (10)
    • February (7)
    • January (12)
  • 2024
    • December (3)
    • November (2)
    • October (1)
    • September (3)
    • August (5)
    • July (10)
    • June (4)
    • May (6)
    • April (2)
    • March (8)
    • February (2)
    • January (14)
  • 2023
    • December (4)
    • October (4)
    • September (6)
    • August (12)
    • July (5)
    • June (15)
    • May (3)
    • April (11)
    • March (5)
    • February (5)
    • January (8)
  • 2022
    • December (5)
    • November (7)
    • October (7)
    • September (9)
    • August (10)
    • July (15)
    • June (12)
    • May (9)
    • April (14)
    • March (15)
    • February (13)
    • January (16)
  • 2021
    • December (23)
    • November (20)
    • October (16)
    • September (6)
    • August (16)
    • July (11)
    • June (16)
    • May (4)
    • April (10)
    • March (11)
    • February (15)
    • January (14)
  • 2020
    • December (10)
    • November (13)
    • October (15)
    • September (6)
    • August (9)
    • July (9)
    • June (17)
    • May (15)
    • April (14)
    • March (21)
    • February (16)
    • January (13)
  • 2019
    • December (17)
    • November (14)
    • October (16)
    • September (10)
    • August (8)
    • July (16)
    • June (11)
    • May (13)
    • April (18)
    • March (12)
    • February (19)
    • January (23)
  • 2018
    • December (15)
    • November (14)
    • October (19)
    • September (18)
    • August (23)
    • July (20)
    • June (20)
    • May (23)
    • April (15)
    • March (23)
    • February (19)
    • January (23)
  • 2017
    • December (21)
    • November (24)
    • October (22)
    • September (21)
    • August (23)
    • July (21)
    • June (24)
    • May (21)
    • April (21)
    • March (23)
    • February (20)
    • January (23)
  • 2016
    • December (17)
    • November (18)
    • October (22)
    • September (18)
    • August (23)
    • July (22)
    • June (17)
    • May (24)
    • April (16)
    • March (16)
    • February (21)
    • January (21)
  • 2015
    • December (5)
    • November (10)
    • October (9)
    • September (17)
    • August (20)
    • July (17)
    • June (4)
    • May (12)
    • April (9)
    • March (8)
    • February (25)
    • January (17)
  • 2014
    • December (22)
    • November (19)
    • October (21)
    • September (37)
    • August (24)
    • July (23)
    • June (13)
    • May (19)
    • April (24)
    • March (23)
    • February (21)
    • January (24)
  • 2013
    • December (23)
    • November (29)
    • October (27)
    • September (26)
    • August (24)
    • July (24)
    • June (23)
    • May (25)
    • April (26)
    • March (24)
    • February (24)
    • January (21)
  • 2012
    • December (19)
    • November (22)
    • October (27)
    • September (24)
    • August (30)
    • July (23)
    • June (25)
    • May (23)
    • April (25)
    • March (25)
    • February (28)
    • January (24)
  • 2011
    • December (17)
    • November (14)
    • October (24)
    • September (28)
    • August (27)
    • July (30)
    • June (19)
    • May (16)
    • April (30)
    • March (23)
    • February (11)
    • January (26)
  • 2010
    • December (29)
    • November (28)
    • October (35)
    • September (33)
    • August (44)
    • July (17)
    • June (20)
    • May (53)
    • April (29)
    • March (35)
    • February (33)
    • January (36)
  • 2009
    • December (37)
    • November (35)
    • October (53)
    • September (60)
    • August (66)
    • July (29)
    • June (24)
    • May (52)
    • April (63)
    • March (35)
    • February (53)
    • January (50)
  • 2008
    • December (58)
    • November (65)
    • October (46)
    • September (48)
    • August (96)
    • July (87)
    • June (45)
    • May (51)
    • April (52)
    • March (70)
    • February (43)
    • January (49)
  • 2007
    • December (100)
    • November (52)
    • October (109)
    • September (68)
    • August (80)
    • July (56)
    • June (150)
    • May (115)
    • April (73)
    • March (124)
    • February (102)
    • January (68)
  • 2006
    • December (95)
    • November (53)
    • October (120)
    • September (57)
    • August (88)
    • July (54)
    • June (103)
    • May (89)
    • April (84)
    • March (143)
    • February (78)
    • January (64)
  • 2005
    • December (70)
    • November (97)
    • October (91)
    • September (61)
    • August (74)
    • July (92)
    • June (100)
    • May (53)
    • April (42)
    • March (41)
    • February (84)
    • January (31)
  • 2004
    • December (49)
    • November (26)
    • October (26)
    • September (6)
    • April (10)
RavenDB - High-Performance NoSQL Document Database
  previous post next post  
Oct 11 2009

The problem with running on virtual hardware…

time to read 1 min | 82 words

I tested this in both GoGrid & EC2, and got similar results:

image

The disk I/O is really bad. Just to give you an idea about the difference, here is the same process on my local machine (1TB 7,200 RPM WD HD):

image

Tweet Share Share 13 comments
Tags:
  • Hardware

  previous post next post  

Comments

Nic Wise
11 Oct 2009
18:02 PM
Nic Wise

Wow. According to my mac (well, the network monitor on it) I can copy to/from my Mac Mini (from my Macbook) at about 30 MB/sec, over gigabit ethernet. No special drive - I think it's a 5400 on both ends.

Is it a windows thing? Or some vista setup problem? I thought ScottHa talked about it a while back... and ended up with 20MB/sec or more.

@Nic
11 Oct 2009
20:17 PM
@Nic

Nic, look at the number of items being copied - 7448. Lots of small files always incurs a penalty hit.

James Kovacs
11 Oct 2009
20:35 PM
James Kovacs

Expanding on @Nic's comment... 30MB/s is likely the sustained sequential write speed of the destination drive. The reason that Oren is getting 2.26MB/s is that he is writing lots of little files. Random write performance is a lot lower than sequential. It is not a Windows issue, but a hardware limitation. The low random write performance in a VM is interesting. Virtualized disk I/O performs worse than virtualized CPU and is often the limiting factor when determining whether a particular workload should be virtualized or hosted on physical hardware. For example, web servers (running apps) virtualize better than file servers.

Mr_Simple
12 Oct 2009
01:31 AM
Mr_Simple

Just like the 1000 other dumb technologies we have seen come and go, Cloud is dead. Long live local IO.

With respect to backing up, rotate a spare HD at your safe deposit box or run Carbonite on an unused computer because it will take that loooong to copy your data offsite.

zvolkov
12 Oct 2009
02:05 AM
zvolkov

At least use robocopy or TeraCopy when copying many files over network, they support retry on network failures.

Ayende Rahien
12 Oct 2009
02:16 AM
Ayende Rahien

zvolkov,

Look at what I am actually _doing_, I am unzipping a file, not doing network ops.

Phillip Haydon
12 Oct 2009
04:15 AM
Phillip Haydon

I just setup a virtual server on the weekend, and unzipped 400mb file of 20k files in under a minute...

Maybe your sharing resources with other people on what ever virtual environment your on.

zvolkov
12 Oct 2009
13:31 PM
zvolkov

Oh!

configurator
12 Oct 2009
23:42 PM
configurator

Also note that ayende is copying (unzipping) files on the same drive, that would obviously work slower than copying from a different drive.

Same as when I copy two large files from E: to C: at the same time I get about 25% speed on each.

Will Hughes
13 Oct 2009
03:22 AM
Will Hughes

Ayende: With EC2 you will find the local drives suck.

If you attach an EBS Volume, you'll get significantly better IO. (And the data on that won't go away if the machine is turned off)

It'd be worth asking GoGrid what their disks are attached using - I'd hazard a guess that it's iSCSI over 1-10GbE to a SAN box.

I know our VM farm at our 'litt'e dev shop has no issues with disk IO, even when we've got 40 VMs up over two VM servers.

Dan Finch
14 Oct 2009
19:20 PM
Dan Finch

you can solve this problem by clicking "Fewer details"

Ayende Rahien
14 Oct 2009
19:29 PM
Ayende Rahien

Um, you do realize that my issue is the performance, not the dialog, right?

Dan Finch
14 Oct 2009
19:35 PM
Dan Finch

yes, bad joke

Comment preview

Comments have been closed on this topic.

Markdown formatting

ESC to close

Markdown turns plain text formatting into fancy HTML formatting.

Phrase Emphasis

*italic*   **bold**
_italic_   __bold__

Links

Inline:

An [example](http://url.com/ "Title")

Reference-style labels (titles are optional):

An [example][id]. Then, anywhere
else in the doc, define the link:
  [id]: http://example.com/  "Title"

Images

Inline (titles are optional):

![alt text](/path/img.jpg "Title")

Reference-style:

![alt text][id]
[id]: /url/to/img.jpg "Title"

Headers

Setext-style:

Header 1
========
Header 2
--------

atx-style (closing #'s are optional):

# Header 1 #
## Header 2 ##
###### Header 6

Lists

Ordered, without paragraphs:

1.  Foo
2.  Bar

Unordered, with paragraphs:

*   A list item.
    With multiple paragraphs.
*   Bar

You can nest them:

*   Abacus
    * answer
*   Bubbles
    1.  bunk
    2.  bupkis
        * BELITTLER
    3. burper
*   Cunning

Blockquotes

> Email-style angle brackets
> are used for blockquotes.
> > And, they can be nested.
> #### Headers in blockquotes
> 
> * You can quote a list.
> * Etc.

Horizontal Rules

Three or more dashes or asterisks:

---
* * *
- - - - 

Manual Line Breaks

End a line with two or more spaces:

Roses are red,   
Violets are blue.

Fenced Code Blocks

Code blocks delimited by 3 or more backticks or tildas:

```
This is a preformatted
code block
```

Header IDs

Set the id of headings with {#<id>} at end of heading line:

## My Heading {#myheading}

Tables

Fruit    |Color
---------|----------
Apples   |Red
Pears	 |Green
Bananas  |Yellow

Definition Lists

Term 1
: Definition 1
Term 2
: Definition 2

Footnotes

Body text with a footnote [^1]
[^1]: Footnote text here

Abbreviations

MDD <- will have title
*[MDD]: MarkdownDeep

 

FUTURE POSTS

No future posts left, oh my!

RECENT SERIES

  1. Recording (16):
    29 May 2025 - RavenDB's Upcoming Optimizations Deep Dive
  2. Webinar (6):
    27 May 2025 - RavenDB's Upcoming Optimizations Deep Dive
  3. RavenDB News (2):
    02 May 2025 - May 2025
  4. Production Postmortem (52):
    07 Apr 2025 - The race condition in the interlock
  5. RavenDB (13):
    02 Apr 2025 - .NET Aspire integration
View all series

RECENT COMMENTS

  • What a massive presentation! As a person who spent some time with a db written in .NET I can strongly relate to some points. ...
    By Scooletz on Recording: RavenDB's Upcoming Optimizations Deep Dive
  • I’d love to learn your thoughts on SPANN https://arxiv.org/abs/2111.08566 that with centroids and keeping the posting lists s...
    By Scooletz on Comparing DiskANN in SQL Server & HNSW in RavenDB
  • Joel, The DiskANN paper talks about it being viable for more than a billion vectors datasets.  In such a scenario, it would ...
    By Oren Eini on Comparing DiskANN in SQL Server & HNSW in RavenDB
  • Do you know why they chose DiskANN? These things are usually about tradeoffs but it seems DiskANN is just worse in every way.
    By Joel on Comparing DiskANN in SQL Server & HNSW in RavenDB
  • Scooletz, Yes, we look at other stuff. Most of those are _not_ allowing you to build themselves incrementally nor are they...
    By Oren Eini on Scaling HNSW in RavenDB: Optimizing for inadequate hardware

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats
}