Ayende @ Rahien

Hi!
My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

, @ Q c

Posts: 18 | Comments: 66

filter by tags archive

Time series feature design

time to read 2 min | 289 words

I mentioned before that there is actually quite a lot that goes into a major feature, usually a lot more than would seem on the surface. I thought that this could serve as a pretty good example of a series of blog posts that we can use to show how we are actually sitting down and doing high level design of features in RavenDB.

I’ve actually already spoken, and we have seen an implementation of the backend, about time series data. But that was just the basic storage, and that was mostly to explore how to deal with Voron for more scenarios, to be honest.

Now, allow me to expand that into a full pledged feature design.

What we want: the ability to store time series data and work with it.

Time series data is a value (double) that is recorded at a specific time for a specific series. A good explanation on that can be found here: https://tempo-db.com/docs/modeling-time-series-data/

We have a bunch of sensors that report data, and we want to store it, and get it back, and get rollups of the information, and probably do more, but that is enough for now.

At first glance, we need to support the following operations:

  • Add data (time, value) to a series.
  • Add bulk data to a set of series.
  • Read data (time,value) from a series or a set of them in a given range.
  • Read rollup (avg, mean, max, min) per period (minutes, hour, day, month) in a given range for a set of series.

In the next few posts, I’ll discuss storage, network, client api, user interface and distribution design for this. I would truly love to hear from you about any ideas you have.


Comments

Comment preview

Comments have been closed on this topic.

FUTURE POSTS

  1. RavenDB 3.0 New Stable Release - about one hour from now
  2. Production postmortem: The case of the lying configuration file - about one day from now
  3. Production postmortem: The industry at large - 2 days from now
  4. The insidious cost of allocations - 3 days from now
  5. Buffer allocation strategies: A possible solution - 6 days from now

And 4 more posts are pending...

There are posts all the way to Sep 11, 2015

RECENT SERIES

  1. Find the bug (5):
    20 Apr 2011 - Why do I get a Null Reference Exception?
  2. Production postmortem (10):
    31 Aug 2015 - The case of the memory eater and high load
  3. What is new in RavenDB 3.5 (7):
    12 Aug 2015 - Monitoring support
  4. Career planning (6):
    24 Jul 2015 - The immortal choices aren't
View all series

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats