Time series database design?

How would you design a time series database?

The implementation should be as fast as possible for lookup of data between date-time ranges and be capable of storing massive amounts of data.

Insertions should ideally be as fast as possible, although interested to hear of trade offs between read and write performance.

I realise this is an open ended question but am just looking to get an understanding of some of the considerations that inform the design. I'm not looking for recommendations on existing implementations, just curious as to how one might work on the inside...

This question and answers originated from www.stackoverflow.com
Question by (10/19/2011 9:24:37 PM)

Answer

A typical way of such database design is to have a table with data with 3 fields (parameterID, time, value) and then index by (parameterID, time). We have several terabyte(s) sized databases with these design and run smoothly.

It is important to choose an storage engine that allows SELECT and INSERTS simultaneously. If you use any storage engine with table locking (e.g. MyISAM) you will experience query contention that will slow your system down.

Answer by
Find More Answers
Related Topics  design  language-agnostic  time-series
Related Questions

  • Reconstructing state from time series data events

    For a particular project, we acquire data for a number of events and collect variables about those events at the same time. After the data has been collected, we perform a user-customizable analysis…

  • C# Design decision for time series access with a database

    I'm looking for a "best practise" way to handle incoming time series data. One data point consists for example of time, height, width etc. for every "tick". Is it a good idea to save n data point…

  • Time series database for java?

    I want to store millions of time series entries (long time, double value) with Java. (Our monitoring system is currently storing every entry in a large mysql table but performance is very bad.) A…

  • What are some compact algorithms for generating interesting time series data?

    The question sort of says it all. Whether it's for code testing purposes, or you're modeling a real-world process, or you're trying to impress a loved one, what are some algorithms that folks use…

  • Database suggestions for time series of events

    For one of my projects, I have to enter a big-ish collection of events into a database for later processing and I am trying to decide which DBMS would be best for my purpose. I have: About 4…

  • MongoDB as a Time Series Database

    I'm trying to use mongodb for a time series database and was wondering if anyone could suggest how best to set it up for that scenario. The time series data is very similar to a stock price histo…

  • Time series in R

    I am tracking my body weight in a spread sheet but I want to improve the experience by using R. I was trying to find some information about time series analysis in R but I was not succesful. The …

  • Relative Time Series

    I am looking for a standardized method for arranging data in relative time. Applications include accounting data such as FY1,FY2,etc... and economic data such as the term structure of interest rates…

  • time series in R

    here is my question: i have these data summary(data) Date 1990/01: 1 1990/02: 1 1990/03: 1 1990/04: 1 1990/05: 1 1990/06: 1 (…

  • simple time series db

    I am looking for a simple time series database. The sort of database that could take raw stock tick data for example and for company X and company Y in a split second return end of day data, or data…