Skip to content

Instantly share code, notes, and snippets.

@desa

desa/non_time_series.md

Last active January 23, 2019 18:57
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save desa/4e802ee4a4f6afd5a06e8630628631c1 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save desa/4e802ee4a4f6afd5a06e8630628631c1 to your computer and use it in GitHub Desktop.
Download ZIP
InfluxDB thoughts
Raw
non_time_series.md

Add support for storing generic non time series data

InfluxDB is great at storing and working with time series data, however that data is often useless without some additional non time series data.

Take Chronograf as an example. Without the ability to store dashboards, Chronograf would be nothing more than a visual query builder. The rich experience that Chronograf provides is only possible since can store some amount of non time series data.

I propose that we allow users to store Tables of non time series data. This would truly allow application developers to build on top of the platform. Currently all backend structures are very tightly coupled to the requirements of a single application (Chronograf). This severely limits the potential applications that users can build on top of InfluxDB.

In addition to allowing users to store non time series data, I propose that Chronograf be implemented exclusively on top of the Table primitives we provide.

Chronograf features could be developed without any backend code changes.

Chronograf feature development that requires api changes currently has the following cycle:

  1. API gets requirement
  2. API negotiation between frontend team and api team takes place
  3. Frontend team implements the frontend piece with stubbed out values
  4. API teams implements the API
  5. Feature is delivered

Furthermore, any future changes to that API require both frontend and backend changes. If Chronograf had access to some kind of Table structure they could make changes without requiring any backend work. The frontend team would be able to make schema changes at their discretion, with out requiring any backend changes.

The feature development cycle would look something like the following:

  1. Frontend team gets requirements
  2. Frontend implements feature
  3. Feature is delivered

Application developers could build apps on top of InfluxDB

The specific application that I'm imagining is an IoT gateway. Currently, if a user wanted to have an IoT gateway that stored the device information and the time series data associated with the device, they would have to have two databases. One for the non time series data and one for the time series. On top of that, if they wanted to have a UI that combines the device data with its time series, they would need to build a server the combines the data and serves the UI. This is a ton of extra work.

If we had user definable tables, and a plugin system like Grafana, this type of system could be built entirely on top of InfluxDB and embedded inside of Chronograf. Allowing users to build on top of the platform would allow application developers to build interesting non-trivial applications on top of the InfluxDB platform.

Details

InfluxDB should contain the following following primitives:

  1. Users
  2. Tokens
  3. Orgs
  4. Buckets
  5. Tasks
  6. Secrets
  7. Labels
  8. Tables
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment