Timescale Blog

I see a lot of really divergent results with these time series database benchmarking posts. Timescale's open source benchmark suite[0] is a great contribution towards making different software comparable, but it seems like the tasks/metrics heavily favor TimescaleDB. This article has Clickhouse more-or-less spanking TimescaleDB, but the blog post it references[1] is basically the reverse. Are the use cases just that different? ----- 0. https://github.com/timescale/tsbs 1. https://blog.timescale.com/blog/what-is-clickhouse-how-does-it-compare…

(Post author) This is a great post to give you some talking points: https://blog.timescale.com/blog/what-the-heck-is-time-series-data-and-… I also love this recent one we did with some non-standard time-series data that the NFL provided! Really fun working on that data set. https://blog.timescale.com/blog/hacking-nfl-data-with-postgresql-times…

TimescaleDB performs quite well. One of our unique insights is that it is quite possible to build a best-in-class time-series database on top of Postgres (although it’s not easy ;-) Here is one benchmark: https://blog.timescale.com/blog/timescaledb-vs-influxdb-for-time-serie… There are some challenges with building on Postgres - but what we’ve been able to do is build innovative capabilities that overcome these challenges (Eg columnar compression in a row-oriented store, multi-node scale out). We also have some exciting things that we are announcing this week. Stay tuned :-) PS - Where did you find that PDF? Thought we took it down (it was hard to keep it up to date :-) )

This is a great piece. I've (been on teams that) explored every one of these points and I cannot think of others. The only big thing (which the article does call out) is that this is about scaling _traditional_ SQL databases. Traditional SQL databases don't have builtin horizontal scaling. The article is a little hazy about "specialized extensions": > When we need to store and work with geospatial, time series and other specialized data, we can sometimes use a database extension like TimescaleDb or PostGIS to make the data processing and storage more efficient. The point I'd call out is that specifically TimescaleDB allows you to horizontally scale PostgreSQL which is a pretty radical improvement on vanilla Postgres. https://blog.timescale.com/blog/timescaledb-2-0-a-multi-node-petabyte-…

I've been considering TimescaleDB for some use cases. Specifically to keep a transactional log of messages sent out. Has anyone independently benchmarked TimescaleDB vs Postgres w/ B-Tree vs Postgres w/ BRIN? Given that most timescale data would benefit greatly from BRIN, I'm surprised their various blog posts never cover it. I have discovered one person who tried to replicate some of the benchmarks around Postgres vs TimescaleDB (back in 2019), but they came up with data that contradicts the latest blog post around TimescaleDB vs Postgres performance, which makes me reconsider whether I really need what TimescaleDB provides. Maybe there have been large changes since then? http://blog.coelho.net/database/2019/09/13/postgresql-vs-timescaledb-d… https://blog.timescale.com/blog/timescaledb-vs-6a696248104e/

Hi @spmurrayzzz thanks for the feedback. (Timescale person) Always strive to do the best and fairest benchmarks we can, and for that reason, all our benchmarks are fully open-source for both repeatability and improvements/contributions: https://github.com/timescale/tsbs/blob/master/docs/mongo.md We also really did spend a lot of time investigating approaches with MongoDB, so you'll see our benchmarks actually evaluate two _different_ ways to use time-series data with MongoDB (culled & optimized from suggestions in MongoDB forums). But always welcome to feedback: https://blog.timescale.com/blog/how-to-store-time-series-data-mongodb-… Thanks!

Elastic Search developed a custom license to prevent strip-mining by AWS. Other companies soon followed, including Timescale, which produced different licenses in 2018, then 2020, for different purposes to address needs at appropriate stages. Their write-up on the process addresses other licenses that might be of interest to you: https://blog.timescale.com/blog/how-we-are-building-an-open-source-bus…

> No special "TSDB" needed. I think that is a large simplification. I recommend this (relatively) short article: https://blog.timescale.com/timescaledb-vs-6a696248104e The first graph is particularly salient.

I'm curious whether the parent poster knows about the most recent HA features in Postgres, such as those introduced in v10, and if so, what is still missing for comparable HA. Here's a link to the HA docs for out-of-the-box postgres: https://www.postgresql.org/docs/10/static/high-availability.html As for scaling, it certainly scales out of the box with partitioning, but admittedly a partitioned table doesn't offer all the features that a non-partitioned sql table offers. A great assessment of the current state of postgres scaling can be found here: https://blog.timescale.com/scaling-partitioning-data-postgresql-10-exp…

We are working on benchmarks comparing ourselves to other solutions so hopefully we'll have concrete numbers on those soon. (Incidentally, we did a blog post on us vs plain PostgreSQL today: https://blog.timescale.com/timescaledb-vs-6a696248104e) At a high level though, we do find that having native support for full SQL to be a big win. Also, if you already store metadata or other relational data that you want to combine with your time-series data, it's great to be able to use one DB instead of separate solutions. Performance wise we do believe we are competitive and in some cases much better, and we have the 20 years of stability from PostgreSQL to build on.

(NB: Post author here!) This is all true! We have a bit more info on that choice in this post [1]. But in general the Toolkit extension [2] is meant to have a bit of a lighter touch than the core TimescaleDB extension [3], it's meant for moving faster and working almost entirely with the custom functions/types/operators type approach rather than the core extension which is integrated much more deeply into the planner & executor and deals with a number of other hooks. It would be possible to do that in Rust, but because it's so deeply integrated with the main Postgres codebase there would just be a lot of translation going on and you'd get a lot less of the benefits of working in Rust. So C is more natural for much of the core DB extension. PGX [4] is also a huge help for writing this sort of extension. The point of Toolkit is really to move much faster, try lots of things, mostly using the more SQL facing API (CREATE FUNCTION, CREATE AGGREGATE and the like). And PGX helps manage all of that really nicely. The rest of Rust being available is also great and we like working in the language, but without PGX it'd be a lot harder and we couldn't move nearly as fast. [1]: https://blog.timescale.com/blog/introducing-hyperfunctions-new-sql-fun… [2]: https://github.com/timescale/timescaledb-toolkit [3]: https://github.com/timescale/timescaledb [4]: https://github.com/zombodb/pgx

Timescale employee here. While we do offer plans starting at $24/month [1] on Timescale Cloud, it's worth noting that DO Postgres (and other hosted Postgres solutions) are providing an undifferentiated open-source solution, while on Timescale Cloud you get Postgres underneath (for relational workloads) and TimescaleDB (for time-series workloads), plus other features which provide an enhanced time-series experience (like the explorer dashboard [2], autoscaling, automated upgrades, auto-tuning of configs, just to name a few). So do keep that in mind when comparing prices. Many users actually find Timescale Cloud to be cheaper because it replaces both a time-series and relational db with a single database that can store (and analyze) both kinds of data. [1] https://www.timescale.com/cloud [2] https://blog.timescale.com/blog/announcing-explorer-a-better-way-to-un…

I'm very towards Postgres and resultingly Timescale for being one of it's biggest shining stars. Combine that with a cynical view that this is essentially S-tier content marketing (in the tech world), and the numbers are still bonkers. Might as well add on a bit here -- if you're into this sort of thing you might enjoy Timescale thrashing other purpose-built databases (which have since also improved so YMMV): - Timescale vs Influx[0] - Timescale vs Mongo[1] - Timescale vs Cassandra[2] These blog posts are all from the timescale side, but the numbers are undeniable and their methodology is open source -- not sure there's more you can ask. Cassandra probably scales better at FB/Twitter scale, but with the recent work done to improve Timescale's scaling features[3] and the permissive license (unless you're an Amazon) I'm not sure even that is a real benefit. I'd love to see a response from Amazon -- if HN is so lucky to have a dev who worked on AWS Timestream that would be awesome. [0]: https://blog.timescale.com/blog/timescaledb-vs-influxdb-for-time-serie… [1]: https://blog.timescale.com/blog/how-to-store-time-series-data-mongodb-… [2]: https://blog.timescale.com/blog/time-series-data-cassandra-vs-timescal… [3]: https://blog.timescale.com/blog/timescaledb-2-0-a-multi-node-petabyte-…

That's a very cool demo, good job! I have a few questions if you don't mind. - Could you share the specs of the hardware powering the demo? - For IoT-type usage, one common access pattern is "get the readings of sensor X from device Y in the given time frame". But unfortunately, the data set in the demo doesn't include something like "cab_id" column that would allow me to simulate such a query. Is there a demo that has such a column? - Does QuestDB require rows to be inserted strictly in order, or is there some leeway? - Are you using any sort of compression? Timeseries columns are extremely compressible, and a good compression algorithm can greatly reduce IO wait times. We're using the Postgres-based TimescaleDb at work, and its compression capabilities have been a game changer [0]. - Do you plan on supporting GROUP BY clauses? [0]: https://blog.timescale.com/blog/building-columnar-compression-in-a-row…

Fellow Timescaler here. Thanks for the feedback. While we do not directly compare ingestion protocol and specific features, like continuous queries and retention polices (something I guess we could add), we do compare echosystems and third-party tools support, including ingestion (e.g., Kafka, Hibernate) and visualization (e.g., Tableau, Grafana). In fact, the developer behind the Grafana PostgreSQL data source is also a Timescaler, and an upcoming version of the data source will have an improved query builder and first-class TimescaleDB support. Finally, I can assure you that this is more than a few input scripts. In fact, the project is thousands of lines of C code (if that matters) that implement automatic partitioning, query optimizations, and much more. Our code is open source here: https://github.com/timescale/timescaledb We have other blog posts (http://blog.timescale.com) and system documentation (http://docs.timescale.com) that explain what we're doing if you're interested in learning more about the details.

> What's a good way to develop a mental model about what's happening under the hood in EdgeDB? At the physical schema level [0] or at the conceptual schema level [1]? This answer from edgedb CTO might clear the latter up; https://news.ycombinator.com/item?id=30291538 As for the former, I guess it is the same as however Postgres (pg) chooses to represent the edge-db tables. EdgeDB (graph on pg) sounds like Timescale (timeseries on pg [2]). [0] https://en.wikipedia.org/wiki/Physical_schema [1] https://en.wikipedia.org/wiki/Conceptual_schema [2] https://blog.timescale.com/blog/timescaledb-vs-influxdb-for-time-serie…

I am sure they'll change their license, like some others have done: https://www.confluent.io/blog/license-changes-confluent-platform/ https://blog.timescale.com/blog/building-open-source-business-in-cloud… I don't think there is any place for GPL in databases and similar software, specially if it is to be built and run by for-profit companies.

I work in the Database industry and have enjoyed these posts this year. - https://arctype.com/blog/postgres-notify-for-real-time-dashboards/ - https://heap.io/blog/speeding-up-postgresql-queries-with-partial-index… - https://blog.timescale.com/blog/how-postgresql-aggregation-works-and-h… - https://blog.crunchydata.com/blog/understanding-postgres_fdw

I encountered this problem in Postgres 13. https://blog.timescale.com/blog/how-we-made-distinct-queries-up-to-800…

Wait was not TimescaleDB completely leave Clickhouse in the dust both in terms of query performance and compression ? https://blog.timescale.com/blog/what-is-clickhouse-how-does-it-compare… I think this series of posts confirms the first law of Benchmarketing - for any system one can come up with "unbiased" benchmark which confirms its superiority

Thanks for the response. I enjoy reading your blog. What you said reminds me of the post [0] in which you compared Timescale with InfluxDB and argued that SQL is better. Has your position changed due to new observation regarding usability? [0] https://blog.timescale.com/blog/sql-vs-flux-influxdb-query-language-ti…

(NB: Promscale team member) Thanks for the positive feedback! Is there anything in particular you are missing in Promscale to be used as a backend for multiple Prometheus instances? We added support for multi-tenancy a couple of months ago (https://blog.timescale.com/blog/simplified-prometheus-monitoring-for-y… And thanks to a community contribution by 2nick on github Promscale can be integrated with Thanos :) (https://github.com/timescale/promscale/pull/664)

Here's some posts from users on the Timescale blog https://blog.timescale.com/tag/dev-q-a/

I introduced Timescale at an IIOT company, we had thousands of sensors regularly sending data up and wanted to efficiently display such metrics to users on a per-sensor basis, and one "tick" of data had a lot of metrics. Timescale let us go from Postgres for metadata and OpenTSDB (awful, stay far away) for time series to just one Timescale instance for everything. Huge win for us. We had enough data that doing the same with vanilla Postgres would have performed much worse (billions of rows). We wrote more about this for an earlier Timescale blog post: https://blog.timescale.com/blog/how-everactive-powers-a-dense-sensor-n…

Timescaledb adds a custom scan for speeding up distinct on out of the box. Just install extension, no extra work is needed. https://blog.timescale.com/blog/how-we-made-distinct-queries-up-to-800… I tried it and it really is that fast.

Timescale: "TimescaleDB vs. Amazon Timestream: 6000x higher inserts, 5-175x faster queries, 150x-220x cheaper" https://blog.timescale.com/blog/timescaledb-vs-amazon-timestream-6000x…

I don't think that's right. Citus has a post from 2017 talking about allowing for horizontal scaling of a postgres instance. [0] Same thing with Timescale. [1] -------------------------------------------- [0] https://citusdata.com/blog/2017/02/16/citus61-released/ [1] https://blog.timescale.com/blog/timescaledb-2-0-a-multi-node-petabyte-…

I wouldn't choose InfluxDB over TimescaleDB. There's a reasonably balanced comparison here from the Timescale guys: https://blog.timescale.com/blog/timescaledb-vs-influxdb-for-time-serie… The benchmarks are interesting, showing TimescaleDB to be the clear winner in most scenarios. For me that's nice, but it's a bigger deal to me personally that I already have Postgres and SQL experience that translates directly to TimescaleDB, I don't have to learn a new tool and query language. Development is complex enough and I have to learn too many things as it is. The older I get the less enthusiastic I am about adding something new to the stack.

To really appreciate the ludicrousness of what WeWork is doing, you have to take that example even further and assume that WeWork needs to track every single working adult on the globe clocking in and out. That's about 100k events per second. You can handle that using TimescaleDB running on a single Azure DS4 Standard node.[1] [1]: https://blog.timescale.com/blog/timescaledb-vs-6a696248104e/ Edit: Fixed my maths and language, and linking to hard data instead.