QuestDB Blog

Thank you for posting this benchmark, great addition to the TSDB community. We look forward to benchmarking our SQL time-series database QuestDB[1] versus both using the TSBS framework soon! [1] https://questdb.io/

Always a fun time to bring up my favorite writeups in database benchmarking: https://questdb.com/blog/lies-damn-lies-and-database-benchmarks/ There’s also an excellent paper called “Fair Benchmarking Considered Difficult: Common Pitfalls In Database Performance Testing”. Benchmarks are genuinely hard. Modern systems and hardware are notoriously complex and have different behaviors across runs and with minor tweaks to performance settings.

QuestDB at least is gaining some popularity: https://questdb.com/ I was hoping to learn about some new potentially viable alternatives to InfluxDB, alas it seems I'll continue using it for now.

Looks like they have an interesting range of customers[0] so my take on "why even add the Rust" is because their customers are already using Java, a total rewrite might be considered irresponsible as it would be incompatible with their existing customer base. I do have to wonder though, if some serious Java refactoring in any way, would have helped at all. How many code smells do they have going on in their codebase? Or is it to the best of their knowledge a really clean Java codebase? Note Discord themselves has used Rust for bottlenecks with Erlang/Elixir/Beam[1]. [0]: https://questdb.io/customers/ [1]: https://discord.com/blog/using-rust-to-scale-elixir-for-11-million-con…

Are you sure it is the best performing? The article only compares to influx and timescale, which are very far from the fastest timeseries databases out there. How does it compare to clickhouse? I found this https://questdb.io/blog/2022/05/26/query-benchmark-questdb-versus-clic… but the numbers are so far off (and clickhouse is much slower than timescale in their examples) that I kind of struggle to believe them.

If integration with PostgreSQL is not a requirement, I would suggest looking at QuestDB[1] for the best performing open-source timeseries database currently on the market. [1] https://questdb.io/blog/2021/07/05/comparing-questdb-timescaledb-influ…

I'm writing docs for https://questdb.io/ which offers SQL-based analytics for time series. We have quite a few users already who use us for ML applications that rely on performance. Instead of writing an MQTT -> PostgreSQL bridge, you could use Telegraf to listen to MQTT topics and write data to QuestDB over InfluxDB line protocol when it meets certain criteria. One of our users shared the tooling they use in industrial IoT and a tutorial here for exactly this use case: https://questdb.io/tutorial/2020/08/25/questitto/#stack

Have you checked questdb [1]? the data structure is arrays with data that lands in order and SQL queries on top. The fallback was that it was difficult to deal with out of order data, but we have just solved this by re-ordering data on the fly in memory before it hits the disk. Performance wise probably not far from kdb itself (will be sharing some bench results soon vs open source tsdbs) NB: I'm one of the co-founder of questdb [1] https://www.questdb.io

It looks very proming, congrats. I use ClickHouse in production and I'd love to see how this project evolves. My main disappointment is the amount of aggregation functions: https://questdb.io/docs/functionsAggregation Clickhouse provides hundreds of functions, many of which I use. It would be hard to even consider QuestDB with this amount of functions. I'll stay tuned, anyway. Keep up the good work!

We try to stick to ANSI SQL as best as we can, but we have own own dialect. This should be a good entry point: https://www.questdb.io/docs/select

QuestDB (https://www.questdb.io) might help. Import your file(s) by drag and drop directly into the web console. Once data is loaded, you can run SQL queries in the console to order, format, concatenate, join (including time-based joins), etc. Once finished, you can export your formatted results back to CSV. Also, it's pretty quick!

I edit a database newsletter – https://dbweekly.com/ – so tend to always have my eyes out for new releases, what's coming along, and what not. And I thought I'd share a few more things that have jumped out at me recently in case anyone's in the mood for spelunking. 1. QuestDB – https://questdb.io/ – is a performance-focused, open-source time-series database that uses SQL. It makes heavy use of SIMD and vectorization for the performance end of things. 2. GridDB - https://griddb.net/en/ - is an in-memory NoSQL time-series database (there's a theme lately with these!) out of Toshiba that was boasting doing 5m writes per second and 60m reads per second on a 20 node cluster recently. 3. MeiliSearch - https://github.com/meilisearch/MeiliSearch – not exactly a database but basically an Elastic-esque search server written in Rust. Seems to have really taken off. 4. Dolt – https://github.com/liquidata-inc/dolt – bills itself as a 'Git for data'. It's relational, speaks SQL, but has version control on everything. TerminusDB, KVRocks, and ImmuDB also get honorable mentions. InfoWorld also had an article recently about 9 'offbeat' databases to check out if you want to go even further: https://www.infoworld.com/article/3533410/9-offbeat-databases-worth-a-… Exciting times in the database space!

Also cool that when you open it in a new tab, the svg [0] is interactive! You can zoom in by clicking on sections, and there's a button to reset the zoom level. [0]: https://questdb.com/images/blog/2026-01-13/before.svg

Implement some none-trivial algorithm and then try to optimize the heck out of it. Focus on the concrete CPU, architecture, cache, and memory in the hardware you have in front of you. Forget portability for a moment. This comes to mind: https://www.morling.dev/blog/one-billion-row-challenge/ Read how others have done it. Here's an example in Java that goes relatively low-level to squeeze out performance: https://questdb.io/blog/billion-row-challenge-step-by-step/

71 seconds https://questdb.io/blog/billion-row-challenge-step-by-step/

Depends what you call “naive”, but the “idiomatic Java solution” from last week’s post (https://questdb.io/blog/billion-row-challenge-step-by-step/) clocked in at 71 seconds, or 1:11. And just running it on graal was down to 66. “Very memory constrained” would be a massive factor here, 1BRC is not really constrained (let alone very much so), it has 1 billion rows on a 32GB machine.

And yet people use ClickHouse quite effectively for this very problem, see the comment here: https://news.ycombinator.com/item?id=39549218 There are also time-series databases out there that are OK with high cardinality: https://questdb.io/blog/2021/06/16/high-cardinality-time-series-data-p…

QuestDB had a recent similar but probably clearer blog post about time-series specific SQL queries: https://questdb.io/blog/olap-vs-time-series-databases-the-sql-perspect…

> Unless I'm missing something, hashing function is fast compared to random bouncing around inside ram – very much faster then random memory accesses. So I can't see how it make a difference. In a GROUP BY, you may have a few hundred million rows, but only a few hundred groups within them. A slow function would slow down things dramatically in that case since the hash table remain small and data access is potentially linear. > Then you've got sorted data, in which case use a merge join instead of a hash join surely. This property is beneficial for GROUP BY which includes a timestamp or a function over timestamp. QuestDB organizes data sorted by time, so relying on insertion order may help to avoid redundant sorting if there is an ORDER BY clause with the timestamp column. As for merge join, we also use it in ASOF join: https://questdb.io/docs/reference/sql/join/#asof-join

They started with Java and tried to use as little GC as possible and started writing hot loops in C, C++ and Rust. Logic is in Java, other stuff is mostly native. https://questdb.io/blog/leveraging-rust-in-our-high-performance-java-d…

QuestDB engineer here: We use jlink to create images for selected platforms. This means not even JRE is a dependency: You unpack a tarball and you are good to go. See: https://questdb.io/docs/get-started/binaries/

it supports different commit modes (see [1]) - nosync, async, and sync. you can choose the more expensive but safer async or sync modes if you're willing to tolerate higher commit latency. [1] https://questdb.io/docs/reference/configuration/#cairo-engine

> They make zero claims that QuestDB is faster than Clickhouse overall Are you sure? Just one look at their website says differently. https://questdb.io/time-series-benchmark-suite/ I don't use these tools. I just wanted to point out that what you're saying is disingenuous.

Hmm - it was QuestDB, but maybe I didn't see it on HN (can't find it now..) Anyway here is the job link: https://questdb.io/careers/senior-cloud-engineer/

Thanks for reporting this! This is an old link, please use https://questdb.io/docs/guide/crud instead. I am currently updating the README and removing all dead links.

https://questdb.io/docs/crudOperations Has js errors and is not loading/page not found

Congrats! Also thank you for your awesome blog[0]! It's really the kind of technical gem I enjoy reading late at night :) [0] https://questdb.io/blog

something is off with your website. I just see images https://questdb.io/blog/2020/07/24/use-questdb-for-swag/

Just wanted to point out a small typo in the docs: "LASTEST BY" (https://www.questdb.io/docs/crudOperations)

Yes and no. Statically typed languages only know that data stored in some piece of memory was conforming to some kind of shape/ interface when it was first stored there. That's why tricks like SIMD Within A Register (SWAR) work at all. E.g. when you need to parse temperatures from string input very fast like in the 1BRC: https://questdb.com/blog/billion-row-challenge-step-by-step/ How does your type system help there? With static typing, you are doing specification and optimization at the same time, which is maybe necessary because compilers and languages are not sufficiently smart but also because of this mix it complicates reasoning about correctness and performance. Also static typing introduces a whole universe of problems with itself. That's why we have reflection or stuff like memory inefficient IP address objects in Java: For a simple IPv4 address normally representable using 4 bytes/ 32 bits Java uses 56 bytes. The reason for it is Inet4Address object takes 24 B and the InetAddressHolder object takes another 32 B. The InetAddressHolder can contain not only the address but also the address family and original hostname that was possibly resolved to the address. For an IPv6 address normally representable using 16 bytes/ 128 bits Java uses 120 bytes. An Inet6Address contains the InetAddressHolder inherited from InetAddress and adds an Inet6AddressHolder that has additional information such as the scope of the address and a byte array containing the actual address.…