Turbopuffer Blog

It works great. We’ve had SPANN in production since October of 2023 at https://turbopuffer.com/

Surprised no one has mentioned Turbopuffer yet [1] which natively supports dense vector similarity and BM25 keyword indexes out of the box [1]. https://turbopuffer.com/

Very interesting, thanks for sharing. This has a lot of nods to Turbopuffer's architecture [0]. My impression is they've spent a lot of time optimizing at the hardware/firmware layer to achieve extremely fast query results. Inarticulately - how ~close is OpenData Vector to Turbopuffer in terms of performance today and where are the major gaps + mountains to scale? Really excited to keep an eye on the repos, great read! [0]https://turbopuffer.com/blog/turbopuffer

For local dev + testing, we recommend just hitting the production turbopuffer service directly, but with a separate test org/API key: https://turbopuffer.com/docs/testing Works well for the vast majority of our customers (although we get the very occasional complaint about wanting a dev environment that works offline). The dataset sizes for local dev are usually so small that the cost rounds to free.

With the new GC I really love the trend of understanding memory bandwidth is the bottleneck for many things and the combination of locality and SIMD is a big performance unlock. Reminds me of the WAND vs MAXSCORE discussion by turbopuffer: https://turbopuffer.com/blog/fts-v2-maxscore

> My other mild criticism is in the discussion on TigerBeetle’s consensus: yes, it seems quite clever and has no other dependencies, but it’s also not trying to deal with large rows. When you can fit 8,190 transactions into a 1 MiB packet that takes a single trip to be delivered, you can probably manage what would be impossible for a traditional RDBMS. Isn't that the point? They're saying to separate out transactions workload from other workloads. They're not saying they'll replace your OLGP db, you remove transactionally important data into another db. It's something similar that we see with another db: https://turbopuffer.com/

Turbopuffer is awesome, really recommend it. Also they have extra features like automatic recall tuning based on you data, option to choose read after write guarantees (trading latency for consistency or vice versa), BM25 search, filtering on the filed and many more. Really recommend to check them out if you need a vector DB. I tried qdrant and zilli cloud solutions and in terms of operational simplicity turbopuffer just killing it. https://turbopuffer.com/docs/query

Anyone interested in this space should look at https://turbopuffer.com - I think they were first to market with S3 backed vector storage, and a good memory cache in front of it.

Website is open source at https://github.com/TN1ck/tn1ck.com. Tech stack is literally just a Next.js website and I write my articles in React. I tried other things before e.g. Jekyll, but I found that dynamic content is really hard to do there, annoyingly hard. I keep it super simple and don't do "the proper way" of things at times (e.g. the blog index is manually done by me). But that keeps it simple & independent to me. Next.js here is just a detail, I can always move to some other React-based static site generator. It's hosted at Cloudflare. The design is heavily inspired by https://turbopuffer.com/, I don't deserve any praise for that.

We use pinecone and it is not ideal, looking at https://turbopuffer.com/ now. They look quite promising :)

Many of them are open source and you can host them yourself. That would make it more cost effective. Also someone mentioned https://turbopuffer.com/. That seems like a good alternative if you're looking for something economical.

You might want to add https://turbopuffer.com/ as well now in the benchmarks.

What a surprise that you are also a reader of books rather than other content: https://www.goodreads.com/user/show/7288759-vasco Reading a book is arguably easier than reading technical content or other articles, because they don't tend to be as information dense. I prefer reading things like this: https://commoncog.com/becoming-data-driven-in-business/ But someone might like reading documentation for e.g. databases, which is also extremely information dense. I have briefly done this (https://turbopuffer.com/docs) and read other database content (https://artem.krylysov.com/blog/2023/04/19/how-rocksdb-works/), but it's not my favourite content. Really weird to say "you read X, I read Y. We are not the same" and then follow-up with "oh but it's fine, you don't have to. But I'm X and you're Y". I could replace a few words in your comment and make it about thinking reading books is the same as reading extremely dense technical content. It's a silly argument.

Copy aside I think the approach is similar to https://turbopuffer.com/blog/object-storage-queue The quiet improvements to S3 over the last 3 years like conditional puts unlocked a lot of use cases not previously feasible

(author here) The 92% mentioned in this post is showing recall@10 across all 100B vectors, calculated by comparing to the global top_k. turbopuffer will also continuously monitor production recall at the per-shard level (or on-demand with https://turbopuffer.com/docs/recall). Perhaps counterintuitively, the global recall will actually be better than the per-shard recall if each shard is asked for its own, local top_k!

Your vector store should let you store the original text as well as metadata, where you can store the version. For e.g. turbopuffer lets you filter on attributes https://turbopuffer.com/docs/query#filtering

This feels similar to TurboBuffer, which is also built on top of S3 storage. TurboBuffer has been a leader in this space, powering vector search for major companies like Cursor, Linear, and Notion. It seems AWS is leveraging its strong S3 brand to compete directly in the vector database market. For more context, check out the TurboBuffer architecture docs and Notion’s presentation from the Data Council: https://turbopuffer.com/docs/architecture https://www.youtube.com/watch?v=_yb6Nw21QxA (anti-disclaimer: I'm not affiliated with TurboBuffer in any way.)

It seems that some of the goals and functionalities of Nixiesearch overlap with those of Turbopuffer [^1], though the latter is only focusing on vector search. I also resonate that search engine should be stateless and affordable to deploy for everyone. [1]: https://turbopuffer.com/blog/turbopuffer

At very high scale, there's less usage of graphs. Or there's a set of clustering on top of graphs. Graphs can be complex to build and rebalance. Graph-like data structures with a thing, then a pointer out to another thing, aren't that cache friendly. Add to that, people almost always want to *filter* vector search results. And this is a huge blindspot for consumers and providers. It's where the ugly performance surprises come from. Filtered HNSW isn't straightforward, and requires you to just keep traversing the graph looking for results that satisfy your filter. HNSW came out of a benchmark regime where we just indexed some vectors and tried to only maximize recall for query latency. It doesn't take into account the filtering / indexing almost everyone wants. Turbopuffer, for example, doesn't use graphs at all, it uses SPFresh. And they recently got 200ms latency on 100B vectors. https://turbopuffer.com/docs/vector