Dgraph Blog

I believe the best of the best use of Golang is being done by Dgraph. They are the The world’s most advanced native GraphQL database with a graph backend. Checkout their GitHub repo. https://dgraph.io/ https://github.com/dgraph-io/dgraph

PostgreSQL in earnest - I would love to see that in DGraph https://dgraph.io/

Author of https://dgraph.io here. I'd say this is one of the best collaborations we've done. Due to Jepsen tests, we were able to proactively identify and fix issues in a complex real time distributed graph DB, that is Dgraph. Dgraph's graph sharding, replication and transaction implementation is unique and it took more work to make it all work correctly, while still achieving good performance, and without utilizing any third party system. I'd like to thank Kyle for his work on Dgraph. We'll continue to expand Jepsen tests to try and identify more issues and fix the remaining ones. I'm around if you have questions! Cheers, Manish

My favorite one is Dgraph https://dgraph.io/ https://github.com/dgraph-io/dgraph This is an excellent choice if you are looking for a rock solid scalable graph database. Its written in Go, which for me is my language of choice for backend scalable fast solutions. The community is helpful and available. Plus it uses Badger for long term persistence. Which is also a big bonus... https://github.com/dgraph-io/badger The combination of these two open source projects should definitely be reviewed by anyone considering a new project using graph databases... One nice thing about Badger is that it is a tool similar to Rocks DB... So Badger can be used standalone if you want a nice fast key / value DB written in Go as well.

Check out https://dgraph.io/ . I haven't had the chance to use it yet, but I have been following its development for a while. Definitely worth looking into.

I've been watching https://dgraph.io with great interest. Looks like it could become a very solid contender as a horizontally scalable open source graph database engine.

DGraph | Remote What - DGraph(https://github.com/dgraph-io/dgraph) is an open source Scalable, Distributed, Low Latency, High Throughput Graph Database written in Go. We want to build the best Graph database in the world. Hiring - We are looking for outstanding backend engineers with 3+ years of experience. The process includes 2 technical interviews, followed by 2 months of full time paid Bootcamp program to ensure that you’re a great fit and vice-versa. Perks * Our salaries are open and transparent * Macbook pro * $2500 towards a workstation and desk of your choice * Kindle for every team member, read only Fridays and unlimited books. * Medical coverage for you and your dependents and many more For more details see https://dgraph.io/#jobs. If you feel you would be a good match, get in touch with us at https://angel.co/dgraph-labs or [email protected]

Dgraph Labs | Palo Alto, Bangalore, Remote | Hiring Community Engineers, Technical Writers, Distributed Systems, Frontend, and Fullstack Engineers | Full-time, VISA | https://dgraph.io/ Dgraph Labs was started by ex-Google engineers and is VC funded. We are a team of dedicated engineers with a mission to build the world's best graph database. Nothing less excites us! Dgraph has solid open-source traction with over 13,000 GitHub stars and over 2 million Docker Hub pulls and is being used in production at multiple Fortune 500 companies. We are hiring for community engineers, frontend engineers, fullstack engineers, technical writers, and distributed systems engineers. We love what we do and take care of each other while hand-crafting the best graph database. Stack: Go, ReactJS, Javascript, Docker, k8s More details: https://jobs.lever.co/dgraph If this sounds exciting, drop-in a mail to [email protected], also mention HN in the subject line.

Looking forward comparison with Dgraph ( https://dgraph.io/ ) — I mentioned Dgraph in other, older, posts. I'm not a shill, just a Dgraph user who's looking for alternative.

import "C" import "Unsafe" is all you need for manual memory management and pointer arithmetic with Go. Since the GC is fairly light and snappy, I find it's best of both world to only resort to Unsafe for the performance critical parts, and leave the GC deal with the rest. For more serious performance requirements, consider using jemalloc: https://dgraph.io/blog/post/manual-memory-management-golang-jemalloc/

> TinyLFU does not handle dynamic changes to demand to the cache like 2Q does That was resolved in a follow up paper [1], though Ristretto does not implement this. It is implemented by Caffeine, mentioned briefly in this article, which is popular in the Java ecosystem. [1] https://dgraph.io/blog/refs/Adaptive%20Software%20Cache%20Management.p…

The post introducing Ristretto, a similar Go cache is also a great technical read: https://dgraph.io/blog/post/introducing-ristretto-high-perf-go-cache/

EdgeQL seems like a custom graphQL. Any reasons you did not follow dgraph's DQL or graphQL-compatible language? https://dgraph.io/docs/get-started/

I've been intrigued by dgraph (https://dgraph.io) and used it to good effect in a (toy) project where it felt easy to create and evolve it's data model given changing requirements. Dgraph uses graphql as its native query language. Anyone here has some experience to share on it? ... Since it isn't mentioned in the article.

Not sure why, but the blog post in question (same URL), has been sneakily replaced with one replacing "graph database" with "GraphQL" in the title. And changing the tagline on relational database from "Relational databases: a software engineering fail" to "Relational databases: engineering success is not always the best software engineering fit" https://dgraph.io/blog/post/graphdb-for-your-next-app/ Fortunately internet archive still has the previous version, that the comments here were discussing. https://web.archive.org/web/20200709192510/https://dgraph.io/blog/post…

I'd say dgraph (https://dgraph.io/) is closest competitor but Neo4j (https://neo4j.com) as well which has longer heritage. I'd also include redis because of the graph module (https://oss.redislabs.com/redisgraph/). I've likely missed a bunch of others. Add them as I'm interested in graph db and have only scratched the surface myself.

ArangoDB is fine, but dgraph [0] can scale easily to that size (we have order of magnitude larger graph). In reality, you can use any resilient key/value store and use hexastore [1] as a storage format. For instance JanusGraph [2] has support for a lot of different backends, built by the old team behind TitanDB. [0] https://dgraph.io [1] http://karras.rutgers.edu/hexastore.pdf [2] https://janusgraph.org

> In general, compared to more typical solutions, you have more expensive ... query costs Not to detract from your general point, but curious whether you looked at Dgraph in your analysis. It's quite fast and was built for speed. https://dgraph.io/

yeah, my experience was the same. It's a really nice database to play with or for a small project but if you need a multi-node HA cluster, the price quickly becomes prohibitive. I guess if I ever need a graph database again I will probably go for dgraph (although I haven't used it in any production environment) - https://dgraph.io or any other graph database that at least has HA setup without 100k/year bill :)

I agree it seems like potentially the same problem, and it manifests in forcing devs to write 'resolvers' which looks like horrible drudge work. So I wonder if there are decent offerings without the impedance mismatch between GraphQL queries and the database (so we don't have to translate them into SQL). I did a very quick look and at least found this: https://dgraph.io/ I'm getting the impression that if nothing else though, I'm probably gonna need to wait a few years for this tech to stabilize more. Which I find very unfortunate, because the standard way of doings things these days feels very unpleasant to me (I hate writing boilerplate more than anything: have an overuse injury, so typing is the worst part of coding)—and the GraphQL queries do seem to make a lot of sense (although forcing clients to have explicit knowledge of schema structure seems a little dangerous...).

Dgraph.io (https://dgraph.io) | Backend Engineer, Golang | Sydney, Australia | ONSITE, REMOTE | Full time You will be responsible for the design, architecture, and implementation of our native and distributed open source graph database, Dgraph (https://github.com/dgraph-io/dgraph). Apply at https://boards.greenhouse.io/dgraph

Dgraph.io (https://dgraph.io) | Backend Engineer, Golang | Sydney, Australia | ONSITE, REMOTE | Full time You will be responsible for the design, architecture, and implementation of our native and distributed open source graph database, Dgraph (https://github.com/dgraph-io/dgraph). Apply at https://boards.greenhouse.io/dgraph

Exactly what https://dgraph.io is built for. Low latency, high throughput.

Good work! An alternative is Dgraph https://dgraph.io/ which I am considering for my next project.

I've followed Dgraph off and on. I enjoy looking through their open sourced work. A lot of it is interesting. Manish seems to be THAT person. The technically competent, unwavering to the technical vision, doesn't work well with others person at your company. He went on to found a startup so he could do what he wants, and unfortunately has now realized board members are very similar to managers at Google. Here's a blog post where he stepped down from being CEO to being CTO: https://dgraph.io/blog/post/changing-guard/ > I’m very confident in my abilities to bring in a solid VP team and lead the GTM motion. But, I’ve also realized that Dgraph is an intrinsically technical product. While I can find other S&M folks, finding a true engineering replacement for me has been, and will continue to be a real challenge. Reviews from Glassdoor confirm this inability to let others touch his baby: https://www.glassdoor.com/Reviews/Dgraph-Labs-Reviews-E1284992.htm > Massive layoffs with no warning and almost no severance pay. This is a sinking ship, not a rocket ship - avoid it! Inept founder (was CEO and CTO while I was there) who only believes in his ability and doesn't trust employee expertise. He would rather be right than make money or see his company succeed. Manish doesn't listen to employees or managers and even disputes data showing false assumptions or failed business approaches. Micromanages in departments where he has minimal knowledge or understanding, leading to low morale…

LMDB and RockDB are both great projects, but how much performance you can get out of them would depend on the larger architecture of the system they are being integrated into. Both projects provide tens of flags to adjust read/write amplification to customize them for a particular use case. You will often see public benchmarks of these systems being updated with suggestions from both sides on less-than-optimal configurations being used! In the case of Typesense, RocksDB is not even a top-10 contributor to the overall latency involved in serving the result. In any case, it would be good to clarify a few things: > As we don’t use RocksDB but LMDB, we use a lot less real memory than key-value stores that uses a user-side cache system. Typesense stores only the raw-data in RocksDB. All indexing data structures for filtering, faceting etc. are compact in-memory data structures stored outside. The only fixed memory cost from RocksDB is an in-memory table that is used to buffer writes (see the next point) before being flushed to disk. In practice, this is a trivial percentage of memory used when compared to other data structures. > LSM-Tree key-value store like RocksDB that needs to merge/compact pages of key-values pairs before being able to return it too you This happens in-memory and is flushed to the disk in batches. Merging of on-disk SST files happens in the background with no real impact on reads. The advantage of this approach though is that it gets you really good…

Looks like the big challenge is managing a large, LRU cache, which tends to be a difficult problem for GC runtimes. I bet the JVM, with its myriad tunable GC algorithms, would perform better, especially Shenandoah and, of course, the Azul C4. The JVM world tends to solve this problem by using off-heap caches. See Apache Ignite [0] or Ehcache [1]. I can't speak for how their Rust cache manages memory, but the thing to be careful of in non-GC runtimes (especially non-copying GC) is memory fragmentation. Its worth mentioning that the Dgraph folks wrote a better Go cache [2] once they hit the limits of the usual Go caches. From a purely architectural perspective, I would try to put cacheable material in something like memcache or redis, or one of the many distributed caches out there. But it might not be an option. It's worth mentioning that Apache Cassandra itself uses an off-heap cache. [0]: https://ignite.apache.org/arch/durablememory.html [1]: https://www.ehcache.org/documentation/2.8/get-started/storage-options.… [2]: https://blog.dgraph.io/post/introducing-ristretto-high-perf-go-cache/

Dgraph Labs | San Francisco, CA or Bangalore, India | ONSITE We announced our Series A funding this week[1]! Dgraph is the world’s most advanced graph database. It is horizontally scalable, synchronously replicated, transactional and distributed. It can do arbitrarily deep joins, while minimizing network communication and disk seeks, scaling well as cluster size increases. It is a low-latency, high-throughput database, serving complex queries over multiple independent data sources in real-time[2]. Dgraph Labs was started by ex-Google engineers. We are a small team of dedicated engineers with a mission to build world’s best graph database. Nothing less excites us! Open Positions: - Quality Assurance Engineer - DevOps Engineer - Developer Advocate - Distributed Systems Engineer - Full-Stack Engineer Apply here: https://dgraph.io/careers. [1] https://blog.dgraph.io/post/how-dgraph-labs-raised-series-a/ [2] https://github.com/dgraph-io/dgraph

You can allocate memory manually in Go and manage it yourself using Go's `unsafe` package, so there's an escape route once Go's GC became a limiting factor. Here's a blog post about it in the context of Dgraph: https://dgraph.io/blog/post/manual-memory-management-golang-jemalloc/