Aerospike Blog

Hi Gary4gar, Read the points that you stated in your question. If you focus on growth (clearly your wins are going to bring more data updates as your increase the customer base) you should look into a solution that does not tie you down with data model and also does not expect you to worry about sharding in the future. Having done sharding and when one keeps growing the data volume and having to do shards it becomes painful and hurts the development flow. Been there many time and done it quite a bit myself. (Am a technical guys myself, http://www.linkedin.com/in/sukantag) Look for a NoSQL solution which will help in the flexi-schema model. Helps you add different types of data models relatively easily. Check out Aerospike (http://www.aerospike.com), the product scales like scaling problems have disappeared from this planet. All of the characteristics that you would need based on what you have stated is in the product. Best of all it is free for startups. Just give it shot. Open Source server and client libraries in multiple programming languages. (Am a big "C" and Python guys so like them client libs a lot). Lots of startups and many large software companies use them. If you like it and works for your application tell others, if not tell us what did not work. I ain't sellin and you ain't buyin; Ciao

Interesting project. Very similar to KeyDB [1] which also developed a multi-threaded scale-up approach to Redis. It's since been acquired by Snapchat. There's also Aerospike [2] which has developed a lot around low-latency performance. 1. https://docs.keydb.dev/ 2. https://aerospike.com/

Nice to see Linode adding another data center! Off-top, but (hopefully) interesting: We've been using Linode to host an Aerospike [1] cluster. They're one of the few cloud providers that give you direct raw access to the SSD drives. (Raw being you can mount an unformatted SSD partition). Performance has been stellar. [1] http://www.aerospike.com/

The rotated A in the Aerospike logo reminds me of a system that's fallen over, and now you can't unsee it: http://www.aerospike.com/

What do you guys think of Aerospike (http://www.aerospike.com/)? I heard of a few cases that used it successfully. Any issues with it?

Aerospike - distributed key value store has a little bit different of a use case than RethinkDB - distributed JSON document databases. Basically the only valid use that I have seen for Aerospike, and that is the one that they are advertising is a distributed key value profile store for Ad-tech or marketing companies (http://www.aerospike.com/overview/). Keep in mind that to actually use all of Aerospike's features , especially the one they are really proud about - the cross datacenter replication - you need the commercial license. So..I suggest you figure out what your requirements are and then use the best tool for the job.

Why do programmers databases with JavaScript? Lua is more embedded language than JavaScript. http://www.aerospike.com/ http://redis.io http://tarantool.org use LUA and it more powerful and easy to use as procedures.

Their site http://aerospike.com However, news of their VC funding (mentioned in the TNW article) has not made it onto their blog yet.

Unlikely you are going to overwhelm Redis in a non-deterministic fashion. What do you even mean by overwhelm? The post only concerns itself with in-memory workloads; I don't think Aerospike is competition in this space, while their advantage is against workloads working against SSD backed datasets. "After Google published a blog post “Cassandra Hits One Million Writes Per Second on Google Compute Engine” – using 300 nodes, we followed the same steps and documented how Aerospike Hits One Million Writes Per Second With Just 50 Nodes On Google Compute Engine. [...] Aerospike on SSD is very comparable to that of RAM. At 100% write, the SSDs are able to sustain 226,000 transactions per second compared to 239,000 for RAM." [0] Redis would have no problem hitting 250K IOPS on just a single core provided by GCE (or AWS EC2 or similar). [0] http://www.aerospike.com/resource/aerospike-soars-in-the-google-cloud/

This sounds like a job for http://www.aerospike.com/

You may also want to consider Aerospike database. Aerospike is an open-source database known for its high performance. If you can represent your timestamp as an integer, you can built a secondary index on top and do a range query on it. The range definition looks something like this: Filter.range(name, begin, end) ) More details: http://www.aerospike.com/docs/client/java/usage/query/query.html The advantages of Aerospike is its going to be superfast as the index is co-located with the data and also the secondary indexes are always updated inline with writes. So, the indexes always reflect the current state of truth. While doing the secondary index query, its follows a scatter gather approach thereby achieving distributed parallelism of the query. More details: http://www.aerospike.com/docs/architecture/secondary-index.html On top of this, if at all you need to perform an aggregation on the secondary query result, you can use the Aerospike's Distributed Streaming Aggregation framework (http://www.aerospike.com/docs/guide/aggregation.html). Where you an do the aggregation on the server side itself without pulling all the data to the client. Only the final aggregation happen on the client.

Putting a few important metrics in the logs every 10s is something that the Aerospike datastore also does (https://aerospike.com/docs/database/observe/latency). This is useful because when you contact support, they run a script to generate a table of historical latencies from the log without depending on you having set up Prometheus, CloudWatch, etc.

Linkified https://aerospike.com/products/features-and-editions/

“In hundreds of tests of SC mode through network partitions, 3.99.1.5 and higher versions have not shown any sign of nonlinearizable histories, lost increments to counters, or lost updates to sets.” – Kyle Kingsbury, Aerospike 3.99.0.3, 12-27-2017 - https://aerospike.com/blog/aerospike-4-strong-consistency-and-jepsen/ - https://jepsen.io/analyses/aerospike-3-99-0-3

> Redis doesn't provide a data structure for that. Both Redis and Aerospike provide more than enough mechanisms for one to roll their own composite hierarchy. Both provide server side scripting via Lua as well so the database can maintain consistency and prevent chatty access to the store. KV database modeling is just as rich as its relational counterpart, but if one isn't using a KV store for a compelling reason, use Postgres. https://www.aerospike.com/docs/guide/data-types.html https://www.aerospike.com/docs/guide/udf.html

This isn't new. I wrote this blog about beacons back in 2015 for the NoSQL vendor Aerospike; there aren't just audio (high frequency) beacons. There are also RF and other spectrum beacons: https://www.aerospike.com/blog/silverpush-unifies-people-devices-data/

Also PHP7 addresses a lot of the performance issues, getting it up into the node.js range and ahead of Python 3 / Ruby in the microbenchmark wars at least: https://www.aerospike.com/blog/php7_php5/ https://benchmarksgame-team.pages.debian.net/benchmarksgame/faster/php…

We do use it in production here at Adform. More about it here: https://www.aerospike.com/adform-divorces-cassandra-scales-4x-with-2x-…

you're right, the performance of the local storage on i3 instances is a lot better than normal EBS but still nothing close to what you would get from just buying an Intel P4500 or similar though. http://www.aerospike.com/benchmarks/amazon-ec2-i3-performance-results/

Not directly related to TRIM, but AeroSpike has a nice test suite for SSDs, probing for IOPS and latency: https://github.com/aerospike/act They share their test results for both physical and cloud-based storage, I figured this would be of interest: http://www.aerospike.com/docs/operations/plan/ssd/ssd_certification.ht…

How would you model timeseries on top of Aerospike? Using 1KB chunks of datapoints as simple K/V? Or using Large Data Types [1] like Large Stack [2]? Large Stack A Large Stack collection is naturally aligned with time series data because the stack preserves the insert order. Stacks provide Last In First Out (LIFO) order, so it's a convenient mechanism for tracking "recent" data. Usage examples include: Show me the last N events Search the last N actions with a filter Show me all documents in reverse insert order [1] https://www.aerospike.com/docs/guide/ldt.html [2] https://www.aerospike.com/docs/client/java/usage/ldt/ldt.html

As a Cassandra Committer, I would really appreciate you updating your ycsb integration so we can corroborate your benchmarks, as I currently doubt their authenticity/honesty. It looks very much like you compared non-durable performance in Aerospike to durable performance in Cassandra. It also appears from your documentation that you do not support any kind of safe active-active multi-dc mode, even in your paid-for offering (http://www.aerospike.com/docs/architecture/xdr.html), so even if faster than Cassandra, users should carefully read the fine print before deciding to use Aerospike, unless as the application grows in size and necessity (i.e. uninterruptibility in event of disaster) you want to find multi-dc is not actually viable for you. NB: I'm unaware of any adform contributions to Cassandra, although a senior developer there has filed a few bug reports.

Here's a benchmark from 2013 from Thumbtack Technology, comparing Aerospike, Cassandra, MongoDB and Couchbase... http://www.odbms.org/2013/01/ultra-high-performance-nosql-benchmarking… http://www.odbms.org/wp-content/uploads/2013/11/NoSQLBenchmarking.pdf In this benchmark, Couchbase gets some impressive results, but it does appear that Aerospike is the overall winner when it comes to speed and reliability. Anyway, the code is free to install, it's easy enough to validate the speed claims... http://www.aerospike.com/blog/aerospike-doubles-in-memory-nosql-databa…

Cool idea, I believe aeorspike uses ssd's for key value store like this. Check out http://www.aerospike.com/performance/architecture/ .