MapD

This is a great intro into the subject. We use LLVM to at MapD (http://mapd.com) to compile SQL queries to CPU and GPU machine code - it has given us a major boost over an interpreter based approach. For more see here - http://devblogs.nvidia.com/parallelforall/mapd-massive-throughput-data… If you have a background in LLVM or compilers in general and are interested in tackling problems like this please reach out at [email protected].

If you have a local setup, a ton of data, and a few good GPU(s)+ I would go bare metal with MapD (http://www.mapd.com) vs. the cloud.

It's not just about accelerating ML, specifically deep learning. There are many other enterprise technologies that can benefit from GPUs. One example: OLAP-focused databases (such as MapD - https://www.mapd.com/). For some benchmarks, check out this blog: http://tech.marksblogg.com/benchmarks.html. The DL "training" use-case is well-known at this point, but there are many others which are emerging.

MapD home: https://www.mapd.com/ Past discussion: https://news.ycombinator.com/item?id=9764726

Hi jwcrux, you can find more info on our website (http://www.mapd.com) and more specifically about the product here (http://www.mapd.com/product). In general, MapD is sold as licensed software but we sell to everyone from enterprise to startups, both on premise and in the cloud so if its of interest please write us at [email protected].

For further mappings of big multidimensional datasets (datacubes) to heatmaps: http://vis.stanford.edu/projects/immens/ , http://www.nanocubes.net/, and http://www.mapd.com/ . I'm not as familiar with the GIS space, but suspect a bunch there too.

In terms of "in general", there's mapd, a startup doing something on a very similar space http://mapd.com/ It's definitely the case that our work is not ready to be shrink-wrapped and sold. But the need for it is there. I'm no longer at AT&T Research and can't answer for their current needs, and I unfortunately cannot talk about the usage scenarios when I was there. There's the academic paper you can read, http://nanocubes.net/paper.pdf, which has a few examples. In practical terms, the difference in a system that has latency of a few seconds to one in which the queries come back in 60fps is significant, and enables a bunch of interesting stuff that might not be obvious at first sight.

Here is a paper on how to accelerate SQL database operations on a GPU: http://www.cs.virginia.edu/~skadron/Papers/bakkum_sqlite_gpgpu10.pdf (University of Virginia, 2010) and an ealier work: http://www.researchgate.net/profile/Qiong_Luo2/publication/237618362_R… (Hong Kong University of Science and Technology, 2009) And here are similar pojects: http://www.mapd.com/ https://github.com/antonmks/Alenka

This reminds me of MapD (http://www.mapd.com), though I am not sure which database MapD builds their stuffs on.

It's for in-memory databases: https://www.mapd.com

See also: https://www.mapd.com/

This article is a bit out of date at this point (2017), but it gets the history correct. To see an updated Tweetmap demo (the demo shown in the carousel), check out: https://www.mapd.com/demos/tweetmap/

FRom a series of Google searches and webcrawls i maintain for myself for my middle and upper managers. So this is not technical stuff http://www.infoworld.com/article/3123747/data-center/faster-with-gpus-… https://blazingdb.com http://diginomica.com/2016/04/11/do-gpu-optimized-databases-threaten-t… https://www.kinetica.com https://www.mapd.com and of course many many more scholarly articles through arxiv

From their product page [1]: > MapD Core is an in-memory, column store, SQL relational database that was designed from the ground up to run on GPUs. However, memory is getting cheaper and cheaper these days. [1] https://www.mapd.com/products/core/

The demo https://www.mapd.com/demos/tweetmap/ is pretty laggy, I think you should probably make it more performant so people don't think it's the DB that is causing the problem.

The link to the demo is broken in the blog post. It's actually: https://www.mapd.com/demos/taxis/

Hi infinite8s, to get additional information on how that chart was made, you can to go https://www.mapd.com/product/ scroll down to the bar chart, and click “See Details” under the chart. Shows the machines used, queries, and the source data set and size. Note that the machine configurations used to generate the chart were normalized for equivalent cost on AWS, i.e. the chart is hardware-dollar normalized.

Source: https://www.mapd.com/product/ It tops off at 192GB (8 x 24GB). Assuming in a couple of gens (#slots & gpu mem) will put it in 1TB range.

MapD does some of this, but with GPUs: https://www.mapd.com/product/

more info ( from : 2013 ) http://data-informed.com/fast-database-emerges-from-mit-class-gpus-and… now: Demo: http://tweetmap.mapd.com/desktop/ Blog http://www.mapd.com/blog/2015/06/04/mapd/

hi - i took a look at the sqlite db. not to nitpick, but you have 4 indexes there? the overhead of indexes at tiny data sizes like this is minimal, so no surprise that sqlite or pg will compare favorably at laptop scale (FWIW, i ran this on a 4-core macbook pro with no GPU (i.e CPU only) and without indexes, mapd runs the query at ~1.5 sec while sqlite takes ~6). however, the bigger point is about how this experience scales. take a look at this public demo - https://www.mapd.com/demos/ships. it's 11 billion geo data points, and you'll see the same low millisecond response that FridgeSeal is talking about - again, no indexing (to verify, go to dev tools in your browser, turn on SQLLogging(true) in the console and look at the query times reported for the round trip. i'm sure you appreciate that the overhead (both creation and maintenance) of indexing scales with data size, plus sqlite/pg dont (AFAIK) offer anything by way of GPU-rendered charting.