Highly Scalable Blog

highlyscalable.wordpress.com · Personal blog · on HN since 2012

Highly Scalable Blog covers NoSQL data modeling, distributed algorithms, probabilistic data structures, and big data processing for backend engineers.

What HN saysCommenters recommend its NoSQL data modeling and probabilistic data structures articles as good reading, and one lists it among the blogs they subscribe to.
38HN people shared it
10linked it in comments
33stories submitted
1.2Kstory points
3 years agolast shared
2012people per year2026

Most-shared posts

  1. 1
    NoSQL Data Modeling Techniques
    10 people · 537 points · first shared 2012 · 30 comments on HN
  2. 2
  3. 3
    Distributed algorithms in NoSQL databases
    7 people · 112 points · first shared 2012 · 12 comments on HN
  4. 4
    In-Stream Big Data Processing (2013)
    5 people · 37 points · first shared 2013 · 3 comments on HN
  5. 5
    MapReduce Patterns, Algorithms, and Use Cases
    3 people · 226 points · first shared 2012 · 5 comments on HN
  6. 6
    Tricks with Direct Memory Access in Java
    3 people · 9 points · first shared 2012 · discussion on HN
  7. 7
    Fast Intersection of Sorted Lists Using SSE Instructions
    3 people · 8 points · first shared 2012 · 1 comments on HN
  8. 8
  9. 9
    Speeding Up Hadoop Builds Using Distributed Unit Tests
    1 people · 1 points · first shared 2012 · discussion on HN
  10. 10
    Performance of Priority Queue Sorting with Pagination
    1 people · 1 points · first shared 2012 · discussion on HN
  11. 11
    Homepage
    1 people · first shared 2012

What HN people wrote when they linked it

mynegation · 2018-04-14
zaroth · 2015-06-15
I read about an interesting technique, an "all-or-nothing tracker" in a blog post from an Apache Spark engineer. You dispatch n jobs, where n is quite large, and you want to know; have all n jobs have completed, or has less than n jobs have completed. How to do so with a small fixed number of bytes with very high probability? Give each job a random 128-bit ID number. XOR each ID number together as you start each job, and XOR into the same value as each job completes. If all the jobs have completed, the result is 0. The chance of zero turning up randomly if all jobs are not complete is negligible. The technique is mentioned here under 'Lineage Tracking': https://highlyscalable.wordpress.com/2013/08/20/in-stream-big-data-pro… but there's a better blog post I remembered reading but can't find at the moment...
virmundi · 2014-11-03
I don't think this addresses your question directly, but there is always Unsafe. You can do a lot of direct memory manipulations through it. http://highlyscalable.wordpress.com/2012/02/02/direct-memory-access-in…
bochi · 2014-04-01
This blog post is a nice survey about other interesting probabilistic data structures: http://highlyscalable.wordpress.com/2012/05/01/probabilistic-structure…
jbochi · 2013-10-28
There is nice summary of other interesting probabilistic data structures here: http://highlyscalable.wordpress.com/2012/05/01/probabilistic-structure…
SkyMarshal · 2012-03-04
As a fan of Codd/Date/Pascal, relational algebra, and ACID, I used to feel the same about NoSQL, but have since come to appreciate it as an option. Schemaless is nicer for exploratory programming with frequent data model changes, for one example. Here's a good article on NoSQL data modeling that covers it better, along with some good discussion in the comments: http://highlyscalable.wordpress.com/2012/03/01/nosql-data-modeling-tec…
sargstuff · 2023-12-31
NIST Dictionary of Algorithms and Data Structures[0] Is there a specific subject area(s) of application? aka big data analytics[a] / data mining and web analytics[b] / streaming applications[c] / other Random picks with mix of tutorials/examples/explainations/resouces from search engine term "probabilistic data structures": [1][2][3][4], probabilistic DS vs. postgresql[4] --- [0] : https://xlinux.nist.gov/dads/ [1] : https://www.geeksforgeeks.org/introduction-to-the-probabilistic-data-s… [2] : https://iq.opengenus.org/applications-of-different-data-structures/ [3] : https://dzone.com/articles/introduction-probabilistic-0 [4] : https://blog.devgenius.io/probabilistic-data-structures-1-definition-a… [5] : https://www.postgresql.eu/events/fosdem2020/sessions/session/2934/slid… ---- [a] big data : https://www.researchgate.net/publication/335418069_Probabilistic_data_… [b] data mining / web analytics : https://highlyscalable.wordpress.com/2012/05/01/probabilistic-structur… [c] streaming : https://www.usaclouds.org/blog/streaming-with-probabilistic-data-struc…

Keyboard shortcuts

/Search
j kNext / previous blog
EnterOpen selected blog
vVisit the blog site
fFollow / unfollow
lSwitch grid / list view
dSwitch dark mode
g hHome
g pTop posts
g tTopics
g fFollowing
g lLiked
g yHistory
g qFAQ
EscBack / close