Heap Blog

Highly recommend Heap [1] - they have a neat approach that doesn’t require you to ‘decide’ which analytics you want to track ahead of time. Disclaimer: I was an early engineer at Heap. [1] https://heap.io/

Heap (YC W13) | Software Engineer | San Francisco, CA | New York, NY | Remote | Fulltime | https://heap.io Heap is automating the collection of product data by capturing everything: clicks, submissions, taps, etc. so that our customers can analyze product metrics instantly and retroactively, without writing code. We're a ~200-person company, looking for communicative, collaborative engineers to build a great product with us. If you're interested, visit https://heap.io/careers/jobs. I'm an engineer in the platform engineering area, so if you have questions about general engineering-related things, you can email [email protected]

Heap¹ is virtual-first and actively hiring (not to mention, scaling like crazy). I have colleagues in North America, Europe, Asia, and Australia. We have an excellent engineering culture and need more hands on deck. :) Hit me up (email in profile) if any of the roles listed here² look interesting to you. ¹ https://heap.io/ ² https://grnh.se/d7ce9bfc1us

For business it's: twitter(https://www.twitter.com), linkedin((https://www.linkedin.com), heap(https://www.heap.io/), and zigpoll (https://www.zigpoll.com/) For life it's: HN, reddit, WSJ, NYTimes It does help to pause the loop for as long as possible though. I think the frequency of checking increases linearly with anxiety.

Aren't services like Heap effectively worse versions of this? On their landing page they outright list this as their value prop: > Heap collects all the data on your customers - automatically. What they click. Where they go. What they do, even when you’re not looking. All without the need for engineers. [1]: https://heap.io/

For any who want this kind of pipelining way-of-writing-SQL that has the benefit of existing in live production databases today, I highly, highly recommend looking into Postgres' and Snowflake's LATERAL JOIN keyword. https://heap.io/blog/postgresqls-powerful-new-join-type-lateral https://stackoverflow.com/questions/28550679/what-is-the-difference-be… https://docs.snowflake.com/en/sql-reference/constructs/join-lateral.ht… The TL;DR is that they allow you to reuse annotated and aggregated columns in an incredibly elegant way. Compared to OP's proposal, you still do need to start the query with what columns you want to come out at the end, and normal SQL weirdnesses still apply - but it's far, far, easier when writing massive analytics queries to see the flow of variables from one stage to another.

Author here, surprising to see this blog post of mine trending, but it has held up pretty well! Happy to answer any questions about my time at Stack Overflow. I've since moved on to Heap (https://heap.io/) and we are hiring. We're virtual first so we don't have private offices, but we have the same values about treating developers well and giving them the space they need to do deep work. We're hiring for basically everything https://heap.io/careers/departments/engineering-product-and-design?tea…

Good read - fwiw if this is your blog some of your links are broken and think they are local - https://heap.io/blog/%E2%80%9Dhttps://github.com/apache/spark/pull/240…

Heap (YC W13) | Software Engineer | San Francisco, CA | New York, NY | Remote | Fulltime | https://heap.io We’re a small team building the analytics engine that powers 6,000 companies and helps them make smarter decisions. Heap automates away the pain of data. Other analytics tools require you to define events upfront and manually instrument code. Instead, Heap automatically captures everything: clicks, submissions, taps, etc. This lets our users analyze data instantly and retroactively, without writing code. Our app-layer stack is TypeScript, React, Node.js, Redis, and PostgreSQL. Under the hood, Heap is powered by ZooKeeper, Kafka, and Citus. If you are interested, please email at [email protected] or visit https://heap.io/careers/jobs.

For those not familiar with Posthog, it's open an open source product analytics tool. That means you install some javascript on your website. It then collects information on what features people are using on your website and let's people run charts and graphs to ask questions about feature engagement and product usage. Product analytics is a pretty large space. The three main existing product analytics tools are Mixpanel[0], Amplitude[1], and Heap[2] (I used to work at Heap). Notably, Mixpanel was valued at $865MM as of 2014 and Amplitude was valued at $1B as of two weeks ago. Posthog is an open source competitor to these tools that you are able to self host. This is interesting for several reasons. If you don't feel comfortable sending data to third party tools, you control the data yourself. In addition, existing product analytics tools can get expensive. After you get off the public pricing, you will likely find yourself paying >$12k/year for one of these tools. There are also challenges I see for them going forward. In my experience, the person who makes the decision to buy a product analytics tool isn't in engineering. It's usually a product manager or sometimes a marketer. I imagine the fact that it's open source is less interesting to one of the traditional product analytics purchasers. Compare this to a company like Gitlab where the end users are engineers. Scaling these tools is also a huge challenge. My job at Heap was to basically scale Heap. If you want to…

A truncated Y-axis exaggerates the relative change, it's the simplest and most common method of being misleading with data. https://www.heap.io/blog/how-to-lie-with-data-visualization

Indeed, see for example https://www.heap.io/blog/how-to-lie-with-data-visualization

I work in the Database industry and have enjoyed these posts this year. - https://arctype.com/blog/postgres-notify-for-real-time-dashboards/ - https://heap.io/blog/speeding-up-postgresql-queries-with-partial-index… - https://blog.timescale.com/blog/how-postgresql-aggregation-works-and-h… - https://blog.crunchydata.com/blog/understanding-postgres_fdw

Postgres is not designed for OLAP, but you can push it a lot farther than one would expect with the correct schema and indexing strategy. See https://heap.io/blog/running-10-million-postgresql-indexes-in-producti… for a little more detail about how we schematize for distributed OLAP queries on Postgres at scale.

Good point, we didn’t really get into that part of the story. At Heap, we have a concept of a “virtual event”, in which there’s a raw event data set, and then you (the Heap customer) would “define” events on this raw data through our UI. The resulting data set is what you’d normally get from traditional `analytics.track` code, except it would be retroactive to the point when you installed Heap, you can change/delete/create events (without losing the underlying data), and you wouldn’t have actually written any additional tracking code. We have some material on virtual events on https://heap.io/product/virtual-events, and the code that extracts the raw events from these interactions can be found at https://github.com/heap/react-native-heap/blob/c2dae7ef2442a67619670e7… (the function our babel plugin hooks into) and https://github.com/heap/react-native-heap/blob/c2dae7ef2442a67619670e7… (the function that creates most of the raw event we send to Heap)

> That couch business wasted both your time AND the sales rep's time. Who does it help to not post prices? Because there isn't a price to show. The missing detail I left out in the original comment is that the sale rep also said "We have hundreds of fabric and color options available, and we can accomodate any configuration." Sure, I guess they could post prices for the models sitting on the floor, but that price would only be for that exact configuration. The point is, the sale person wants to spend 1-2 hours working with the customer to determine their fabric preferences, then color preferences, then configuration / shape of the couch. The store does not cater to people who want the ability to walk in, point to something and say "I want that, how much?" - those people are not their customers. > For B2B, sure, you can't give an exact quote until you figure out the custom logic In the couch example, it's the same, exact "custom logic" is instead fabric, color, configuration, pull out vs. no pull out, ottoman vs no ottoman, etc. (For the record, I hate non-transparent pricing as much as anyone. Heap is an example that has been on my mind lately since we're looking at analytics solutions at our company: https://heap.io/pricing - it's impossible to infer the "contact us" pricing, since the only other plan they have is "Free" -- we're on their "Startup" plan for $499/mo, and I have no idea if we're being ripped off, or if that's what everyone else is paying too...)