Piekniewski's Blog

I don't know why HN loves Sam Altman so much. Has he ever written a significant piece of software or invented anything? Has he ever built a significant, profitable business from the ground up? The guy was a fundraiser superstar when money was easy to get in the 2010s but I have yet to see technical or business competence sufficient to grant him the level of respect many in this community seem to grant him. There are people who are technically competent or genuinely intelligent who can also raise funds. The guy reminds me of Elizabeth Holmes or Eliezer Yudkowski of "lesswrong." A sophomoric guy with minimal technical or business accomplishments who is just smart enough to tell rich people what they wanna hear to fuel his game. There is an old term for this personality type. Snake oil salesman aka con artist. See https://blog.piekniewski.info/ For a sane take on AI by someone who actually got an education and is an expert in his field.

Yes, The thing about self-driving you can get to 90% or 99% or even 99.9% success and then find the cost of closing part of that extra sliver of non-success is the same as all the work you've done. Especially since testing those corners cases is hard since they are unusual. There's even some part of the sliver that even is "AI complete". Very small but you don't know where it is, etc. But I'd say that's going to happen about as quickly as changing the infrastructure in this country. Key point! Automation has traditionally been much, much easier when you are engineering a situation where you control all the variables. Automated factories are common. Automated warehouses are coming. Automated dock systems exist. In these situation, there's a hard border between "inside" and "outside". Roads are generally regular but the border between insight and outside fuzzy. Google can run it's self-driving taxis in a suburb in Arizona where roads are as regular as could be imagined but expanding beyond that is much, much hard than one imagines. Note: I recommend this blog which periodically tracks progress in self-driving cars as well as making intelligent, critical comments on current AI. https://blog.piekniewski.info/

Yes, this is very cool stuff and I've been advocating for it in previous posts. There is indeed the problem of prediction. Prediction seems to be an omnipresent paradigm in many aspects of perception, even in early vision. This, contrary to beating deep learning models to death, may hold the key to artificial intelligence. I have my own little model which I propose for prediction, I call the Predictive Vision Model, more info here: http://blog.piekniewski.info I'm looking for more people who see the potential of this.

In a sense, I would say yes there are learning laws, but it's still early in codifying them. Along one axis, you could compare: supervised, semi-supervised, self-supervised and unsupervised learning. Along another axis, consider that there are versions of each method that take into account temporal/dynamic data, versus others that require randomly shuffled static data. In the current problems of visual perception, I think the field would benefit greatly a shift to focus on multiscale interaction/dynamics rather than on (static) statistics as it is currently (for more on this, see my colleague's blog: [1]). [1] Statistics and dynamics. (http://blog.piekniewski.info/2016/11/01/statistics-and-dynamics/)

That reminds me of deep learning. Investment in computing resources is definitely increasing exponentially over time: https://openai.com/blog/ai-and-compute/ https://techcrunch.com/2019/11/10/the-post-exponential-era-of-ai-and-m… Tthere doesn't seem to by any evidence of exponential gains in performance, even on their own benchmarks (which importantly don't measure "intelligence", but something more narrow): https://blog.piekniewski.info/2019/11/18/late2019-the-wizards-of-oz/ Even though we are doubling the compute available to AI every 3.5 months, so a factor 10x per year, since say 2015, so some 10.000x, the classification performance as measured by top-1 accuracy on ImageNet had barely moved. Now arguably top-1 accuracy on ImageNet is probably not the best measure, but still that seems somewhat striking. The diminishing returns are visible elsewhere too, even that Open AI Rubic's cube is somewhat indicative. From an outside perspective, it seems that everyone is coming around to the fact that self-driving cars is harder than expected. There are no "exponential gains" in that field -- to the contrary, it seems like 90% of the problem was solved several years ago, and you can now take undergraduate courses that teach the basics. But going from 90%, to 99%, to 99.9%, etc. is taking decades, not months or years. There seems to be something fundamentally wrong with the current approach. I believe at the time the OpenAI blog post was supposed to be…

There are dozens of papers "explaining" neural networks, using basic math, highly abstract math and even theoretical quantum physics[1] models. I don't see any of them used in constructing, improving or interpreting results of actual neural networks[0]. Moreover, while most of them have math that would take me weeks to fully understand, they rarely make concrete, testable predictions. Most of the recent advances in ANN learning came from using more data and using different types of layers. A real theory of neural nets would explain why architecture A works better than architecture B for a particular problem. Or why a certain architecture plateaus after working with X samples, while other can learn from many more. The way I understand it, the core problem of understanding deep learning is that it does not scale down. You can make a simple framework of 3-4 neurons and observe how it works. Then you make it bigger and it works differently. No one properly explained why this happens. People who do applications seem to just state their intuitions and then throw a couple formulas on top to make it seem scientific. There is a great essay on that by Piekniewski[2]. No research I've seen lists all the parameters/architectures/datasets they tried that didn't work. [0] - Maybe with the exception of the paper on neural networks as ODEs. [1] - http://neurophysics.huji.ac.il/sites/default/files/PhysRevA.32.1007.pd… [2] -…

I've found this series illuminating. https://blog.piekniewski.info/2020/03/01/av-safety-myths-and-facts-202…

Except that autonomous cars vastly far away despite all the hype by silicon valley. They still cannot deal with edge case scenarios at all well and are far from being proven to be safe. Relying on this being in place by 2030 + lifetime of 2nd hand cars is a huge gamble. See https://blog.piekniewski.info/2020/03/01/av-safety-myths-and-facts-202… and his blog in general for a more in-depth analysis of the situation. It's likely we need new AI techniques to have fully autonomous vehicles that function correctly.

What do you make of this 2018 blog post and predictions discussing Uber, Waymo, comma.ai, Tesla, Google, General Motors, etc. https://blog.piekniewski.info/2018/10/29/ai-winter-update/ ?

Here's a good example from 2018: "Deep learning (does not) scale... We can't just scale up AlexNet and get respectively better results - we have to fiddle with specific architectures, and effectively additional compute does not buy much without order of magnitude more data samples, which are in practice only available in simulated game environments." https://blog.piekniewski.info/2018/05/28/ai-winter-is-well-on-its-way/ But Gary Marcus is the gold standard. Check out his 2018 take-down attempt, which is clearly mostly wrong: https://arxiv.org/ftp/arxiv/papers/1801/1801.00631.pdf

Switching speed is very fast with silicon, but we seem to be nowhere near the computing power of the brain, which incidentally runs on an absurdly low 20W. I think this power usage aspect is quite underappreciated. You might like this article: https://blog.piekniewski.info/2018/08/28/fun-numbers-about-the-brain/

So Lex Fridman curated this page and is the subject of many videos within it. Seems like Missy Cummings and Filip Piekniewski's assumption that MIT has scrubbed him from the site is unfounded. [1] https://twitter.com/missy_cummings/status/1179497003635662850 [2] https://blog.piekniewski.info/2019/11/18/late2019-the-wizards-of-oz/ Lex's blocking of dissenting opinions on twitter is still egregious though.

https://blog.piekniewski.info/2016/11/28/more-thoughts-on-self-driving… Is pretty good. For a more recent take he has also written some blogs on the short comings of AI in general which has some self driving car content. https://blog.piekniewski.info/2018/05/28/ai-winter-is-well-on-its-way/ https://blog.piekniewski.info/2018/10/29/ai-winter-update/

AI Circus, Mid 2019 Update https://blog.piekniewski.info/2019/05/30/ai-circus-mid-2019-update/ Stories of AI Failure and How to Avoid Similar AI Fails in 2019 https://www.lexalytics.com/lexablog/stories-ai-failure-avoid-ai-fails-… How IBM Watson Overpromised and Underdelivered on AI Health Care https://spectrum.ieee.org/biomedical/diagnostics/how-ibm-watson-overpr… Automakers Are Rethinking the Timetable for Fully Autonomous Cars https://www.designnews.com/electronics-test/automakers-are-rethinking-…

Looks like AI winter update needs another update https://blog.piekniewski.info/2018/10/29/ai-winter-update/

A couple days ago we were discussing on here how AI winter is coming and will likely arrive because of failure of driverless cars: https://blog.piekniewski.info/2018/05/28/ai-winter-is-well-on-its-way/ Are Waymo somehow much better at handling corner cases, identifying obstacles and their trajectories correctly and not having to disable emergency breaking to stop the car from randomly breaking at speed? Or are they just more careful/skilled with their marketing and publicity? Are they only planning to deploy the vehicles in specific, fairly controlled conditions, rather than claim general self-driving capabilities?

The process you are describing can be automated a bit. Imagine your feed reader automatically subscribes you to feeds of articles you liked (star/thumbs up/favorite). Let's say you start subscribed to an HN feed of items that got more than 100 upvotes: https://hnrss.org/newest?points=100&count=100 Then you find the OP link and like it in your feed reader: https://www.niemanlab.org/2022/12/this-is-the-year-of-the-rss-reader-r… Your reader automatically subscribes you to the source feed of this article: https://feeds.feedburner.com/NiemanJournalismLab You like more content => you get subscribed to more feeds automatically. Your feed reader can also take care of unsubscribing you from feeds that are posting too often or are not posting interesting to you content. If you don't like the content of a feed for awhile then your connection to that feed becomes weaker and the content from that feed is ranked less prominently. The content from feeds with a high signal-to-noise ratio (ie, how often you like content from that feed) is ranked higher for you. This is how my little project https://linklonk.com works. If you'd like to give it a try: "thumb up" a few items from the above HN feed: https://linklonk.com/feed?url=https:%2F%2Fhnrss.org%2Fnewest%3Fpoints%… and then check your recommendations on the home page. Through this mechanism I am currently subscribed to 201 RSS feeds and I know that I won't miss content from high "signal-to-noise ratio" feeds. Here is a sample…