Numenta Blog

Redwood City Numenta (http://www.numenta.com) is developing technology modeled on the neocortex. Get in on the ground floor of what, we think, is already groundbreaking machine intelligence. Senior devs with a passion for stellar, simple, usable design wanted. Experience with Python would be great but if you bring a deep skillset in a web stack of your choice then our team is always open to new ideas! Numenta is prepared to take on the world, and has the technology and experienced leadership to do it. Apply here (resume-eng at numenta dot com) or contact me through info in my profile for questions. P.S. Paid intern opportunities for the fall are open as well! Other keywords for page searchers: AI, Machine learning, front-end

I interned at Numenta [0] in 2012. Numenta is building an open-source machine intelligence product [1] based on the human brain. Specifically, the algorithms are based on the theory of "hierarchical temporal memory" (HTM) [2] as described by Numenta founder Jeff Hawkins [3] in his book On Intelligence [4]. The basic idea is that the neocortex has a generalized learning framework that acts on generalized input from all five senses. For example, one study enabled blind people to "see" a ball in front of them and grasp it with their hands, by encoding the image of the ball onto electrical impulses on taste buds, using a device in contact with the tongue. I can't find a link to the original study, but here's an article that describes one such device. [5] The conclusion drawn from this is that, even though blind people have lost their sight, their neocortex is able to understand inputs from any sensory source (in this case, tastebuds). The Numenta algorithms generalize this idea to feed time series data into a virtual "neocortex" that builds custom ML models for each data source and chooses the best one (a form of online learning that eliminates the need for data scientists to create custom models for each data source). Because the Numenta algorithms are optimized for time-series data, their areas of strength are prediction and anomaly detection. When I worked there (5 years ago, they've made a lot of progress since then), one of their main "case studies" was energy usage in…

Oh don't get me wrong, the things being done with ANNs are incredibly impressive. Far more so than anything I've seen with HTMs to date. I was just pointing out that Jeff is putting his money where his mouth is. Also, much of the resurgence in interest in AI came about 12 years ago in large part due to Jeff's thinking and writings (which was mostly consolidating existing ideas into a cohesive theory), as Andrew Ng and others have stated[1]. Jeff helped kick AI out of a plateau and avoid another AI winter. For interesting things that Numenta has done, just check their website[2]. Their most popular product is likely Grok[3], which predicts things like server loads. It's nothing that ANNs couldn't do, and ultimately isn't that impressive at face value, but the implementation is interesting and has far reaching implications (w/regards to things like energy efficiency and other areas). I think the future holds incredible things for ANNs and we'll continue to see great breakthroughs with them, but HTMs hold a lot of promise too. ANNs have been around quite a bit longer and have orders of magnitude more researchers working with them, so I'd argue it's premature to dismiss HTMs just because a handful of researchers in Redwood, CA haven't solved every AI problem within 12 years. It was only 15-20 years ago that ANNs were considered to have peaked and been surpassed by bayesian networks and support vector machines. How times have changed since then! [1]…

Redwood City, CA - Numenta Build the future of machine learning and big data. Learn more on our website: http://www.numenta.com/ Job Requirements: * BS or MS in Computer Science, or equivalent experience * Outstanding programming skills, preferably in Python * 3-5 years experience implementing high traffic web applications * 3-5 years experience with high performance systems in a commercial setting * Familiarity with entire web stack: Javascript, HTML, CSS, Memcached, application layer, and databases * Experience with SQL and no-SQL databases * Experience with unix/linux shell * Experience programming Amazon EC2 a plus * Experience with Agile / Scrum and/or test driven methodologies a plus * Able to demonstrate strong intellectual capabilities and significant individual contributions * Excellent written and verbal communication skills Qualified applicants should email a resume and cover letter to [email protected].

The implementation of these algorithms is relatively straightforward. The challenge is the state of the art in computer vision is not currently at a point where it is possible to reliably detect 10s-100s of object categories in real time on current systems. It is currently possible to build systems that get decent real-time performance on detecting a few categories concurrently, or offline systems that get around 60% accuracy across hundreds of well-defined categories. Thus, (1) faster general purpose hardware, (2) better algorithms, or (3) running the best algorithms in ASICs designed for CV is necessary. The top labs now typically use GPU clusters to train / run their algorithms, with the computationally expensive stages usually being feature extraction and/or classifier training. Google Predict (http://code.google.com/apis/predict/) offers a general machine learning API geared towards those who want to apply machine learning to their applications without subject-specific knowledge. I've not used it so I can't speak to its accuracy, but it is not geared towards computer vision and I imagine it would fail miserably at such tasks (since computer vision is highly dependent on domain-specific feature extraction techniques), and I imagine it performs well at NLP tasks. The primary limitation of such a system is that it acts as a black box - you throw data in and get answers out without any knowledge of the process behind it. This black-box model is limiting for three…

" With only 7M parameters, TRM obtains 45% test-accuracy on ARC-AGI- 1 and 8% on ARC-AGI-2, higher than most LLMs (e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the parameters" That is very impressive. Side note: Superficially reminds me of Hierarchical Temporal Memory from Jeff Hawkins "On Intelligence". Although this doesn't have the sparsity aspect, its hierarchical and temporal aspects are related. https://en.wikipedia.org/wiki/Hierarchical_temporal_memory https://www.numenta.com

If you’re looking for a neuroscience approach, check out Numenta https://www.numenta.com/

Highly recommend A a thousand Brains by Jeff Hawkins if you’re interested in this https://numenta.com/blog/2019/01/16/the-thousand-brains-theory-of-inte… He basically argues we all have 1000s of little, but interacting models of all sorts of things going on in our brain all at the same time. He calls them reference frames and it’s those that create intelligence. ‘Understanding’ would come naturally out of having those. Fascinating book which I’m probably explaining much less well that he does.

I was disappointed by A Thousand Brains. The last part of the book veered into ... I don't really know. On the other hand, I loved Jeff Hawkins' earlier book on this topic: On Intelligence [1]. It's easily one of the top 5 books that I've read and the first part of A Thousand Brains recaps much of that work, so still relevant today according to the author. [1] https://numenta.com/resources/on-intelligence/

I'm surprised that Numenta [1] did not get a mention in the comments. As a mostly lay person (with moderate exposure to Computational Neuroscience) I quite like their approach even though it seems their results have slowed down a bit of late. I still haven't finished parsing BAMI [2], but seems very interesting. [1] https://numenta.com/ [2] https://numenta.com/resources/biological-and-machine-intelligence/

For those interested, the book probably expands on the following paper from a few months ago: https://numenta.com/blog/2019/01/16/the-thousand-brains-theory-of-inte… (paper linked to from blog post) There's also a recent Lex Fridman podcast episode where he interviews Jeff Hawkins on this theme: https://lexfridman.com/jeff-hawkins/

If you liked this paper definitely check out Numenta’s research: https://numenta.com/

Their paper from few days ago is here: https://www.biorxiv.org/content/biorxiv/early/2018/10/13/442418.full.p… Rest of them and the code can be found in https://numenta.com/ The ideas represented and level of detail in their theory is such that the most likely response from everyone else is just 'Nice theory' and a shrug. Hawkins and Co. have been working on this for years. Numenta sends frequent newsletters and try to keep people interested, but there is little concrete there except some interesting ideas.

Numenta have been doing this for many years now, https://numenta.com/

Numenta [1] (co-founded by Jeff Hawkins, author of "On Intelligence") has been working on a model of ANN designed around how the brain operates. Hierarchical temporal memory (HTM) [2] is one aspect of their model. 1. http://numenta.com 2. https://en.wikipedia.org/wiki/Hierarchical_temporal_memory

Have you heard of Numenta? Their machine intelligence algorithms are great for anomaly detection in streaming data, including a product "Grok" for IT analytics (http://numenta.com/grok/). And all open source in NuPIC: https://github.com/numenta/nupic

What about Numenta? They claim to have a general, neocortex-inspired learning algorithm. http://numenta.com/

Numenta (http://www.numenta.com) is working on simulating how the brain (the neocortex in particular) recognizes patterns. It's an exciting recent field of research, and has little to do with neural networks, or other supposedly brain-inspired methods that don't actually bear much resemblance to neural processes. This TED talk by co-founder Jeff Hawkins (of Palm fame) gives an overview of what they are doing. http://www.ted.com/talks/jeff_hawkins_on_how_brain_science_will_change…

I've skimmed through that book and looked at the API over at Numenta: http://www.numenta.com/. They are focused on practical pattern recognition applications. Kurzweil has always been about strong AI and almost sci-fi type scenarios in his books. It's a way more ambitious goal, but he has predicted a lot of things correctly before. I'd check out some of his earlier books beforehand, just to get a better idea of his mindset.

The Blue Brain Project is attempting to recreate the brain in a super computer. http://en.wikipedia.org/wiki/Blue_brain And Jeff Hawkin's (the founder of Palm Computing) has already put forth a fairly comprehensive theory on how the brain works in his book On Intelligence, and his group Numenta http://www.numenta.com/ are currently working on a simplified computer implementation. Having read a fair amount into the topic, I'd say there's already enough information there to change how you might look at programming a certain task (especially if Numenta keeps pushing along). But it makes me think of humans more as complex memory machines rather than "compute-ers"

Agreed. If you can program in anything else, you can pick up enough Python to get started in a day (obviously longer to be really productive). I've read the O'Reilly Collective Intelligence book, and Python was a nice fit for expressing the ideas in there. That book would definitely get you started with practical uses of ML quickly. I'm not sure any of it is that cutting edge though in the academic sense. Also just finished On Intelligence by Jeff Hawkins last week. Good read. I plan to check out his company's ML ideas when I get some free time - they have free software for download too: http://www.numenta.com/for-developers/software.php

Jeff Hawkins (the author, creator of PalmPilot) has always been interested in how the brain works, and once he made his Palm money, he started a neuroscience institute. That research along with his own and what he read led him to his theory of how the brain works. It's extracted from lots of real, hard research. He then started a company (Numenta http://www.numenta.com/) to create a software implementation of that theory, which has a set of tools you can download and play with. They've had good results consistent with the theories in the book and had some impressive results. There's also a recording of a talk he gave at an O'Reilly conference: http://itc.conversationsnetwork.org/shows/detail3499.html So long story short, it is for real. When I read it, I had to stop every 20 minutes or so to catch my breath I found it so exciting and enlightening.

win the "go" competition. check out http://evolvedmachines.com those guys might like to help you out. also there is http://www.numenta.com/ that might be interested in some good algorithyms

The link the article uses to source the 60 GWh claim (1) appears to be broken, but all of the other sources I found give similar numbers, for example (2) which gives 50 GWh. This is specifically to train GPT-4, GPT-3 was estimated to have taken 1,287 MWh in (3), so the 50 GWh number seems reasonable. I couldn't find any great sources for the 200 plane flights number (and as you point out the article doesn't source this either), but I asked o1 to crunch the numbers (4) and it came up with a similar figure (50-300 flights depending on the size of the plane). I was curious if the numbers would be different if you considered emissions instead of directly converting jet fuel energy to watt hours, but the end result was basically the same. [1] https://www.numenta.com/blog/2023/08/10/ai-is-harming-our-planet-2023/ [2] https://www.ri.se/en/news/blog/generative-ai-does-not-run-on-thin-air [3] https://knowledge.wharton.upenn.edu/article/the-hidden-cost-of-ai-ener… [4] https://chatgpt.com/share/678b6178-d0e4-800d-a12b-c319e324d2b6

For people wanting to look into HTM (Hierarchical Temporal Memory), do check out Numenta's main website [1], in particular the papers [2] and videos [3] sections. Otherwise, HTM inventor Jeff Hawkins' book "On Intelligence" is one of the top 3 or so most fascinating books I've ever read. It doesn't cover HTM though, just how the brain works at a conceptual level, but in a way I haven't seen anyone else explain. Jeff clearly has an ability to see the forest through the trees in a way that is not too commonly found. This is one of the reasons I think HTM might be on to something, although it of course has to prove itself in real life too. But we should remember for how long classic Neural Networks was NOT overly successful, and almost dismissed by a lot of people (including my university teacher who was rather skeptical about them, when I took an ML course on like 12 years ago and personally believed a lot in them). We had to "wait" for years and years until enough people were eventually throwing enough work on finding out how to make them really shine. [1] https://numenta.org/ [2] https://numenta.com/neuroscience-research/research-publications/papers… [3] https://www.youtube.com/user/OfficialNumenta [4] https://www.amazon.com/Intelligence-Understanding-Creation-Intelligent… Edit: Fixed book link.

If they had a solution that was better than what was out there, and they are publishing their research, then it follows we should be able to find their novelty in their publications, right? So, I took a look at their publication track record on this topic [0], and there are a few red flags that stood out to me. First, they're publishing at a pretty astounding rate, with new submissions roughly every 1-3 months for the past 3 years, with the same authors. I'm skeptical that a small (<5?) research team would be able to produce novel contributions at such a rate, which leads me to believe they're probably remixing previous works into incremental publications. Second, all of their publications are in neuroscience journals, which may not be up to date with the latest research going on with HTM in (for example) computer science. NIPS does have a computational neuroscience track, after all - I would have expected at least some interdisciplinary leakage if the novelty was very high. Finally, the journals that they're publishing in are relatively low-ranking, which also seems to indicate that what they're doing is not necessary very novel (as you said, perhaps just a re-hash of existing HTM research). Also, this article was co-written with a marketing exec, so I think it's fair to bring a certain level of skepticism. [0]: https://numenta.com/resources/papers/

Dr. Hassabis is awesome, but that video and the language is misleading to a layman. He is distinguishing between expert driven systems that rely on heuristics/feature engineering and between systems that learn from raw input and derive their own optimal set of features (unsupervised learning). This is a far cry from AGI. I think Dr. Hassabis rather in a tongue and cheek manner played with the terminology in the video. Deep learning and all the modern AI stuff you hear about is within the realm of "narrow AI", or more formally, applied AI. In his video, he uses "narrow AI" to define systems that rely on expert based heuristics and feature engineering, and general purpose AI to be what they are currently doing with reinforcement learning. Whilst it's wonderful that their advancement in reinforcement learning has been applied to various different problems successfully, it shouldn't be confused with AGI. AGI is on a totally different playing field. I don't think we are substantially closer to AGI than we were 50 years ago, and I would be very interested in anyone arguing the opposite. I think at this point the only company trying to seriously tackle AGI is: https://numenta.com/

Numenta (http://numenta.com/ http://numenta.org/)

So I figured it out. Basically, they take the idea of AGI seriously, and actually consider and talk about the repercussions, and therefore you dismiss them and their ideas as fringe and not worth investigating. I know that, because if you had investigated at all, you would see that all of those projects had really interesting results and these people are not being vague and hand-waving. Not all of those projects I listed identify themselves as AGI. However, they should go in the same group. And anyway, all of those projects have demonstrated progress. If you looked into them at all then you would see that. Ben Goertzel is using some aspects of his AGI research in mainstream (narrow) AI projects. OpenCog has released a number of solid demonstrations of current features. And Goertzel isn't hand-waving or bullshitting in his numerous books and scientific papers, for example Probabilistic Logic Networks: A Comprehensive Framework for Uncertain Inference (336 pages). Hawkins has demonstrated very interesting progress with his software and has a commercial application https://www.numenta.com/grok_info.html Voss is using his system at Adaptive AI as a commercial enterprise. Qualcomm is funding Brain Corporation (Izhikevich et al) so obviously they are taking it seriously. A bakery in Tokyo has tested Brain Corporation's machine vision technology to power a semi-automated cashier system http://www.diginfo.tv/v/12-0145-r-en.php

The specific news is that they are open sourcing their Thousand Brains Project. [0] Technical document: https://www.numenta.com/wp-content/uploads/2024/06/Short_TBP_Overview.… [0]: https://x.com/Numenta/status/1798406708987142171?t=PWgsdVsZlRpkP_kQdR8…