Chris Olah

Worth noting that Chris Olah (who wrote this comment) has led much of the interesting work in making feature visualisations useful. If you look for "Visualizing Neural Networks" on his page https://colah.github.io/ you'll find lots of other interesting links in this area.

If you’re interested in machine learning I can recommend Christopher Olah’s blog: https://colah.github.io/

I'm also starting to follow people and communities that work with deep learning in new ways. Here are some of my favorites: [1] http://colah.github.io/ [2] https://iamtrask.github.io/ [3] https://distill.pub [4] https://experiments.withgoogle.com/ai

this focuses on NLP http://www.wildml.com this is also great http://colah.github.io

If we succeed in training an accurate enough machine learning algorithm, I think the primary purpose will be to screen for asymptomatic atrial fibrillation, and then use a Holter or Zio Patch for a final diagnosis. The main benefit is that you can prevent cryptogenic strokes, which are often caused by undiagnosed atrial fibrillation: http://www.nejm.org/doi/full/10.1056/NEJMoa1311376 For machine learning, I think you're on exactly the right path. The Udacity TensorFlow course taught by the Google Brain team is good; if you can complete that, you'll be as good as anyone at applying neural networks. The Stanford courses (http://cs231n.stanford.edu/, http://cs224d.stanford.edu/) are likewise great, as is Chris Olah's blog for intuition (http://colah.github.io/). And down the road, if you're ever looking for an internship, definitely let us know! [email protected].

Don't miss C Olah's blog where Figure.1 was copied from http://colah.github.io/ Very very visually insightful on the nature of Neural Nets, Convnets, Deep Nets...

Yes for sure. I am very math-averse (almost failed calculus 1, skated by pre-calc in high school... not my intelligence area) and though I don't have a full time job doing something in AI, I do feel like I can build models that solve real problems (and am doing so in an internship right now). Probably start with Andrew Ng's Machine Learning course. It has a significant amount of Math in it-- try to understand it, but seriously do not worry about it. Just get the high level concepts, try to get some intuition on machine learning ideas and techniques. You don't need to do the assignments or work too hard on the course (but obviously it's helpful if you do). Then read these. Don't worry if you're still confused at first. It's fine. Just go through 'em kinda slowly and try to see what's going on. https://iamtrask.github.io/2015/07/12/basic-python-network/ http://karpathy.github.io/neuralnets/ By now if you're still into it, I highly recommend Chris Olah's blog: http://colah.github.io/ It has some pretty complicated ideas in there, but the articles are illustrated and explained very well so you can get more of a feel for Neural Networks while also getting very excited about them. Then it's probably time to start really building things. I would use Keras (https://keras.io/) at first, because it's very easy to get cool results without understanding everything under the hood. Do a tutorial or two, it's pretty intuitive and if you've done everything above you should understand…

If you are looking to learn about LSTMs and RNNs, I can't recommend this post enough: https://colah.github.io/posts/2015-08-Understanding-LSTMs/ Another great one (just on RNNs) is: http://karpathy.github.io/2015/05/21/rnn-effectiveness/

Each time I teach neural nets to an engineer, there's only a 50% chance they can write down the chain rule. Colah's blog on backprop used to be my favorite resource to leave them with (https://colah.github.io/posts/2015-08-Backprop). The explanation of the calculus in this tool is equally fantastic. And the art is very cute. There are many ways to skin a cat, of course, but this is as good a tutorial as I've seen for getting you through backprop as fast as possible.

For interesting reads in Deep learning I follow these: - https://distill.pub/ - http://colah.github.io/ - https://lexfridman.com/ai/ - http://iamtrask.github.io/ And some interesting reads on Uncertainty in Deep Learning http://mlg.eng.cam.ac.uk/yarin/blog_2248.html

Love this article. It makes statistics enjoyable and accessible. Most of Olah's old stuff is also really good, especially the one on manifolds and neural networks [1] [1] https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/

Here is a great primer on entropy that you might find helpful - http://colah.github.io/posts/2015-09-Visual-Information/

For 1, take a look at this guy's blog: http://colah.github.io

Thanks for a great plugin. I've been trying to get a handle on what convolutions really are. Which ever description I read is usually so abstract as to be meaningless. But your plugin is a beautiful way to play with them an understand them in a fundamental way. N.B. I came across it because of Chris Olah's blog post [0]. [0]: http://colah.github.io/posts/2014-07-Understanding-Convolutions/

The following is a concise and good explanation of necessary knowledge of information theory: http://colah.github.io/posts/2015-09-Visual-Information/

I'm glad to see the shout out to http://colah.github.io/ (even if the linked post is described as "monster stories")

http://www.datatau.com/ http://spectrum.ieee.org/blog/automaton http://highscalability.com/ A selection of subreddits, among which: r/apachespark/, r/java/, r/programming/, r/InternetIsBeautiful/, r/dataisbeautiful/, r/MachineLearning/ I'd like to follow a bunch of people on twitter, on the subjects they are experts in; but find the signal to noise ratio is not high enough. Also, I stalk a few people on various forums because I really like their way of thinking or expertise. If you say something clever, I'm going to go through your post history. On top of that, some youtube channels, and about 400 blogs of individuals. Best ones: https://www.youtube.com/user/GoogleTechTalks, http://colah.github.io/, https://amplab.cs.berkeley.edu/blog/, http://acko.net/, http://karpathy.github.io/.

Chris Olah also has a great introduction: http://colah.github.io/posts/2015-08-Understanding-LSTMs/

One of my favourite articles recently was this one by Christopher Olah http://colah.github.io/posts/2015-09-NN-Types-FP/. As a haskeller it really helped put things in terms I can understand. I think there's also value for those who aren't functional programmers because it teases out the fundamental abstractions of the building blocks to deep learning very well.

Do you recommend any good primers on topology? I thought this (https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/) was an interesting article and I see what looks like some great papers and videos available at http://www.ayasdi.com/approach/data-scientist/, but I don't know the difference between homotopy and homology (yet) :) . What kinds of infrastructure/tech do you think will have the most utility for topological data analysis in the near future? E.g., GPUs, Apache Spark, FPGAs, etc. Any thoughts on an Ayasdi public offering? I'd like to consider investing but I don't have millions of dollars (yet) :) . Thanks for your time.

Anyone reading this who hasn't already should do themselves a favour and read the other articles on colah's blog. Beautifully presented demonstrations of ML algorithms, a number running live in your browser http://colah.github.io/

You might be interested by colah's (Christopher Olah's) blog: https://colah.github.io/ Two articles in particular are good introductions to looking at neural networks in terms of higher dimensional data lying on lower-dimesnional manifolds: Neural Networks, Manifolds, and Topology: https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/ Visualizing MNIST: An Exploration of Dimensionality Reduction: https://colah.github.io/posts/2014-10-Visualizing-MNIST/

Everything on this guy's blog is fantastic. I love the post on neural network manifolds (http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/).

I haven’t done much with neural networks, but Christopher Olah wrote a fantastic post on visualizing low-dimensional ones: http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/

Hi! I lead interpretability research at Anthropic. I also used to do a lot of basic ML pedagogy (https://colah.github.io/). I think this post and its children have some important questions about modern deep learning and how it relates to our present research, and wanted to take the opportunity to try and clarify a few things. When people talk about models "just predicting the next word", this is a popularization of the fact that modern LLMs are "autoregressive" models. This actually has two components: an architectural component (the model generates words one at a time), and a loss component (it maximizes probability). As the parent says, modern LLMs are finetuned with a different loss function after pretraining. This means that in some strict sense they're no longer autoregressive models – but they do still generate text one word at a time. I think this really is the heart of the "just predicting the next word" critique. This brings us to a debate which goes back many, many years: what does it mean to predict the next word? Many researchers, including myself, have believed that if you want to predict the next word really well, you need to do a lot more. (And with this paper, we're able to see this mechanistically!) Here's an example, which we didn't put in the paper: How does Claude answer "What do you call someone who studies the stars?" with "An astronomer"? In order to predict "An" instead of “A”, you need to know that you're going to say something that starts with a…

I wish I knew what it was like to have the math background, and don't at all mean to suggest one should not bother, but as a person on a team of people who have made a few production NN models over the past year: it is completely possible to conceptually and practically grasp iterated reverse differentiation with convolutions well enough to use deep learning to do novel work, without having the deep background math knowledge. For example, I barely know what a Support Vector Machine is, nor could I do a linear regression without a lot of hand holding. But I can design a passable tensorflow model and improve it. It would definitely be very hard to do any meaningful research from this position, but I know enough to be useful (er, or dangerous) and can read papers and code to keep up with recent advancements, and try things out like different convolution designs, layer designs, gates, functions, feedback, etc. (On the topic of reading: Karpathy[1] and Colah[2]'s posts have a wealth of introductory conceptual information and images in them, and helped our team discussions a lot while we've learned. 1 - https://karpathy.github.io/ 2 - https://colah.github.io/)

nope: http://colah.github.io/

No, you don't need the degree. Denny Britz, He drops school (his phd) to go into industries. Christopher Olah: Same story, at least he doesn’t communicate about his education (http://colah.github.io/cv.pdf). Both are well-known for their expertise, and followed by a large amount of people in the deep learning community. You can def become an great machine learning engineer who used deep learning technique, without going to any university. --- Ian Goodfellow: "One of my biggest heroes is Geoffrey Hinton. His PhD was in experimental psychology ( . Biographical Sketch ). In mechanical engineering, I think you learn quite a lot of the mathematical tools that are used in machine learning, so you won’t be starting from scratch. In general, we often find that revolutions in one field happen when someone from another field arrives with fresh ideas that developed elsewhere." (https://www.quora.com/Would-you-encourage-people-from-another-field-me… How can someone with almost no technical knowledge learn ML? Ian Goodfellow::"It’s important to master some of the basics, like linear algebra, probability, and python programming. But you don’t need to know all of linear algebra (I don’t think I’ve ever used a QR decomposition outside the classroom), all of probability (we don’t usually do much involving combinatorics / re-ordering of sequences in deep learning), or all of python programming (a lot of the more obscure language features are…

Although probably not sufficient for AGI, network architecture is essentially guaranteed to be important, because of both ample empirical evidence of the importance of architectures and ample reason, from facts about numerics, to believe that it is important. In the first category (empirical evidence), - The discrete leap from non-LSTM RNN to LSTM network performance on NLP was essentially due to a "better factoring of the problem": breaking out the primitive operations that equate to an RNN having "memory" had a substantial effect on how well it "remembered." - The leap in NMT from LSTM seq2seq to attention-based methods (the Transformer by Google) is another example. Long-distance correlations made yet another leap because they are simply modeled more directly by the architecture than in the LSTM. - The relation network by DeepMind is another excellent example of a drop-in, "pure" architectural intuition-motivated replacement that increased accuracy from the 66% range to the 90% range on various tasks. Again, this was through directly modeling and weight-tying relation vectors through the architecture of the network. - The capsule network for image recognition is yet another example. By shifting the focus of the architecture from arbitrarily guaranteeing only positional invariance to guaranteeing other sorts, the network was able to do much better at overlapping MNIST. Again, a better factoring of the problem. These developments all illustrate that picking the…