Alex Smola

The senior author, Alex Smola, is the director of AWS (and is a big deal: https://alex.smola.org/), which helps explain the choice of Apache MXNet. That said, the authors write that "even if you use other deep learning frameworks in your research or work, we hope that these codes will help you better understand deep learning algorithms."

I watched some of the videos they are quite good. The data streams lectures are definitely worth a watch: http://alex.smola.org/teaching/berkeley2012/streams.html.

http://www.cs.cmu.edu/~epxing/Class/10701/ http://www.cs.cmu.edu/~aarti/Class/10701_Spring14/ http://www.cs.cmu.edu/~./10701/ http://alex.smola.org/teaching/cmu2013-10-701/ All of the above have answer keys for the homework assignments.

Yes, on their site they post Torch CuDNN v4 as faster than Leaf [0]. Seems exciting for an early release. Can it get much faster than something like Torch? I would think if CuDNN is doing most of the computation time it would be hard to see big improvements. Perhaps go the route of Neon and tune your GPGPU code like crazy [1, 2], or MXNet and think about distributed computing performance [3]. [0] http://autumnai.com/deep-learning-benchmarks [1] https://github.com/soumith/convnet-benchmarks [2] https://github.com/NervanaSystems/neon [3] http://alex.smola.org/talks/NIPS15.pdf

Here's the complete course: http://alex.smola.org/teaching/cmu2013-10-701/

You may want to look at the self-evaluation, a qualification exam for understanding the material: http://alex.smola.org/teaching/cmu2013-10-701x/slides/Intro_ML_Self_Ev… It would have been a piece of cake in college, but I have not used that math in a long time :-/

Maybe my undergraduate institution failed mebut I don't feel that having taken linear algebra, multivariable calc, applied stats, and graduate probability over the summer that the PDF below contains readily accessible information to me. Does it require graduate linear algebra? Graduate multi-v? We're my courses not rigorous enough? Am just not smart enough? I graduated summa cum laude, so maybe I should be okay. All I am saying that is that there seems to be a huge leap between having taken the prerequisites everyone says you should take and then seeing Cramer-Rao bounds on page 70. http://alex.smola.org/drafts/thebook.pdf

was mostly thinking of courses that had lectures/vids up, but that also reminds me of this berkeley course i came across on HN a couple of weeks ago: http://alex.smola.org/teaching/berkeley2012/

Before reading the Blog's summary of the paper, I thought hey they're going to analyze the distributed algorithms[1][2] exclusively and give conclusions on how heterogenous hardware usgage of GPU/CPU works (best) when spread over the network/cloud. My guess was that MXNet would win, because Amazon [3][4] and benchmarks favored it[5]. Unfortunately surveys like this are too complex and out of date, when released, due to the fast progress in our field. >> @agibsonccc What exactly would YOU, your company and your team like to see reviewed and what would you make different? A standard benchmark, like the Open-Source, Automated Benchmark provided by the Phoronix-Test-Suite[6] or the benchmarksgame[7] are what we need. Everybody should be able to contribute easily, or even anonymously like with Steam's hardware survey[8]. The main problem with distributing operations with Caffe in example is the need to learn another system for a different programing flavor. -- References [1] https://en.wikipedia.org/wiki/Distributed_algorithm [2] https://en.wikipedia.org/wiki/Category:Distributed_algorithms [3] http://www.infoworld.com/article/3144025/cloud-computing/why-amazon-pi… [4] http://www.allthingsdistributed.com/2016/11/mxnet-default-framework-de… [5] See Page 52 - http://alex.smola.org/talks/NIPS15.pdf [6] https://www.phoronix-test-suite.com/ [7] http://benchmarksgame.alioth.debian.org/ [8]…

There's quite a few good free ones of varying perspectives http://metaoptimize.com/qa/questions/186/good-freely-available-textboo… Esp: http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.HomeP… http://www.inference.phy.cam.ac.uk/itprnn/book.html http://alex.smola.org/drafts/thebook.pdf ___________ and MLPP errata: http://www.cs.ubc.ca/~murphyk/MLbook/errata.html

Berkley Course: http://alex.smola.org/teaching/berkeley2012/statistics.html Scikit-Learn Docs: http://scikit-learn.org/dev/ Google Research: http://research.google.com/pubs/papers.html Caltech Course: http://work.caltech.edu/telecourse.html Basics in R: http://www.statmethods.net/advstats/glm.html R packages for data science: http://blog.yhathq.com/posts/10-R-packages-I-wish-I-knew-about-earlier…