Machined Learnings

Another review which covers some different papers: http://www.machinedlearnings.com/2015/05/iclr-2015-review.html

Vowpal Wabbit (one of the fastest learning implementations, from John Langford, also of Yahoo!) now has an "online" LDA implementation (Hoffman et al 2010: http://www.cs.princeton.edu/~blei/papers/HoffmanBleiBach2010b.pdf). It's probably the fastest single-core LDA implementation. http://hunch.net/?p=1594 http://www.machinedlearnings.com/2010/12/lightning-fast-lda.html

Related -- there's a 2011 post from Paul Minero with fast approximations for logarithm, exponential, power, inverse root. http://www.machinedlearnings.com/2011/06/fast-approximate-logarithm-ex… Minero's faster approximate log2, < 1.4% relative error for x in [1/100, 10]. Here's the simple non-sse version: static inline float fasterlog2 (float x) { union { float f; uint32_t i; } vx = { x }; float y = vx.i; y *= 1.1920928955078125e-7f; return y - 126.94269504f; } This fastapprox library also includes fast approximations of some other functions that show up in statistical / probabilistic calculations -- gamma, digamma, lambert w function. It is BSD licensed, originally lived in google code, copies of the library live on in github, e.g. https://github.com/etheory/fastapprox It's also interesting to read through libm. E.g. compare Sun's ~1993 atan2 & atan: https://github.com/JuliaMath/openlibm/blob/master/src/e_atan2.c https://github.com/JuliaMath/openlibm/blob/master/src/s_atan.c

I actually think Machine Learning is relatively easy. There are a lot of resources, the community is very open, state-of-the-art tools are available, and all it needs to get incrementally better is trying out more stuff on different data sets. I worked in SEO before, which had far more elements of "black magic". Perhaps SEO helps with the transition to ML, because you are basically reverse engineering a model (Google's search engine) / crafting input to get a higher ranked output. It's feature engineering, experimentation, and debugging all-in-one. And front-end development of the old days... debugging old javascript or IE6 render bugs makes ML debugging pale in comparison. You had to make a broken model work, without being able to repair it. As for the long debugging cycles in ML. John Langford coined "sub-linear debugging": Output enough intermediate information to quickly know if you introduced a major bug or hit upon a significant improvement [1]. Machine learning competitions are not so much won by skill, but by the teams iterating faster and more efficiently: Those who try more (failed) experiments hit upon more successful experiments. No Neural Net researcher should let all nets finish training, before drawing conclusions/estimates on learning process. Sure, the ML field is relatively new, and computer programming has a longer history of proper debugging and testing. It is difficult to do monitoring on feedback-looped models running in production, yet no more…