SVDS Blog

Notebooks can be reproducible, they just aren’t automatically so. It requires a little bit of effort and discipline, if reproducibility is a goal. https://www.svds.com/jupyter-notebook-best-practices-for-data-science/ is an excellent starting point. Personally, I use notebooks to keep a record of large computational pipelines. The key is to cache all results to disk. This allows for an iterative process where I modify the notebook, kill the kernel, and rerun everything. Only new calculations will be executed, everything previously calculated will simply be loaded from disk. In the end, I have a reproducible record of the entire project (and rerunning the notebook is fast) This kind of make-like functionality is implemented through the doit Python package (http://pydoit.org). An example workflow for this is http://clusterjob.readthedocs.io/en/latest/pydoit_pipeline.html

>> Rebalancing an imbalanced dataset is common in industry and academicia. You use that when you focus on accuracy, to make claims like: We were 54% accurate on classifying sexuality of females easily interpretable, without needing a distribution-balanced benchmark (you simply know it is a coin flip). You quoted The Strength of Weak Learnability and I figured you must have at least a passing acquaintance with computatinal learning theory. In computational learning theory (such as it is) it's a foundational assumption that the distribution from which training examples are drawn is the same as the true distribution of the data, otherwise there cannot be any guarantees that a learned approximation is a good approximation of the true distribution. The following is a good article on machine learning with unbalanced classes: http://www.svds.com/learning-imbalanced-classes/ I recommend it as a starting point. >> This is a very strong claim to make, in the absence of legit discrediting studies that failed to replicate any predictability, and requires more than guessing the authors rebalancing act was "clearly" to improve the accuracy (with 7% negative class, you could get 93% accuracy by always predicting positive class, so if they wanted to inflate the accuracy, they shouldn't have rebalanced). The gay class was the positive class and the straight class negative, in this case. If you did what you say and identified everyone as straight, you'd get a very high number of false…

I found this to be informative - https://svds.com/getting-started-deep-learning/

Here's an in depth answer: https://www.svds.com/getting-started-deep-learning/

This is very good - I've never seen a RNN-for-speech-in-Tensorflow model before. Note, though that the real problem here is the lack of training data. In a recent podcast I heard that the Baidu speech recognition team uses "small models" of 10,000 hours of speech. I forget how big the production quality models were, but it was at least 5 times that. This model uses ~1500 hours[1]. It's very impressive it does as well as it does just using that. [1] https://svds.com/tensorflow-rnn-tutorial/

This is odd- the original article [0] was posted about a month ago, and this seems to be a repost with some ads thrown in. Hopefully mrubashkin submitted the article to Insight Data Science himself. [0]: https://svds.com/tensorflow-image-recognition-raspberry-pi/

There was a recent post on HN about getting a tensorflow model running on a raspberry pi to detect trains. The blog post describing the process was quite well done. I bet it would apply quite well to detecting deer. https://news.ycombinator.com/item?id=13603225 http://svds.com/tensorflow-image-recognition-raspberry-pi/

My colleagues and myself have a blog post discussing how to do streaming video analysis on the Raspberry Pi: https://svds.com/streaming-video-analysis-python/ On the Pi3, our application processes 320X240 images at 10 FPS without any problems. Let me know if you have any questions!

We've been working on the solution to this issue since our office is overlooking the tracks. You can read more here: http://svds.com/post/listening-caltrain and here: http://svds.com/post/railroad-modeling-hadoop-scale-hadoop-summit-2014…

Ha! A friend sent me this comment when he recognized this project. Unless there happens to be another firm who did the exact same thing we did, I was a part of this project (see this blog post https://www.svds.com/introduction-to-trainspotting/). You misunderstood the point of the presentation. The company was a consulting firm that specialized in data science and engineering. Our clients wanted to kick the tires and see what our technical chops were before hiring us but they didn't want to let us use their proprietary and confidential data for our own tech demos. We didn't want to just use the same open source datasets everyone else did, so we got to thinking about novel datasets we could create that might have applications for industries we sold our services to. From this, the Trainspotting project was born. Many of us commuted via the Caltrain, which was right next to our office, and we were frequently frustrated with the unreliability (this was in ~2016 or so when car and pedestrian strikes were happening seemingly every week), so we made an app that tried to provide more accurate scheduling. We used the official API for station:train arrival times, but we found that it was unreliable, so we wanted some ground truth data on whether a train was passing. Since our office was right next to the Castro MTV station, I had the idea to use a microphone (attached to a raspberry pi) to just listen for when the train went by. In addition to ground-truth data for validating…