Radim Řehůřek

(tutorial author here) Good answer, and I can only recommend Ian's book! I cut the marketing speak down to minimum in my articles and tutorials, but if you're interested in cutting edge machine learning & no-nonsense data mining, get in touch! I run a world class consulting company, http://radimrehurek.com.

The golden standard for word2vec and topic modeling in general is gensim: https://radimrehurek.com/gensim/models/word2vec.html I've been using gensim for a few years for topic modeling. It works great with pyLDAvis for visualizations. How does your implementation differ from the one by Radim?

Neat! One production ready python library I love to use in this space is https://radimrehurek.com/gensim/ It is quite mature and can handle a good amount of data well! It offers topic modeling too and can b helpful to find similar documents also.

We started out with word2vec, but discovered it was easier to work with GloVe for our purpose. There are ways to convert glove vectors to word2vec format such as: https://radimrehurek.com/gensim/scripts/glove2word2vec.html

I know NLTK isn't used for learning by itself, but it's such an essential player in the realm of NLP+Python that I think it should definitely be included (and that the list would be incomplete without it). The best link is probably http://www.nltk.org/book/ which is an excellent place to get started. You should probably also include the gensim package https://radimrehurek.com/gensim/, since it has the most popular python word2vec implementation. One of the links I clicked on mentions it and walks you through using it, but I think it would make sense to point people to it directly in case they don't have time for a tutorial.

Great podcast. The world owes a big THANK YOU to Tomáš Mikolov, one of the creators of Word2Vec[0] and fastText[1], and also to Radim Řehůřek, the interviewer, who is the creator of gensim[1]. The number of software developers and researchers in industry and academia who rely on the work of these two individuals is large and growing every day. [0] https://code.google.com/p/word2vec/ [1] https://github.com/facebookresearch/fastText [2] https://radimrehurek.com/gensim/

From my favorites list, very easy introduction tutorial about ML, good as starting point: http://radimrehurek.com/data_science_python/ - Practical Data Science with spam detection example (Machine Learning, NLP, sklearn, Python).

Try some yourself. http://radimrehurek.com/2014/02/word2vec-tutorial/ Scroll down and you have some text boxes you can try some yourself. Here's some I tried (filtering out repeats and plural forms of input words. Those artifacts seem to happen a lot and would be easy to ignore) cat:dog::bread:butter eh? I guess. sword:shield::attack:protect okay that works. up:down::left:leaving Eh, not great. I guess if you think they're analogous in terms of tense it kind of works. Disappointing, word2vec. (to be fair, "right" was third highest) drive:car::dick:? Whatever it was, it made me giggle immaturely.

Check out gensim if you want to do topic modeling or similarity comparisons in Python. http://radimrehurek.com/gensim/ It has good implementations of various algorithms, some of which support streaming or dirstribution, and it allows loading and dumping data in various formats. I've used it for building content based recommender using tf-idf, lsi and similarity index. After the index is built, queries to it are really fast. It can handle quite large corpuses with little memory.

As with almost all things machine learning, python libraries are where you will find most of the action. I can recommend https://spacy.io for a low fuss solution to get you up and running quickly. Oversimplifying quite a bit, if spacy focuses on syntax, then gensim focuses on semantics (https://en.wikipedia.org/wiki/Distributional_semantics). Gensim has an active community and is well documented. https://radimrehurek.com/gensim/ If you have the data, can spend a few days experimenting and if you want something that can be orders of magnitude faster than deep learning to train, there's vowpal wabbit. Prediction speed is blazing. Results can be nearly state of the art but with a cost that's a great deal less. It's C++ but with bindings for many languages. It's very poorly documented. https://github.com/JohnLangford/vowpal_wabbit/wiki/Learning%20to%20Sea… I've never taken a gander at Facebook's fastText.

This tutorial is excellent, and Richard Socher has been doing some really fascinating (though complicated) work with deep learning for NLP. I also have a few other good references for people who want to investigate this area. One of the most insightful comments I have seen about deep learning for NLP is a reddit r/machinelearning comment seen in this post [1]. As the poster in that thread mentioned, much of the theory about using neural networks for NLP tasks was laid out pretty clearly in a paper by Yoshua Bengio from 2003 [2]. At a high level, the basic learner is very similar to many of the other deep architectures for images and audio. There is also a pretty great blog post [3] by Radim Rehurek (author of gensim) about how he implemented word2vec (a "deep learning"-style model for text) in Python, while also getting a performance improvement over the C version! [1] http://www.reddit.com/r/MachineLearning/comments/2doi7e/what_should_i_… [2] http://jmlr.org/papers/v3/bengio03a.html [3] http://radimrehurek.com/2014/02/word2vec-tutorial/

This is interesting. I went off and searched for paragraph vector code and indeed find doc2vec stuff, including tutorials referring to the paper such as https://radimrehurek.com/gensim/auto_examples/howtos/run_doc2vec_imdb.… It’s not obvious that the results aren’t reproducible (and I realise code is not the same as published results), but I wonder if you could steer us more specifically.

I'm curious about Gensim Doc2Vec Model. I used it 3 years ago and got decent results in vectorizing articles and then finding articles that were similar based on input text (half-written article for example). What is new here? https://radimrehurek.com/gensim/auto_examples/tutorials/run_doc2vec_le…

gensim actually allows you to use both annoy and nmslib with gensim generated vectors as part of the api. https://radimrehurek.com/gensim/similarities/nmslib.html https://radimrehurek.com/gensim/similarities/annoy.html

Yes anyone can post links/facts assuming they have an account with at least one credit. When you submit a story you can specify an initial seed source which will be summarized using the gensim summarization module and used to prepopulate the facts section. If gensim isn't able to generate a summary then we just use the headline of the linked article as the first fact. After that, facts can be added manually by adding some text (max 160 chars) with an accompanied source URL. Gensim: https://radimrehurek.com/gensim/summarization/summariser.html

Note also: https://radimrehurek.com/gensim/models/keyedvectors.html

Just want to say thank you for gensim [1]. [1]: https://radimrehurek.com/gensim/

Reading the guide for fasttext I am not sure how a non NLP researcher would use it. "This library can also be used to train supervised text classifiers" - https://github.com/facebookresearch/fastText#text-classification Looks like you can use it for gensim. https://radimrehurek.com/gensim/models/fasttext.html

Word2Vec and bag-of-words/tf-idf are somewhat obsolete in 2018 for modeling. For classification tasks, fasttext (https://github.com/facebookresearch/fastText) performs better and faster. Fasttext is also available in the popular NLP Python library gensim, with a good demo notebook: https://radimrehurek.com/gensim/models/fasttext.html And of course, if you have a GPU, recurrent neural networks (or other deep learning architectures) are the endgame for the remaining 10% of problems (a good example is SpaCy's DL implementation: https://spacy.io/). Or use those libraries to incorporate fasttext for text encoding, which has worked well in my use cases.

https://radimrehurek.com/gensim/ makes topic modeling really easy in python

If it can be interfered from words alone, try Latent Dirichlet Allocation (e.g. with http://radimrehurek.com/gensim/) to generate tags. Some sources: * http://blog.echen.me/2011/08/22/introduction-to-latent-dirichlet-alloc… * http://alexperrier.github.io/jekyll/update/2015/09/04/topic-modeling-o… * http://engineering.flipboard.com/2017/02/storyclustering Alternatively, if you know tags, just want to see which are similar to each other, methods like word2vec should help, vide: * http://p.migdal.pl/2017/01/06/king-man-woman-queen-why.html

Some fairly widely-used techniques include LSI, LDA, and word2vec or doc2vec. There a lot of different techniques out there! I'm one of the creators of Tagger News, and we used LDA with python's Gensim package. Here's a good tutorial: https://radimrehurek.com/gensim/tut2.html

I'm not sure if there is an example somewhere in the TF docs, but initializing a variable is pretty easy. All you need to do is: session.run(W.assign(numpy_word2_vec_matrix)). W would the embedding matrix created in first layer of the code. [1] Of course you'd first need to load word2vec and filter its vocabulary to match your own vocabulary. That's most of the code and not specific to TensorFlow. You could use gensim [2] for that. [1] https://www.tensorflow.org/versions/master/api_docs/python/state_ops.h… [2] https://radimrehurek.com/gensim/index.html

>a project that requires me to scrape a large amount of text and then use NLP to determine things like sentiment analysis, LSM compatibility, and other linguistic metrics for different subsections of that content. I ran into a similar project and found this helpful working with the unstructured data: https://textblob.readthedocs.org/en/dev/ https://radimrehurek.com/gensim/

Nice read. I did something sort of similar with the same dataset about a year ago. I compared LDA (Latent Dirichlet Allocation) to TF-IDF as tools to find similar beers based on their review text. Lots of intuitive and funny topics discovered. I suggest you play with LDA, it seemed to work really well at generating topics. There is also a lot of fascinating, very readable research using it. Check out SNAPs work on the same dataset [1] and some of the Yelp Dataset challenge winners [2]. If you end up interested in doing so, Gensim [3] was pleasant enough to work with. [1] http://snap.stanford.edu/data/web-BeerAdvocate.html [2] http://www.yelp.com/dataset_challenge [3] https://radimrehurek.com/gensim/wiki.html#latent-dirichlet-allocation

It sounds like you basically want a form of document clustering. Seldon has integrated the Semantic Vectors https://code.google.com/p/semanticvectors/ project which can be used to do this or depending on your technical level to look directly at Semantic Vectors or toolkits such as gensim https://radimrehurek.com/gensim/

Don't forget that the higher level functionality (e.g. the scikit-learn routines Radim uses) are typically wrappers for underlying C/Fortran routines and they're the real bottleneck. The relatively few lines of VM'd Python are 'slow' compared to e.g. C but aren't the bottleneck. The win with Python (and other dynamic languages) is that you can experiment quickly with ideas when you're formulating a solution, that's a big part of exploratory data science. If you're curious about high-speed work in Python - Radim did a blog series on how he sped up word2vec to be faster than Google's original C code: http://radimrehurek.com/2013/09/deep-learning-with-word2vec-and-gensim… I'll also note [self promo!] that I wrote on book on High Performance Python, if that's your cup of tea (and Radim wrote a section in it): http://shop.oreilly.com/product/0636920028963.do

The gensim [0] package has a nice implementation of online LDA that can handle massive streaming datasets. If you want to avoid specifying the number of topics, you can use HDP-LDA. David Blei (inventor of LDA) has a reference implementation on his website [1], along with many other variants of LDA. [0] http://radimrehurek.com/gensim/ [1] http://www.cs.princeton.edu/~blei/topicmodeling.html

For fast and larger scale near-duplicate detection see Google All-Pairs Similarity Search: https://code.google.com/p/google-all-pairs-similarity-search/ . The SimHash algo: http://matpalm.com/resemblance/simhash/ and Gensim's Similarities Class: http://radimrehurek.com/gensim/similarities/docsim.html For blogspam detection can't PG whip up a nice Bayesian model for that ;) ? Again if you want fast and large scale spam filters see Vowpal Wabbit: http://hunch.net/~vw/ . For something more advanced see scikit-learn: http://scikit-learn.org/ which has many algo's up for this task. A new option you may not have considered yet, is to crowdsource this task. Just offer a labelled dataset and let the HN and ML community have a go at it. I'd love to do content and URL-based spam modelling.

that's a well-studied subject. check out CEPT [1], Word2vec [2] (demo implementation [3]) and Wikipedia Miner [4]. [1] http://www.cept.at/demo_retina_viewer.html [2] https://code.google.com/p/word2vec/ [3] http://radimrehurek.com/2014/02/word2vec-tutorial/#app [4] http://wikipedia-miner.cms.waikato.ac.nz/demos/compare/?term1=tumor&am…