Explosion Blog

Highly recommend Spacy and the http://explosion.ai blog!

Before chunking, run coreference resolution to get rid of all of your pronouns and replace them with explicit references. You need to be a bit of careful to ensure you chunk both processed and unprocessed versions in the same places but it’s very doable. If you haven’t seen it, there’s a lovely overview of the idea in one of the SpaCy blog posts: https://explosion.ai/blog/coref

The title is a little misleading. They raised $6 million from SignalFire (who also invested in Grammarly) for 5% of the company [1]. As SignalFire retains that shareholding, they are not less independent (nor more) than that have been in the past couple of years. They are just not looking for more investment and so are more "independent-minded". I used SpaCy quite expensively pre-2020. It still has a lot of great uses, and more predictable than LLM models. But now many NLP tasks that seemed near impossible, or required a lot of expensive annotated training data (which their product Prodigy is used for) can now be coded in very little time with LLMs. [1] https://explosion.ai/blog/weve-sold-5-percent-of-explosion

Everyone has a different way of approaching. I like video walk-throughs of papers. If you are the same, for the theory bit you may find Yannic Kilcher's videos [0,1] helpful. On implementation side, the best project out there is [2] with good documentation. Also spacy has transformer package [3] (I personally haven't tried it), so maybe it will be easier for you to jump in if you have prior experience with spacy. [0] https://www.youtube.com/watch?v=iDulhoQ2pro [1] https://www.youtube.com/watch?v=-9evrZnBorM [2] https://huggingface.co/transformers/ [3] https://explosion.ai/blog/spacy-transformers

Keep up the great work, Matthew and Ines (et al)! This doesn't mention the transformer models [1] that were recently announced, so even though it's not 2.2 related, I'll just point it out because it's recent spaCy news :) I had thought I'd seen somewhere (spaCy IRL?) that this kinda stuff was not supposed to show up until v3. [1] https://explosion.ai/blog/spacy-pytorch-transformers

I would love to see more tutorials explaining how you can use these basic building blocks (also covered by the great documentation) to find semantic meaning for comparing sentences or understanding basic intent like https://explosion.ai/blog/chatbot-node-js-spacy or some other useful tool besides finding all the nouns in a document.

I work in NLP and I would really disagree that NLTK is still an essential player. Some academic courses still teach in it, but I think it's around five years since it has been the best option. I would definitely recommend gensim, but also SpaCy, which is very fast and has a better documented and more extensive API than NLTK. I have nothing to do with SpaCy personally (except being an enthusiastic user) but I recommend this argument: https://explosion.ai/blog/dead-code-should-be-buried There is also a new higher-level library built on Spacy which looks good: https://textacy.readthedocs.io/en/latest/

I definitely agree that the same weights won't be optimal for different domains. If you need to parse tweets, you should have a tweet-trained model. The tweet model probably shouldn't be thinking about Jane Austen novels. We want to open a model store where you can buy language and domain specific models. I think 99% of the time there's one best algorithm, and even one best implementation of it. It's the weights, and sometimes the features, that need to vary. Finally — I love displaCy too. Ines does great work :). Have you seen that we open-sourced this recently? It's now very easy to run locally, and connect up to the model you're developing. You can use this with any other parser, too. https://explosion.ai/blog/displacy-js-nlp-visualizer

A quick note about using natural language/sentiment APIs: trained machine learning models must be used apples-to-apples on similar datasets; for example, you can’t accurately perform Twitter sentiment analysis on a dataset using a model trained on professional movie reviews since Tweets do not follow AP Style guidelines. (e.g. for some reason, training Python's NLTK on the IMDb movie review dataset to predict the sentiment of Donald Trump's tweets is a oddly common Hello World, even though the results are misleading and may cause confirmation bias) Reddit comments are very idiosyncratic, and in this particular case, even moreso than usual. As a result, I am skeptical of trusting the output of such APIs as gospel, even one trained on massive datasets. (however, training a model on a Reddit-only dataset might be interesting, and is an idea I have in the pipeline.) Last year, spaCy trained a model, sense2vec, on the Reddit dataset and got interesting results: https://explosion.ai/blog/sense2vec-with-spacy

:wave: Also: https://explosion.ai/blog/back-to-our-roots-company-update (Interesting tidbit: I got hired by Explosion after a HN comment on model distillation :))

The SpaCy creator has a good blog post on this https://explosion.ai/blog/against-llm-maximalism

> They’re serious practitioners From the article in question: https://explosion.ai/static/1863c4dfa57ad28dbbd68e432bde34e9/2523c/llm…

I think pretty much any multicore ARM CPU with a post ARMv8 ISA is looking pretty strong for local AI right now. Same goes for x86 chips with AVX2 support. Apple Silicon AMX units provide the matrix multiplication performance of many core CPUs or faster at a fraction of the wattage. See eg. https://explosion.ai/blog/metal-performance-shaders https://github.com/danieldk/gemm-benchmark#1-to-16-threads

How interesting! (BTW I don't see where this sentence is coming from, and I looked in the papers too) Here's how Spacy sees it: [1] So in that case it thinks the command is "in 8 hours find a reservation that is far from my work location and is for 8 people at Union Auto Company". This parse is almost certainly incorrect. [1] https://explosion.ai/demos/displacy?text=Find%20a%20reservation%20far%…

Oh, I expect that you'd use NN NLP to do accurate parsing/tagging/NER then write rules based on the output of that parsing. If you look at https://explosion.ai/demos/displacy, you can see the high-level information you can extract from sentences nowadays, which better matches our understanding of how grammar rules work. And that's an approach that does generalize to languages like Japanese.

spaCy makes lots of money. from https://explosion.ai/about "In August 2021, we sold 5% of Explosion to SignalFire for $6 million. Employees are given a stake in Explosion using a virtual share bonus program."

Hey this is really awesome! Well done. You mentioned transformers and BERT for large NLP models. I've been playing around with this too and it's a really powerful approach. Have you used spacy-transformers? [0] The approach is pretty cool and can be used with BERT, GPT-2/Hugging Face etc. I'm just starting to experiment with GPT-J and thinking of trying this approach also [1]. Anyway, totally awesome project and the results are really good. This stuff really is almost unreasonably effective! [0] https://explosion.ai/blog/spacy-transformers [1] https://6b.eleuther.ai/

These days, at least in non-research contexts, people seem to mostly use spacy-transformers for that, already contains all the necessary glue code between huggingface transformers (including BERT) and spacy. https://explosion.ai/blog/spacy-transformers / https://spacy.io/universe/project/spacy-transformers

Tone down the entitlement and the attitude, and think about what the linked article is exactly. It's a bunch of release notes for the 3.0 version. If I click back to the actual homepage[0], it says: spaCy is a free, open-source library for advanced Natural Language Processing (NLP) in Python. It’s designed specifically for production use and helps you build applications that process and “understand” large volumes of text. It can be used to build information extraction or natural language understanding systems, or to pre-process text for deep learning. [0] https://explosion.ai/software#spacy

"spaCy is a free, open-source library for advanced Natural Language Processing (NLP) in Python. It’s designed specifically for production use and helps you build applications that process and “understand” large volumes of text. It can be used to build information extraction or natural language understanding systems, or to pre-process text for deep learning." https://explosion.ai/software#spacy

Although the SO question is focused on the goal, this broad kind of task is quite a common requirement for chatbots, so looking at tools like RASA and Spacy for this are good places to explore (for those not opting for the commercial API approach others sensibly suggested) https://rasa.com/docs/rasa/nlu/entity-extraction/ Spacy have a handy demo of NER (although it appears down for me right now): https://explosion.ai/demos/displacy-ent/

There is also sense2vec which I think tries to do something similar. https://explosion.ai/blog/sense2vec-with-spacy

I don't think spaCy will be hurting any time soon. When SyntaxNet was first released last year, Matthew Honnibal had a good writeup [0] of how spaCy vastly outperforms with speed while keeping reasonable accuracy: >On the time-honoured benchmark for this task, Parsey McParseface achieves over 94% accuracy, at around 600 words per second. On the same task, spaCy achieves 92.4%, at around 15,000 words per second. The extra accuracy might not sound like much, but for applications, it's likely to be pretty significant. If spaCy is able to increment the accuracy and maintain the large performance gap, it'll still be my go-to NLP framework! [0] https://explosion.ai/blog/syntaxnet-in-context

Give Cython a try, you can get huge gains in memory/speed by carefully transitioning from Python to C with it. (Or use it to "write C in Cython"[0]) [0] https://explosion.ai/blog/writing-c-in-cython

I'm not the author but can speak to some of this. Some amount of this was answered by the author himself in his blog post "Dead Code Should Be Buried – Why I Didn't Contribute to NLTK"[1]. While there are portions on NLTK that are useful, it suffers from having too many things under the hood and a lack of consistent maintenance. Many of them are superglued external libraries while others have not been maintained or have never been made to work well with other parts of NLTK. The biggest example of this was that, for some time, it wasn't certain how the NLTK part of speech tagger was trained[2]. While that has been remedied, the remedy uses part of speech tagging code inspired via earlier work by the author of spaCy[3][4]. spaCy has been made from the ground up to support more modern deep learning features (easy integration of Keras is quite cool but I'm actually meaning the default inclusion of word vectors which has existed since early spaCy) while having a consistent and clean API that covers many sane use cases. The API is highly Pythonic but the underlying performance critical components are written in Cython, meaning very few parsers are actually able to beat spaCy in speed (especially when using CPU and not GPU). Regarding speed, multi-threading is also a relatively trivial process - Cython releases the GIL around the syntactic parser. As a reference to speed, they used it to tag and parse every Reddit comment from 2015 to then generate "sense2vec"[5] at over…