Hugging Face Blog
medium.com/huggingface · Company blog · on HN since 2017
The Hugging Face team writes about NLP models, benchmarks, distillation and training techniques for PyTorch and TensorFlow.
What HN saysHN commenters link it for DistilBERT, the PyTorch versus TensorFlow benchmarks and its articles on sparse neural networks.
25HN people shared it
5linked it in comments
34stories submitted
202story points
2 years agolast shared
2017people per year2026
Most-shared posts
- 1Benchmarking PyTorch and TensorFlow on Many NLP Models4 people · 11 points · first shared 2019 · discussion on HN
- 2DistilBERT, a Distilled Version of Bert4 people · 5 points · first shared 2019 · discussion on HN
- 3FastBert – A Simple Deep Learning Library for Bert Models3 people · 6 points · first shared 2019 · discussion on HN
- 4Faster Natural Language Processing in Python2 people · 104 points · first shared 2018 · 3 comments on HN
- 5A high-level introduction to Meta-learning2 people · 14 points · first shared 2018 · discussion on HN
- 6Understanding emotions – from Keras to pyTorch2 people · 9 points · first shared 2017 · discussion on HN
- 7Training NeuralNets on Larger Batches: Tips for 1-GPU,N-GPU and Distrib. Setups2 people · 7 points · first shared 2018 · discussion on HN
- 8How to Build a State-of-the-Art Conversational AI with Transfer Learning2 people · 7 points · first shared 2019 · discussion on HN
- 9From TensorFlow to PyTorch2 people · 7 points · first shared 2019 · discussion on HN
- 10Scaling a State-of-the-Art Deep Learning Model in Production2 people · 5 points · first shared 2019 · discussion on HN
- 11And Most Current of Modern Natural Language Processing2 people · 4 points · first shared 2019 · discussion on HN
- 12Beating the state-of-the-art in NLP with HMTL2 people · 4 points · first shared 2018 · discussion on HN
- 13Simple considerations for simple people building fancy neural networks2 people · 3 points · first shared 2020 · discussion on HN
- 14The State of Universal Word Embeddings and Sentence Embeddings2 people · 3 points · first shared 2018 · discussion on HN
- 15Is the Future of Neural Networks Sparse?2 people · 3 points · first shared 2020 · discussion on HN
What HN people wrote when they linked it
liuliu · 2020-08-18
It is from my own experience. But it looks like inference is a bit faster on CPU: https://medium.com/huggingface/benchmarking-transformers-pytorch-and-t…
madlag · 2020-03-07
OpenAI just announced it will port its sparse libraries to PyTorch, so exciting times ahead!
You can read more here about this (OP here) :
https://medium.com/huggingface/is-the-future-of-neural-networks-sparse…
halflings · 2020-01-09
> Edit: I guess one solution would be to use a pretrained ALBERT and finetune to get the initial model and then use model distillation to get a smaller, faster model.
Huggingface did just that:
https://medium.com/huggingface/distilbert-8cf3380435b5
elcomet · 2019-10-07
This is similar to DistilBert and DistilGPT2 by HuggingFace
https://medium.com/huggingface/distilbert-8cf3380435b5
bitforger · 2019-07-08
The memory constraint is certainly an issue, but I think they can be overcome with some software hacks (for a speed penalty, of course.) For example, gradient checkpointing and gradient accumulation might help:
https://medium.com/tensorflow/fitting-larger-networks-into-memory-583e…
https://medium.com/huggingface/training-larger-batches-practical-tips-…