BentoML Blog
bentoml.com · Company blog · on HN since 2019
BentoML, a company building tools for serving AI models, blogs about LLM inference, deployment, and open source models.
What HN saysHN commenters ask how BentoML compares to other tools and link its posts on DeepSeek-OCR and inference benchmarks, one disputing its claims.
19HN people shared it
7linked it in comments
46stories submitted
539story points
7 months agolast shared
2019people per year2026
Most-shared posts
- 1Is LMDeploy the Ultimate Solution? Why It Outshines VLLM, TRT-LLM, TGI, and MLC5 people · 45 points · first shared 2024 · 8 comments on HN
- 2Homepage3 people · first shared 2022
- 3BentoML: One Model to Rule Them All2 people · 5 points · first shared 2019 · discussion on HN
- 4Building a Production-Ready LangChain Application with BentoML and OpenLLM2 people · 3 points · first shared 2023 · discussion on HN
- 52024 State of AI Inference Infrastructure Survey Results2 people · 3 points · first shared 2025 · 1 comments on HN
- 6Why should you care about compound AI?2 people · 2 points · first shared 2024 · discussion on HN
- 7The Complete Guide to DeepSeek Models: From V3 to R1 and Beyond2 people · 2 points · first shared 2025 · discussion on HN
- 8LLM Inference Handbook1 people · 366 points · first shared 2025 · 26 comments on HN
- 9Navigating the World of Large Language Models1 people · 48 points · first shared 2024 · 4 comments on HN
- 10A List of Top Open-Source Embedding Models1 people · 5 points · first shared 2024 · 1 comments on HN
- 11Nvidia Data Center GPUs Explained: From A100 to B200 and Beyond1 people · 4 points · first shared 2025 · discussion on HN
- 12Building RAG with Open-Source and Custom AI Models1 people · 4 points · first shared 2024 · discussion on HN
- 13The Shift to Distributed LLM Inference1 people · 4 points · first shared 2025 · discussion on HN
- 14Where to Buy or Rent GPUs for LLM Inference: The 2026 GPU Procurement Guide1 people · 3 points · first shared 2025 · discussion on HN
- 15A Guide to Open-Source Image Generation Models1 people · 3 points · first shared 2024 · discussion on HN
What HN people wrote when they linked it
nradov · 2026-04-16
There are already several fully open source LLMs. You can start participating in those projects today.
https://www.bentoml.com/blog/navigating-the-world-of-open-source-large…
michaelmior · 2024-06-14
There's also the option of platforms such as BentoML (I have no affiliation) that offer usage-based pricing so you can at least take the 100% utilization assumption off the table. I'm not sure how the price compares to EKS.
https://www.bentoml.com/
catchclose8919 · 2022-04-21
How does this compare to BentoML (https://www.bentoml.com/)?
sherlockxu · 2025-10-27
Seems there’s still some confusion around what DeepSeek-OCR really does. Learn about the model, Contexts Optical Compression, and its impact on LLMs here: https://www.bentoml.com/blog/deepseek-ocr-contexts-optical-compression…
KaoruAoiShiho · 2024-06-29
Is it really that close in production? The benchmarks I've seen show it's almost 3x slower compared to the SOTA. https://bentoml.com/blog/benchmarking-llm-inference-backends
ignoramous · 2024-02-14
> What about alternatives?
Custom models? Apart from the Big 3 (in no particular order):
- https://together.ai/
- https://replicate.com/
- https://anyscale.com/
- https://baseten.co/
- https://modal.com/
- https://banana.dev/
- https://runpod.io/
- https://bentoml.com/
- https://brev.dev/
- https://octo.ai/
- https://cerebrium.ai/
...
ALLTaken · 2025-06-02
This is really not true my friend. I would love to help you if I had some more time, but let me look for a tutorial.
Because the FineWeb Dataset is already super good. You can train 7B or 32B Param models at home
The >600B Param model isn't really using all the data effectively, but with a MacStudio Farm you can also train that one at home (if you have enough money to buy at least 100).
Here's the easy way: https://github.com/FareedKhan-dev/train-deepseek-r1
More details: https://www.bentoml.com/blog/the-complete-guide-to-deepseek-models-fro…
Here's how DeepSeek-R1-Zero was built, basically from 0 to Hero, including weights the FULL Training Data and everything you need to get it running locally or on servers.https://medium.com/@GenerationAI/how-deepseek-r1-zero-was-reproduced-i…
For $30 USD you can also train a small DeepSeek at home!
https://github.com/Jiayi-Pan/TinyZero
https://huggingface.co/deepseek-ai/DeepSeek-R1-Zero (the model)