Answer.AI
answer.ai · Company blog · on HN since 2023
Answer.AI, an AI research and development lab, writes about open models, fine-tuning, GPU programming and AI tooling.
What HN saysHN links it for posts on fine-tuning a 70B model at home, WebGPU Puzzles, the llms.txt proposal, and its critique of SB-1047.
44HN people shared it
16linked it in comments
48stories submitted
2.5Kstory points
3 months agolast shared
2023people per year2026
Most-shared posts
- 1SB-1047 will stifle open-source AI and decrease safety7 people · 225 points · first shared 2024 · 189 comments on HN
- 2Fine tune a 70B language model at home5 people · 984 points · first shared 2024 · 206 comments on HN
- 3Show HN: WebGPU Puzzles - Learn GPU Programming in Your Browser4 people · 19 points · first shared 2024 · 2 comments on HN
- 4Thoughts on a month with Devin3 people · 285 points · first shared 2025 · 187 comments on HN
- 5Faith and Fate: Transformers as fuzzy pattern matchers3 people · 5 points · first shared 2024 · discussion on HN
- 6A little pooling goes a long way for multi-vector representations3 people · 5 points · first shared 2024 · 1 comments on HN
- 7Gpu.cpp: A lightweight library for portable low-level GPU computation2 people · 242 points · first shared 2024 · 52 comments on HN
- 8MonsterUI: Python library for building front end UIs quickly in FastHTML apps2 people · 168 points · first shared 2025 · 55 comments on HN
- 9Claudette, a new friend that makes Claude 3.5 Sonnet even nicer2 people · 22 points · first shared 2024 · 8 comments on HN
- 10How to Solve It with Code2 people · 13 points · first shared 2024 · 5 comments on HN
- 11Efficient finetuning of Llama 3 with FSDP QDoRA2 people · 13 points · first shared 2024 · 3 comments on HN
- 12The Stripe Experience You Deserve2 people · 7 points · first shared 2025 · 1 comments on HN
- 13Rerankers: A Lightweight Python Library to Unify Ranking Methods2 people · 5 points · first shared 2024 · discussion on HN
- 14ShellSage – Your AI Bash Buddy2 people · 5 points · first shared 2024 · discussion on HN
- 15A new old kind of R&D lab2 people · 5 points · first shared 2023 · discussion on HN
What HN people wrote when they linked it
benreesman · 2024-09-29
Some laws are just bad. When the API-mediated/closed-weights companies agree with the open-weight/operator-aligned community that a law is bad, it’s probably got to be pretty awful. That said, though my mind might be playing tricks on me, I seem to recall the big labs being in favor at one time.
There are a number of related threads linked, but I’ll personally highlight Jeremy Howard’s open letter as IMHO the best-argued case against SB 1047.
https://www.answer.ai/posts/2024-04-29-sb1047.html
srush · 2024-09-23
Here is a port without the visualizer:
https://twitter.com/srush_nlp/status/1719376959572980094
Here is an amazing in-browser implementation in WebGPU
https://www.answer.ai/posts/2024-09-12-gpupuzzles.html
jph00 · 2024-09-04
From the post describing llms.txt (https://www.answer.ai/posts/2024-09-03-llmstxt.html):
"The problem this solves is that today, constructing the right context for LLMs based on a website is ambiguous — do you:
1. Crawl the sitemap and include every page, trying to automatically format into an LLM-friendly form?
2. Selectively include external links in addition to the sitemap?
3. For specific domains like software documentation should you also try to include all the source code?
Site authors know best, and can provide a list of content that an LLM should use."
(There's quite a bit more info there that answers this question in more detail.)
simonw · 2024-07-02
The web interface gives you access to the Artifacts feature where it can build SPAs and render them in the browser.
For terminal access I like using my own https://llm.datasette.io/ tool with the https://github.com/simonw/llm-claude-3 plugin
For Python library access I recommend checking out Claudette: https://www.answer.ai/posts/2024-06-21-claudette.html
mobicham · 2024-03-31
Yes, correct, and that fetching operation is a non-blocking operation, once we dequantize the weights we discard it before moving to the next layer.
Technically, you can do it for the weights as well. But that wouldn't work in many situations. For example, when training with FSDP: the quantized weights stay on the device but you can still offload the meta-data (https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html)
I would like to re-iterate that larger models, which would be more interesting to run at low-bits, are much less sensitive to quantization compared to a 7B. So you could potentially use a larger group-size and just keep it on device, like what is done with 4-bit and 3-bit now using a group size of 64. We just started running some experiments with a 13B llama2 and it looks very good so far (outperforming some full-precision llama2-13B-based models), let's see how far we can push it, ideally get-rid of the reshaping all together will be great.
soheil · 2025-02-07
I get this eerie feeling she has no idea what a bot even is in a technical sense and most likely is being puppeteered by weasels like State Senator Scott Wiener having introduced gems like the AI Models Act [1][2][3]. If the bill passes wouldn't then she have to be the first to disclose she's a bot?
[1] https://www.answer.ai/posts/2024-04-29-sb1047.html (SB-1047 will stifle open-source AI and decrease safety)
[2] https://safesecureai.org/responseletter (Response to inaccurate, inflammatory statements by Y Combinator & a16z regarding Senate Bill 1047)
[3] https://news.ycombinator.com/item?id=41690302 (Gavin Newsom vetoes SB 1047)
breckenedge · 2025-01-26
Just blogspam rehash of https://www.answer.ai/posts/2025-01-08-devin.html#what-is-devin
See https://news.ycombinator.com/item?id=42734681
steve_avery · 2025-01-17
As someone who is wading through a bloated Devin PR right now, this article resonates with me. A direct link might be useful to some: [1]
1: https://www.answer.ai/posts/2025-01-08-devin.html
I have been surprised by the strength of Devin at the beginning of my team's use of it, but lately it has been a letdown. I don't know that we are ready to cancel yet but Devin faces some growing headwind for me, at least.
hugetim · 2025-01-13
Curious how folks think this measures up: https://www.answer.ai/posts/2024-06-23-claudette-src.html
jonathan-adly · 2024-12-05
I would like to throw our project in the ring that solves this problem: https://github.com/tjmlabs/ColiVara
1. Uses half-vecs, so you cut down everything by half with no recall loss 2. Uses token pooling with hierarchial clustering at 3, so, you further cut down things by 2/3rd with <1% loss 3. Everything is on Postgres and pgvector, so you can do all the Postgres stuff and decrease corpus size by document metadata filtering 4. We have a 5000+ pages corpus in production with <3 seconds latency. 5. We benchmark against the Vidore leaderboard, and very near SOTA
You can read about half-vecs here: https://jkatz05.com/post/postgres/pgvector-scalar-binary-quantization/
Hierarchical token pooling: https://www.answer.ai/posts/colbert-pooling.html
And how we implemented them here: https://blog.colivara.com/
litmus · 2024-06-29
chef's kiss
This guy and the person that quit the bullshit industrial complex 6 months ago should get together and launch a startup.
Better yet, we should all go and join Jeremy Howard's answer.ai pro bono. Besides being miraculously headed by a guy who is Not An Asshole, it incidentally also had the most refreshing launch post (in the warm and fuzzy way) this side of the AI bubble.
The launch post concluded with this heading:We Don’t Really Know What We’re Doing.[0]
I mean, for the finest minds in our respective fields, what else is there left to say really?
[0] https://www.answer.ai/posts/2023-12-12-launch.html
mdp2021 · 2024-04-30
Excerpt from https://www.answer.ai/posts/2024-04-29-sb1047.html
DreamGen · 2024-04-29
This could have grave impact on AI development. Here are some responses:
- EFF: https://www.context.fund/policy/2024-03-26SB1047EFFSIA.pdf
- Answer AI: https://www.answer.ai/posts/2024-04-29-sb1047.html
batterseapower · 2024-04-28
The other recent improvement suggested for LoRA is DoRA: https://magazine.sebastianraschka.com/p/lora-and-dora-from-scratch. It really does seem to strongly outperform LoRA - see also https://www.answer.ai/posts/2024-04-26-fsdp-qdora-llama3.html
tveita · 2024-03-11
For an idea of what's possible you might be interested in this story that was just on HN where they fine tune a quantized 70b model in 48GB VRAM: https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html