Humanloop Blog

https://humanloop.com/ Find the prompts users love and fine-tune custom models for higher performance at lower cost

Based on this following article and my experience, I think there is something here. https://humanloop.com/blog/openai-plans

Humanloop (YC S20) | London (or remote) | https://humanloop.com Humanloop is helping the coming wave of AI startups build impactful applications on top of large language models. Our tools add capabilities, evaluate performance and align these systems with human feedback to create real world value. Here's a recent video interview between YC and Raza explaining what we do: https://www.youtube.com/watch?v=hQC5O3WTmuo We're looking for exceptional engineers that can work at varying levels of the stack (frontend, backend, infra), who are customer obsessed and thoughtful about product (we think you have to be -- our customers are "living in the future" and we're building what's needed). Our stack is primarily Typescript, Python, GPT-3. Please apply at https://www.workatastartup.com/companies/humanloop and feel free to reach me at [email protected]

This link still works and seems to explain their product: https://humanloop.com/home

I've been keeping a rolling list of LLMOps/AI agent observability products funded by YC. What problems does Lucidic solve that the others do not? https://hegel-ai.com https://www.vellum.ai/ https://www.parea.ai http://baserun.ai https://www.traceloop.com https://www.trychatter.ai https://talc.ai https://langfuse.com https://humanloop.com https://uptrain.ai https://athina.ai https://relari.ai https://phospho.ai https://github.com/BerriAI/bettertest https://www.getzep.com https://hamming.ai https://github.com/DAGWorks-Inc/burr https://www.lmnr.ai https://keywordsai.co https://www.thefoundryai.com https://www.usesynth.ai https://www.vocera.ai https://coval.ai https://andonlabs.com https://lucidic.ai https://roark.ai https://dawn.so/ https://www.atla-ai.com https://www.hud.so https://www.thellmdatacompany.com/ https://casco.com https://www.confident-ai.com

Altman mentioned[1][2] earlier that they were working on a "stateful" API for release this year. > 2023: A stateful API — When you call the chat API today, you have to repeatedly pass through the same conversation history and pay for the same tokens again and again. In the future there will be a version of the API that remembers the conversation history. Maybe it's an RAG-based thing, but that'd be underwhelming given the promise. Wizard of Oz, or true magic? (In the same interview, Altman also claimed progress toward releasing million-token context windows this year. Wowzers) [1] https://humanloop.com/blog/openai-plans, removed at OAI's request [2] Archived at https://web.archive.org/web/20230531203946/https://humanloop.com/blog/…

It was removed: https://humanloop.com/blog/openai-plans "This content has been removed at the request of OpenAI."

> The APIs for text manipulation in the DOM are abysmal I develop an NLP labelling interface (which is similarly document-focused) in JS all day and can attest to this. Here's some specific examples: - To work out where some text is being being laid out, eg to display some UI around it, the Selection/Range/Rect APIs will help but Rects are always viewport-relative, so you'll need to convert them to element-relative to position your UI, which sucks. - Firefox supports multiple DOM Ranges per Selection (ie allows non continguous text selection), but nearly all other browsers support a single range. - You can't truncate text across multiple lines (the CSS line clamp spec is an experimental fix) - If you want to pop up an interface while some text is selected, you'll need to capture and reimplement selection because as soon as another item is focused the selection will no longer display. If you're into JS and building anything document related, I've written a bunch about the some of the issues at https://humanloop.com/blog/how-to-build-an-ml-labelling-interface-in-j…

RE: Open Source models, there is the AnthropicAI model making the rounds on Twitter[1] and Stability.ai (makers of Stable Diffusion) are working on one [2]. If we use recent history as an example, OpenAI announced DALL-E in Jan 5, 2021 [3], announced v2 and a waitlist for public use in July 20, 2022, and Stable Diffusion shipped an open source model on August 22, 2022 [4] using ~$600K of compute (at retail prices on AWS) [5]. I don't see how it's likely that any company can acquire a durable technology moat here. There are scale barriers to entry, but even VC sized funding can overcome that. [1] https://twitter.com/goodside/status/1611556749726605312 [2] https://humanloop.com/blog/stability-ai-partnership [3] https://en.wikipedia.org/wiki/DALL-E [4] https://en.wikipedia.org/wiki/Stable_Diffusion [5] https://twitter.com/emostaque/status/1563870674111832066?lang=en