Boundary Blog

BAML (https://boundaryml.com/) is really great for handling structured outputs. It's what we use at my company.

Hey, this is Gabe from zenfetch. Been following you guys for a few months now since your first launch. I definitely resonate with all the problems you've described regarding celery shortcomings / other distributed task queues. We're on celery right now and have been through the ringer with various workflow platforms. Only reason we haven't switched to Hatchet is because we are finally in a stable place, though that might change soon in which case I'd be very open to jumping ship. I know a lot of folks are going after the AI agent workflow orchestration platform, do you see yourselves progressing there? In my head, Hatchet coupled with BAML (https://www.boundaryml.com/) could be an incredible combination to support these AI agents. Congrats on the launch

I have anecdotal experience here, but I've found more success when solving the task first, and then returning it as JSON in a separate LLM call[0]. Running a single non-reasoning LLM call from source data (text/image/audio in your diagram) to structured JSON seems fragile with the current state of LLMs. You're essentially asking the model to do two tasks in one pass: parse the input and then format the output. It's amazing it works a lot of the time, but reasonable to assume it won't all of the time. (As a human, when I'm filling out a complex form, I'll often jump around the document) Curious how the benchmarks change when you add an intermediary representation, either via reasoning or an additional LLM call. I'd also love to see a comparison with BAML[1]. [0]In my experience we were using structured outputs as part of an agentic state machine, where the JSON contained code snippets (html/js/py/etc.). In the cases where we first prompted the model for the code, and then wrapped it in JSON, we saw much higher quality/success than asking for JSON straightaway. [1]https://boundaryml.com/

I kind of like BAML https://boundaryml.com/ been using it in production Edit, read the article -its really good- that cycle of AI engineering progression is spot on -read the article too!

We’ve had a lot of success implementing schema-aligned parsing in BAML, a DSL that we’ve built to simplify this problem. We actually don’t like constrained generation as approach - among other issues it limits your ability to use reasoning - and instead the technique we’re using is algorithm-driven error-tolerant output parsing. https://boundaryml.com/

The use case in the article is relatively simple. For more complex structures, BAML (https://www.boundaryml.com/) is a better option.

I have had a lot of success using BoundaryML (https://www.boundaryml.com/) for this. They have also been super responsive for any of my questions.

https://boundaryml.com/ LLM to types and done

Have you tried techniques that don’t require modifying the LLM and the sampling strategy for structure outputs? For example, schema aligned passing, where you build error tolerance into the parser instead of coercing to a grammar. https://boundaryml.com/blog/schema-aligned-parsing

Meanwhile https://dev.to/rishabdugar/crafting-structured-json-responses-ensuring… and https://www.boundaryml.com/blog/structured-output-from-llms You are confusing API response payloads with Structured JSON that we expect to conform to the given Schema. It's carnage that requires defensive coding. Neither OpenAI nor Google are interested in fixing this, because some developers decide to retry until they get valid structured output which means they spend 3x-5x on the calls to the API.

You might also find Semantic Streaming interesting. It's t he same concept but applied to llm token streaming. It's used in BAML (the ai framework). https://www.boundaryml.com/blog/semantic-streaming I'm one of the developers of BAML.

I expected to read about the methods used by the libraries to get the structured output and not a comparison of the language compatibility for each. Fortunately the same author have a blog post (https://www.boundaryml.com/blog/type-definition-prompting-baml) explaining how their approach works and how it compares to instructor (https://github.com/jxnl/instructor). Basically these libraries provide two things: 1. A way to prompt the LLM 2. A way to get a valid JSON For 1. instructor does it through the json schema definition, BAML's innovation is that they use a simplified lossless schema definition that uses less tokens. For 2. instructor does it through reprompting until they receive a valid JSON. BAML's innovation is a fuzzy parser able to to parse non-perfect JSON. Personally I think that there is no need to all these abstractions to get structured outputs from LLMs. A simple .to_prompt() function that takes a pydantic and translate it into some prompt block you can add to your prompt and a retry is sufficient to get the same results.