Qodo Blog

Seems like an interesting benchmark, but my takeaway from the results is that Codex is almost as good enough as their custom solution (no mention of the underlying model) and only requires a $20 ChatGPT subscription to start using it (of course with limits), without having to shell out $$$ for an enterprise Qodo plan to use Qodo Aware - https://www.qodo.ai/products/qodo-aware/. The "free" plan in Qodo Aware only lets users work with 100 hand-picked open-source repositories. It also would be nice if the article clearly mentioned what specific model settings were used for Claude Code and Codex. Both of those allow changing the reasoning level, so if the benchmark was done using the default settings, it seems a little unfair - they have a result of their own agent at high reasoning as a separate entry.

This is in relation to their newly-announced "context agent": https://www.qodo.ai/blog/introducing-qodo-aware-deep-codebase-intellig…

Testing against unspecified other "leading" models allows for shenanigangs: > Qodo tested GPT‑4.1 head-to-head against other leading models [...] they found that GPT‑4.1 produced the better suggestion in 55% of cases The linked blog post goes 404: https://www.qodo.ai/blog/benchmarked-gpt-4-1/

From OpenAI's announcement: > Qodo tested GPT‑4.1 head-to-head against Claude Sonnet 3.7 on generating high-quality code reviews from GitHub pull requests. Across 200 real-world pull requests with the same prompts and conditions, they found that GPT‑4.1 produced the better suggestion in 55% of cases. Notably, they found that GPT‑4.1 excels at both precision (knowing when not to make suggestions) and comprehensiveness (providing thorough analysis when warranted). https://www.qodo.ai/blog/benchmarked-gpt-4-1/

> validates each test to ensure it runs successfully, passes, and increases code coverage This seems to be based on the cover agent open source which implements Meta's TestGen-LLM paper. https://www.qodo.ai/blog/we-created-the-first-open-source-implementati… After generating each test, it's automatically run — it needs to pass and increase coverage, otherwise it's discarded. This means you're guaranteed to get working tests that aren't repetitions of existing tests. You just need to do a quick review to check that they aren't doing something strange and they're good to go.

Apparently that's Codium, who have recently renamed themselves to Qodo: https://www.qodo.ai/blog/introducing-qodo-a-new-name-the-same-commitme… (TIL)

Huh, look at that. I completely mixed them up too. Thankfully they have an article which "helpfully" clarifies things: > We’ve noticed that there’s been some confusion between our company, Codium (now Qodo (formerly Codium)), and another company, Codeium. https://www.qodo.ai/blog/codiumai-or-codeium-which-are-you-looking-for…

All these arguments remind me of early LLMs. In fact, most of the arguments are nearly identical. Heck, even Bitcoin back 10 years ago was demonized and now governments are pushing for national reserves. If you have used GPT o3, you have used Agentic models. If you use Claude Code or Cursor, you have used Agentic models. If you have used Claude + MCP, you have used agentic models. This isn't theoretical - its already being applied. In fact, Langchain just hosted an entire conference to showcase how Fortune 100 companies were using LangGraph and I can assure you if they could have used a more simple architecture they would have. If you wait around until everyone considers it to be acceptable in the mainstream all the good opportunities to create disruptive tech will be gone. So please - keep publishing this luddite trash and throw shade on the sidelines. I will keep doing agent research and building with it. We will see in 10 years who was wrong I guess. https://www.qodo.ai/blog/building-agentic-flows-with-langgraph-model-c…

> Where are the "code-review" agents at? OpenAI's Codex Cloud just added a new feature for code review, and their new GPT-5-Codex model has been specifically trained for code review: https://openai.com/index/introducing-upgrades-to-codex/ Gemini and Claude both have code review features that work via GitHub Actions: https://developers.google.com/gemini-code-assist/docs/review-github-co… and https://docs.claude.com/en/docs/claude-code/github-actions GitHub have their own version of this pattern too: https://github.blog/changelog/2025-04-04-copilot-code-review-now-gener… There are also a whole lot of dedicated code review startups like https://coderabbit.ai/ and https://www.greptile.com/ and https://www.qodo.ai/products/qodo-merge/