Blogs

We present a new tool -- Patched GitHub repo analyzer that helps users discover different kinds of LLM-assisted workflows in their code.

We describe a patchflow for creating a customizable AutoFix tool that can automatically detect and fix software vulnerabilities using large language models (LLMs).

We introduce Generate README Eval, a new benchmark to evaluate how well LLMs can generate README files from entire code repositories. Our results show that Gemini-1.5 is the SOTA model on this benchmark.

OpenAI invited Patched to fine-tune gpt-4o to build the SOTA model for this benchmark. We worked alongside their team to understand how performance can be improved by fine-tuning frontier models.

We compare fine-tuning GPT-4o-mini, Gemini Flash 1.5, and Llama-3.1-8B models using a custom vulnerability fixes dataset, with GPT-4o-mini showing the most significant improvement and setting a new benchmark.

We discuss Mixture of Agents, an inference optimization technique that enhances gpt-4o-mini to outperform gpt-4-turbo at ~1/50th the cost for a variety of software development tasks.