Discover LLM-assisted workflow opportunities in your code

Recently, we have had a huge surge of users for our open-source framework and the patched app. And, after trying out the collection of existing patchflows that automate, vulnerability fixes, code reviews, documentation generation etc. many of the new users wonder how they can build new patchflows or customize existing ones for other opportunities they may have in their code base. To address this we have released a free tool called the Patched GH Repo Analyzer. In this blog, I will briefly describe how it works and show you some sample reports that are generated by the tool to identify LLM-assisted workflow opportunities in some prominent open-source projects.

How does it work?

You just enter your repository URL and take a holistic look at the repository and identify opportunities across code, issues, and pull requests. In order to do so, we split the analysis into three parts – code, issues and prs.

Code Analysis

To analyze the issues with code, we first scan it with an open source static analysis tool like Semgrep. In our previous blog post we had also mentioned Semgrep and how it is a useful tool to detect security vulnerabilities in code. We then parse the findings from the Semgrep report and use that along with brief snippets from all the source code files in the repo to use an LLM and provide an analysis covering:     

1. Patterns in the codebase     

2. Best practices being followed or missing     

3. Areas for improvement     

4. Potential security vulnerabilities or bugs (based on Semgrep results)     

5. Opportunities for LLM-assisted automation in coding tasks

You can refer to the full prompt used here.
‍
Based on the output from the LLM, we format it to a report that looks like below:
‍

Issues Analysis

For analyzing GitHub issues in the repository we take a two phase approach. We first analyze the closed issues to identify patterns and common themes that have already been part of the repository. Then, we analyze the open issues along with the summary of phase 1 to identify specific open issues that can be improved with LLM-assisted workflows. 

Here is an example of what that report looks like:
‍

As you can see from above we have identified the specific Issue #344 and the workflow that can be used to improve it.

Pull Requests (PRs) Analysis

For PRs we analyze them similar to the issues and identify open PRs that can benefit from an LLM-assisted workflow. 

Together these three analyses offer different ideas on how you can leverage LLM-assisted workflows in your repository. Do try out the tool on your own repo. You can also run the analysis on private GitHub repos, just put your personal access token (PAT) in the GitHub Token field in advanced settings and you will be able to analyze your private GitHub repository. Remember it can take a bit of time as it does multiple analysis so depending on the size of repo you may take a few mins to get the results. You can then build your LLM-assisted workflow in the patched app and if you need help or run into issues you can join our discord and ask questions.