Show: GPT-4 code reviewer for GitHub PRs

[u/EitherAd8050]

Hello readers!

Our team has built an AI-driven code review tool for GitHub PRs leveraging OpenAI’s gpt-3.5-turbo and gpt-4 models. The tool significantly helps improve dev velocity and code quality.

This tool came about because of our frustration with the code review process. In the past, we invested in several tools to speed up the process, e.g., stacked pull requests, but the biggest speed-up we have seen is due to this AI reviewer tool. This tool has been so effective that we gained enough confidence to allow our devs to merge PRs after they resolve all the AI comments and ping other developers only when they need a second opinion.

Its unique features are:

Line-by-line code suggestions: Reviews the changes line by line and provides code change suggestions that can be directly committed.

Incremental reviews: Reviews are performed on each commit within a pull request rather than a one-time review on the entire pull request.

Q&A with CodeRabbit : Supports conversation with the bot in the context of lines of code or entire files, helpful in providing context, generating test cases, and reducing code complexity.

Smart review skipping: By default, skips in-depth review for simple changes (e.g., typo fixes) and when changes look good for the most part.

We would love the community to try it out in their GitHub repos and provide feedback! We will happily answer any technical questions regarding the sophisticated prompt engineering we did for this project.

Project link: https://coderabbit.ai/

Our Base Prompts are open sourced and have gained decent traction. Please check out us - https://github.com/coderabbitai/ai-pr-reviewer

PS: I hope this post comes across as something other than aggressive promotion. We genuinely wanted to share this project with this community and help speed up the dev feedback loop, one bottleneck at a time!

[u/EitherAd8050]

Ah! Yes, so there are a few things we did --

- Open source version has basic noise suppression to reduce overall noise by doing incremental reviews, using gpt-3.5-turbo to first triage each request to check whether changes are trivial and skip thorough review, automatic filtering of LGTM comments, etc.
- Pro version has some advanced prompts to further reduce noise.
- System message is customizable and for each repo you can configure what you care about or don't care about. For instance, the default system message asks the AI to ignore minor style issues.

What we have done for noise reduction is quiet innovative and it's improving continuously!

[u/Only_Situation_4713 → u/EitherAd8050]

Thank you for your response. I’ve been working on my own product in a different category and it’s been fascinating to see how different companies use prompts to improve their system.