AI code review for GitHub pull requests
Solemly reviews GitHub pull requests as a GitHub App. Each review is one native pull request review with inline comments on the changed lines, and the result is reported as Solemly / Review, Solemly / Security and Solemly / Summary check runs that you can make required before merging.
Checked against the product on 2026-11-03.
What a review posts on GitHub
- One native pull request review per review. Its body is the summary: what the change does and the overall verdict, or No issues found.
- Inline comments on the changed lines, one per finding, each with its severity.
- A suggested change you can apply in one click, where a fix is short and clear.
- Findings that aren't on a changed line, listed in the summary under Not on a changed line.
- A footer naming the reviewed commit, such as
Solemly AI review · commit abc1234.
The check runs
| Check | Reports |
|---|---|
Solemly / Review | The review: complete, or blocking when a high-severity finding is open and the repository blocks on them. |
Solemly / Security | The security review, on its own check. |
Solemly / Summary | Whether a summary of the change was written into the pull request description. |
Solemly / Acceptance | With Jira connected, whether the change does what its linked ticket asks. |
When a pull request is reviewed
Solemly reviews a GitHub pull request when it's opened, marked ready for review, updated with new commits, or reopened. Draft pull requests are skipped until they're marked ready, and closed pull requests aren't reviewed.
| Review when | Behaviour |
|---|---|
| Every push (default) | Reviews every new commit. |
| Once per PR | Reviews a pull request once. Later pushes are skipped unless you ask with @solemly review. |
| On request | Never reviews automatically. Comment @solemly review when you want one. |
Several commits pushed in quick succession get one review, of the latest. A new push supersedes any review of an older commit that's still running, so you only ever see reviews of the latest code.
Install the GitHub App
- In Solemly, open Repositories and choose GitHub. Only workspace admins can connect a code host.
- Click Install GitHub App.
- On GitHub, pick the account or organization, then choose All repositories or Only select repositories.
- Approve the install. GitHub sends you back to Solemly, and your repositories appear within a few seconds.
Whoever finishes the install on GitHub must be able to see every repository it covers, such as an owner of the organization. Every connected repository starts with AI review turned on, and you can change that per repository.
Permissions it asks for
| Permission | Access | Why |
|---|---|---|
| Pull requests | Read & write | Read the change, post the review and its comments |
| Checks | Read & write | Report the Solemly / … check runs |
| Contents | Read | Read files around the change for context, and index the repository's code |
| Issues | Read & write | Read and reply to @solemly comments |
| Metadata | Read | Required by every GitHub App |
Block the merge
Turn on Block on high severity in the repository's settings in Solemly, and Solemly / Review fails on any CRITICAL or HIGH finding. Then, in the branch protection rule or ruleset on GitHub, turn on Require status checks to pass before merging and add Solemly / Review. The check appears in the search once Solemly has reviewed at least one pull request in the repository. See Block a merge on AI code review findings.
Questions
Which GitHub events trigger a review?
A pull request being opened, marked ready for review, updated with new commits, or reopened. Draft pull requests are skipped until they're marked ready.
Can I choose which repositories Solemly can see?
Yes. When you install the GitHub App you pick All repositories or Only select repositories, and you can change that at any time on GitHub under Settings → GitHub Apps → Solemly → Configure. Solemly picks up the change automatically.
Does Solemly keep a copy of our code?
No. It reads the diff and the related files while a review runs and sends them to the AI model to write the review. What it stores is the review itself - the comments and findings it posted - plus pull request details like title, author and merge time, for your reports.
How do I make the review a required check?
Turn on Block on high severity for the repository in Solemly, then add Solemly / Review to Require status checks to pass before merging in the branch's protection rule or ruleset on GitHub.
In the docs
Related
Try it on your next pull request
Free for 14 days with every feature, no credit card. Install it on a repository and the next pull request gets reviewed.