Block a merge on AI code review findings

A merge gate takes two steps. Turn on Block on high severity in Solemly's repository settings, so the Solemly / Review check fails on any CRITICAL or HIGH finding, then make that check required on your code host. The pull request can't merge until a new push clears the finding, and if Solemly can't complete a review, the check lets the merge through.

Checked against the product on 2026-11-03.

Step 1: turn on blocking in Solemly

In Repository Settings, turn on Block on high severity for the repository. From then on:

any CRITICAL or HIGH finding makes Solemly / Review fail, reading Blocking: N high-severity finding(s)
a CRITICAL or HIGH security finding also makes Solemly / Security fail

With blocking off, the default, the checks always pass and the findings are advice.

Step 2: require the check on your code host

Code host	Where	What to set
GitHub	Settings → Branches, or Rules → Rulesets, on the main branch	Turn on Require status checks to pass before merging and add `Solemly / Review`. Add `Solemly / Security` too to block on security findings on their own.
GitLab	Settings → Merge requests → Merge checks	Turn on Pipelines must succeed, so a failed `Solemly / Review` status blocks the merge.
Bitbucket	Repository settings → Branch restrictions, or Merge checks	Require a minimum number of successful builds with no failed builds. Enforcing needs a Bitbucket Premium plan; other plans show warnings.

On GitHub, the check appears in the search once Solemly has reviewed at least one pull request in the repository.

Which findings block

Severity	Meaning
`CRITICAL`	Must be fixed: a serious bug, data loss or an exploitable security issue.
`HIGH`	Should be fixed before merging.
`MEDIUM`	Worth fixing.
`LOW`	Minor.
`NIT`	Style nitpicks. Dropped by default, so they don't reach your pull request.

CRITICAL and HIGH are the high-severity findings that can block a merge.

How the gate behaves

Fix and push. A new commit gets a new review. When it no longer finds high-severity issues, the check passes and the merge unblocks.
Skipped reviews never block. A commit skipped because of the repository's Review when setting, a paused pull request or plan limits gets a passing status.
Solemly fails open. If a review can't be completed, or the change is too large to review, the check doesn't block the merge, so an outage on Solemly's side never stops your team from shipping.
Acceptance is separate. Solemly / Acceptance fails when a pull request doesn't do everything its Jira ticket asks, whatever the blocking setting. It only blocks a merge if you make it a required check too.

When a finding is wrong

Reply @solemly ignore in the finding's thread. It isn't raised again on that pull request, and the next review re-runs without it.

Questions

What happens if Solemly can't complete a review?

The check doesn't block. Solemly fails open: a review that can't be completed, or a change too large to review, gets a status that lets the merge through, so an outage on Solemly's side never stops your team from shipping.

Can a wrong finding block a merge forever?

No. Reply @solemly ignore in the finding's thread and it isn't raised again on that pull request. The next review re-runs without it.

Can security findings block on their own?

Yes. With blocking on, a CRITICAL or HIGH security finding also fails Solemly / Security. Make that check required to gate merges on security findings alone.

Do skipped reviews block?

No. A commit skipped because of the repository's Review when setting, a paused pull request or plan limits gets a passing status.

In the docs

Try it on your next pull request

Free for 14 days with every feature, no credit card. Install it on a repository and the next pull request gets reviewed.

See pricing