Autonomous Issue-to-PR Verification Stack

Define autonomous work as a bounded contract

An issue-to-PR agent should be autonomous inside a narrow contract, not authorized to reinterpret the product or merge its own work. The issue must state the expected behavior, allowed paths, acceptance commands, external systems in scope, and conditions that require escalation. OpenHands provides a general implementation environment, SWE-Agent specializes in turning software issues into repository changes, and Agent Orchestrator isolates and supervises their sessions. reviewdog and GitHub Actions make verification visible on the pull request, while a human retains final review and merge authority.

This stack works best for reproducible bugs, dependency maintenance, test additions, small refactors, and features with stable interfaces. It is risky for ambiguous product design, production incident response, secret-bearing infrastructure, irreversible data changes, or work whose success depends mainly on subjective judgment. The first decision is therefore eligibility. A triage rule should score issue clarity, repository test coverage, blast radius, data sensitivity, and rollback cost. Issues that fail the threshold stay with a human owner; autonomy is a routing option, not a default label applied to the backlog.

Architecture and agent roles

Agent Orchestrator creates a worktree and branch per approved issue, pins the base commit, and dispatches a worker with the bounded task card. OpenHands is the general lane for repository exploration and implementation across varied stacks. SWE-Agent is the specialist lane for issue-resolution experiments or an independent attempt when the problem is well described by failing tests. Teams may select one as the implementer and the other as verifier rather than allowing both to modify the same files. Their outputs converge only through reviewed Git commits and explicit handoffs.

reviewdog reports supported static-analysis and lint diagnostics directly on the PR, giving maintainers a durable view of actionable failures. GitHub Actions runs the acceptance commands, integration checks, build, and repository policies in a clean environment. The human issue owner verifies that the change actually answers the requirement and that the tests exercise the relevant behavior. No component should convert a green local run into merge permission. The end-to-end state machine is eligible issue, isolated attempt, local evidence, draft PR, automated checks, independent review, human approval, and merge or rollback.

Prepare the issue and environment

Rewrite the issue into a machine-verifiable contract before dispatch. Include reproduction steps or a failing test, expected output, non-goals, allowed dependencies, target files when known, and the exact commands that establish completion. Add a time and cost budget plus a maximum retry count. Provision the worktree with least-privilege credentials and isolated test resources; production access should be absent. When a task needs a database or external service, use disposable fixtures and a namespaced environment so concurrent runs cannot change one another's evidence.

Pin toolchain and dependency versions, record the starting commit, and make environment setup deterministic. The agent may inspect nearby code and documentation, but an unexpected need to change schemas, authentication, billing, deployment, or public APIs should trigger escalation. Store the execution summary with the branch: commands run, test results, important decisions, files changed, and known gaps. If the issue cannot be reproduced, the worker should report the evidence and stop instead of inventing a plausible fix. That behavior protects the backlog from confident but ungrounded pull requests.

Implement, test, and open the PR

The implementation agent should first reproduce the failure, identify the smallest responsible surface, and add or refine a test that fails for the intended reason. It then makes a focused change and runs the acceptance suite. Large opportunistic refactors belong in separate issues. The agent opens a draft PR with the issue link, before-and-after behavior, validation output, and explicit limitations. If OpenHands implemented the change, SWE-Agent can be assigned a read-only verification task, or the roles can be reversed for issue types where the specialist has the stronger path.

GitHub Actions repeats the tests in a clean runner and adds broader checks that would be expensive in every local attempt. reviewdog attaches configured diagnostics without hiding their originating tool or severity. Require evidence for generated migrations, snapshots, security-sensitive paths, and UI behavior as appropriate. A failed check returns the task to the owning lane once; repeated failures escalate with logs and preserved state. The agent must not disable tests, weaken assertions, or edit CI policy simply to obtain green status unless that policy change is explicitly part of the approved issue.

Verify safety and quality

Independent review asks whether the patch fixes the issue, preserves surrounding invariants, and introduces adequate regression coverage. It also checks scope: unexpected dependency changes, generated files, secrets, broad formatting rewrites, or edits outside the allowed paths are stop signals. For data migrations, permissions, payments, or destructive operations, require a domain maintainer regardless of automated confidence. Review comments should point to concrete code and requested behavior. Model-generated reviews may help discovery, but only reproducible findings should block or reshape the patch.

Use staged merge rules: required CI checks, required human approvals, and branch protection. For high-risk repositories, run the change in a preview or sandbox and compare live behavior against the issue acceptance criteria. Record the merge commit and rollback handle. After deployment, monitor the relevant error or performance signal for a bounded period when the issue justifies it. Autonomy ends at the approved repository scope; production rollout, user communication, and incident handling remain separate authorities unless they receive their own explicit workflow and approval.

Failure modes, economics, and rollout

Common failures include an underspecified issue, a test that asserts the agent's implementation rather than the desired behavior, environment drift, repeated retry loops, and review queues that grow faster than humans can clear them. Control these with an eligibility gate, test-first evidence, pinned environments, retry ceilings, and concurrency limits. Preserve failed branches for diagnosis but expire them on a schedule. Track costs by accepted PR rather than attempted run, including model usage, runner minutes, sandbox infrastructure, and reviewer time. Cheap unsuccessful attempts can still be expensive when they consume review attention.

Pilot the workflow on a labeled set of low-risk issues and compare it with the team's normal cycle time and defect rate. Measure eligible-to-draft time, first-pass CI success, human changes requested, abandoned attempts, escaped regressions, and total cost per merged PR. Expand only when the evidence shows reliable gains. Keep OpenHands, SWE-Agent, and the orchestration layer replaceable behind the same task and evidence contracts. The durable asset is not a particular agent; it is the bounded path from issue to independently verified pull request, with a clear human decision at the end.