What This Stack Does
This stack is an OSS-first evaluation workflow for newer terminal coding agents. Pi, Crush, and OpenCode provide open or source-available terminal experiences, while Amp is a hosted commercial lane and must not be described as open source. tmux places all four tools beside the same briefing, test, and review panes so a team can compare interaction style, provider flexibility, patch quality, and operational boundaries without pretending the products share one license or hosting model.
The stack is for early adopters who want evidence before standardizing on a terminal agent. Each tool should receive the same repository snapshot and acceptance criteria in an isolated Git worktree. The outcome is a decision record: which tool handled exploration, implementation, review, and recovery best for the team’s actual codebase. It is not a universal benchmark, and it should not rely on public star counts or marketing claims as a substitute for controlled tasks.
Understand the Four Agent Lanes
Pi is the minimalist lane: use it when you want a compact coding loop and a small set of explicit actions. Crush emphasizes a terminal user interface and can be useful when developers want to watch and steer the agent continuously. OpenCode is the provider-flexible lane and can test how much model choice, local integration, or review workflow matters. Amp is the commercial hosted lane, useful as a contrast to the open-tool paths but subject to its own account, data, and usage policies.
Avoid assigning every lane the same personality. One lane can map the repository, two can implement independently, and one can review the candidates. Rotate roles on a second task if you want a fairer product evaluation. Record the selected provider or model, enabled tools, repository permissions, and whether the agent ran interactively or autonomously. Differences in configuration often explain more than the client name, so preserve them with the result.
Build the tmux Evaluation Bench
Create a tmux session with a briefing window, one window per agent, a test window, and a decision window. Build four worktrees from a single commit and use consistent branch names. Place the task contract in a repository file so every lane reads the same requirements. Keep authentication separate for each tool and never paste credentials into shared tmux history. If a provider supports organization controls, configure those controls before the evaluation rather than after code has been transmitted.
Run a short calibration task first. Confirm that every agent can read the intended paths, edit only its worktree, and execute the approved test command. Then run the real task with a fixed time or spend budget. Do not feed one agent another agent’s patch during the independent phase. Once all lanes stop, export concise diffs and test results to the decision pane and compare them against the original contract.
Evaluate Quality and Developer Experience
Score outcomes on correctness, scope discipline, maintainability, recovery from failed tests, and the amount of human steering required. Also note interaction cost: whether the terminal interface makes tool calls legible, whether permission prompts are understandable, and whether the agent explains risky actions before running them. A polished interface does not excuse an incorrect patch, but poor observability can make a strong model unsafe to operate in a real repository.
Use at least two representative tasks. A mechanical refactor favors fast search and editing, while an ambiguous bug exposes planning and debugging behavior. Include a task that should be declined or escalated, such as a production secret request or a destructive migration without backup. The strongest operational agent is not only the one that writes code; it is the one that respects the stop rule and leaves an auditable, reviewable state.
Licensing, Data, and Cost Controls
Treat the stack name as an OSS discovery angle rather than a claim that every component is open source. Verify the current license and distribution terms for each client before organizational adoption, especially for source-available licenses that change over time. Amp remains the explicitly hosted commercial comparison lane. For every tool, review where prompts, code, telemetry, and error logs are sent, and whether the chosen model provider introduces another data processor.
Bring-your-own-key clients are not costless. Provider charges, retries, context size, and developer review time all matter. Set a per-task budget and capture the account or provider path used by each lane. If one client silently switches models or uses a bundled credit, record it. Do not compare a small local model with a premium hosted model and attribute the entire result to the terminal interface. Separate client ergonomics from model capability in the final decision.
When to Use It and the Bottom Line
Use this stack when a team is choosing a terminal agent, when existing IDE assistants feel too opaque, or when provider portability is a requirement. Skip it when policy already mandates one provider, when the repository cannot be copied into isolated worktrees, or when there is no deterministic test surface. Early tools may change quickly, so repeat the calibration task after material releases instead of assuming a six-month-old result remains valid.
The repeatable approach is to make tmux the visible control plane, isolate every candidate, keep licenses and data paths explicit, and evaluate the same engineering contract across lanes. Pi, Crush, OpenCode, and Amp represent different product choices rather than interchangeable wrappers. A useful decision names the role and boundary for each selected tool; it does not crown a permanent winner from one demo.