methodology / v1
One rubric behind every number on the site: four 0–100 axes, assigned by hand from real use, published under a version so its meaning can never silently change.
applied across the leaderboard, reviews and comparisons
Each reviewed tool carries four scores on a 0–100 scale. The first three are measured dimensions; the fourth is an editorial verdict that deliberately is not their average.
score_speed · 0–100Raw responsiveness in day-to-day use: completion and generation latency, indexing time on a real mid-size repository, cold-start time, and how the tool behaves under a slow connection. Measured hands-on during actual work sessions, not synthetic benchmarks.
score_privacy · 0–100What leaves your machine and under which terms: training on your code, prompt retention windows, telemetry defaults and opt-outs, self-hosting options, and whether a zero-retention or enterprise mode exists. Claims are checked against the vendor’s own privacy policy and documentation, not marketing pages.
score_dev_experience · 0–100How it feels to work with: setup friction, editor and CLI integration quality, configurability, documentation depth, failure modes, and how well it fits into an existing workflow rather than demanding a new one.
score_overall · 0–100The editor’s holistic verdict on a 0–100 scale. It weighs the three dimensions above together with pricing fairness and project health (maintenance activity, license, longevity risk). It is a considered editorial judgment, not an automated average — two tools with identical sub-scores can earn different overall scores when their pricing or trajectory differs.
Every tool is installed and used on real production projects before it earns a score. Ratings come from first-hand use — never from a marketing page or a press release.
Pricing, licensing, and feature claims are cross-checked against official documentation and live repository activity. If a claim cannot be verified, it does not get published.
Tool pages carry a visible "last verified" date, and stale entries are re-checked on a rolling cycle. Discontinued tools move to the Graveyard instead of quietly rotting in the rankings.
Rankings come from the rubric and nothing else. There is no paid placement, no affiliate weighting, and no way to buy a score.
Testing is done by a named human — meet the editor. Nothing on the site is bulk-published by automation.
This page documents v1 of the rubric. If an axis definition, the scale, or the weighting philosophy ever changes, the rubric gets a new version number and this page keeps a changelog — published scores always say which version produced them. Scores may still be revisited under the same version when a re-verification changes the underlying facts (pricing, licensing, maintenance status).
All scores are published as machine-readable open data in reviews.json under CC-BY 4.0. When you cite a number, reference it as “aicoolies Score v1”and link this page or the tool's review — attribution is the only thing the license asks for.