# Cross-editor benchmark — reproduce it yourself

We publish performance numbers comparing RichTextEditor to TinyMCE, CKEditor, Froala and Quill.
A vendor publishing its own benchmark is worth exactly as much as your ability to re-run it, so
this directory is the harness that produced them.

```bash
npm install     # also downloads the Chromium build Playwright drives
npm run bench
```

It prints a table and writes `results.json`. Nothing else is required — no account, no licence
key, no access to our source. Every editor is downloaded from its own public distribution.

## What it measures

| Column | Meaning |
|---|---|
| ready ms | Wall time from navigation start until the editor reports itself ready **and** its editing surface exists |
| transferred | Sum of the browser's own `transferSize` for every resource the page loaded |
| setHTML | Synchronous time to load a 17.5 KB structured document through the editor's public "set content" API |
| readHTML | Synchronous time to read it back through the public "get content" API |
| fidelity | Whether the document survives the round trip with its structure intact (1 h1, 100 h2, 10 tables, 20 ordered lists, 100 links) |

Five isolated Chromium contexts per editor. Nothing is shared between runs, so no editor warms
the cache for the next one. Medians are reported, not means — one scheduling hiccup should not
decide a ranking.

## Why this exists separately from our internal harness

The internal harness (`RTEJSWeb/_tests/performance-fidelity-comparison.js`) served the five
competitors from public CDNs but served RichTextEditor from a local ASP.NET development host.
Two consequences, and the second one matters more than the first:

1. **It could not be run by anyone outside this repository.** So "vendor-run benchmark" was the
   only honest description available, and a reader had no way to check it.
2. **The ready-to-edit column was comparing server stacks, not editors.** Our editor came off an
   ASP.NET dev host; everyone else came off a plain static server. The published artifact carries
   that caveat in prose, which is the correct thing to do about a flaw you have not fixed — but
   fixing it is better.

Here every editor, ours included, is loaded from its public published package through the same
ephemeral static host on `127.0.0.1`. Same server, same viewport, same fixture, same browser.

## What it does not prove

- **The commercial editors run in their free/GPL configurations.** TinyMCE PowerPaste and
  CKEditor's premium features are not exercised. If you are evaluating paid tiers, this is not
  your comparison — paste handling in particular would differ.
- **Ready-to-edit rewards deferring work.** An editor that signals ready and finishes
  initialising afterwards will look faster here than it feels to a user. We report what each
  editor says about itself, plus a check that the editing surface actually exists.
- **Tiptap is deliberately absent.** It is a headless library assembled from ESM modules rather
  than a drop-in bundle, so any number we produced would be measuring our assembly choices as
  much as the library. It stays in the internal harness, where that caveat can sit next to it.
- **One machine, one browser build, one fixture shape.** Absolute milliseconds are a property of
  the hardware. What should reproduce on yours is the *ordering* and the order-of-magnitude
  difference in transferred bytes.
- **We wrote it.** That is the point of shipping it rather than only its output.

## Pinned versions

Competitor versions are pinned in `VERSIONS` at the top of `benchmark.mjs`; our own version is
whatever `@richscripts/richtexteditor` resolves to in `package.json`. A floating version is the
usual reason a benchmark cannot be reproduced six months later, so bump them deliberately and
re-run everything when you do.

Every downloaded asset is cached under `.cache/` and its SHA-256 is recorded in `results.json`,
so you can confirm you measured the same bytes we did.

## If an editor fails to start

It is recorded in `results.json` under `inconclusive`, with the reason, and printed as
`INCONCLUSIVE`. It is never reported as a product failure and never silently dropped — a missing
row reads as "we did not bother", and an editor that would not boot in *our* harness is a fact
about our harness until proven otherwise.

## Changing the run count

```bash
BENCH_RUNS=11 npm run bench
```

More runs tighten the medians. Five is what the published table uses.