An AI rewrite that takes eight seconds feels broken if nothing appears until the end. Streaming fixes the wait: the first words arrive in under a second and the rest follow. The easy way to get streaming is to call the AI provider straight from the browser, and that is the way to avoid. This article explains why the request should pass through your server, and what the editor and the server each do.
Why the browser should not call the provider
- The key. A provider key in browser code can be read by anyone who opens the page. They can then spend your budget.
- The limits. Only a server can enforce a per-user rate limit or a monthly budget. A limit in browser code is a suggestion.
- The record. A server can log who asked for what and what it cost. That log is what you need when a bill is higher than expected.
- The policy. A server can refuse a request, for example for a document the user may not read, before any text leaves your network.
The shape of a streamed request
The browser sends one request to an endpoint on your server. Your server calls the provider with its own key, and passes each piece of the answer back as it arrives. Server-Sent Events are the usual transport: one HTTP response that stays open, with a small frame for each piece of text, then a final frame with the complete structured result, and a closing frame.
In the editor, the AI Toolkit has a helper for the browser side. It reads the stream, calls you back for each piece, and can be cancelled:
var stream = editor.aiToolkit.streamRequest({
url: "/richtextbox/ai/stream",
body: { mode: "rewrite", selectionText: "..." },
onDelta: function (chunk, full) { /* show the text so far */ },
onResponse: function (payload) { /* the final structured result */ },
onDone: function (fullText) { },
onError: function (err) { }
});
// stream.abort() cancels mid-responseThe helper uses fetch with a readable stream, and falls back to a normal request in a browser that cannot stream. The exact frame format, and the ASP.NET Core endpoint that produces it, are in the AI providers and streaming documentation.
What the server endpoint must do
- Authenticate the user with your normal session, exactly as for any other write request.
- Limit the input. Cap the size of the text you forward, and cap the answer length. Both are what you pay for.
- Apply a rate limit per user, not only per IP address.
- Stop when the user stops. When the browser cancels, cancel the provider call too. Otherwise you pay for an answer nobody reads.
- Turn off buffering. A proxy or a compression layer that buffers the response delivers the whole answer at once, which defeats streaming.
- Record usage for each request: who, which mode, and the token counts the provider reports.
Do not write streamed text straight into the document
Show the streamed text in a preview, and change the document only when the user accepts it. A stream can fail halfway, and half a rewritten paragraph is worse than none. Treat the final text like any other untrusted input: sanitize it before it becomes part of the stored document. The stored XSS article covers the checks.
A short test list
- Open the page’s network panel and confirm no request goes to the AI provider from the browser.
- Cancel a long answer and confirm the provider call stops on the server.
- Send requests faster than your limit allows and confirm the server refuses them.
- Disconnect the network mid-answer and confirm the document is unchanged.
- Check the response through your production proxy, not only on your own machine: buffering often appears only there.
For how the editor keeps provider keys on your server, see the bring-your-own-key AI editor page, and for the costs, see AI usage and pricing.
RichTextEditor is a perpetual-licence JavaScript editor — one purchase, self-hosted, no metered editor loads. Download the evaluation or see how it compares.