Repository navigation
Conversation
benchmarks/streaming/ measures how many browsers each server setup can stream to (Flask/gunicorn, FastAPI and Quart on uvicorn, 1 or 4 workers), with simulated browsers that follow the renderer's StreamClient. benchmarks/publish.py builds a static site from the renderer timings and the streaming sweep (embeddable chart pages, raw data with history, shields badges), and benchmarks-publish.yml pushes it to gh-pages from dev.
benchmarks/callbacks/ sweeps browser counts for plain callbacks over HTTP and over the websocket on Flask (HTTP only), FastAPI and Quart, 1 and 4 workers, recording round trip, throughput, errors and CPU. The server and client plumbing it shares with the streaming test moves to benchmarks/loadkit.py.
benchmarks/swarm/ drives real Chromium users (Playwright) against a deployed app: plain HTTP callbacks, websocket callbacks and streaming, timed in the page from click to DOM update. Agents run locally or on remote hosts over SSH in a Docker image, start together at a shared wall-clock time, and report mergeable histograms; steps stop when the server saturates or when the swarm stops delivering its load (failed agents or users, agents over 85% CPU, late starts).
The benchmark site gains a callbacks section: browsers served per setup coloured by transport, p95 round trip for 1 and 4 workers (websocket dashed), server CPU per 1,000 calls a second, the full table, badges and data with history. The publish workflow runs the callback sweep weekly and on demand, like the streaming one.
Contributor
Dash performance benchmarks
growth = late-third / early-third per-op time; ~1 is flat, a large value means the per-op cost scales with accumulated state. machine scale vs baseline: 0.56x - divided out of the baseline ratios so they compare like for like (the absolute warn/fail ceilings are left un-scaled); calibrated on |
Share the sweep loop between the streaming and callback runners (loadkit.sweep), split the callback client's and swarm agent's long loops into small steps, write the swarm report outside the event loop, dedupe the site builder's sections and literals, run the agent image as its unprivileged user, and pin the third-party setup-chrome action to a commit. The site output is byte-identical for the same inputs.
camdecoster
marked this pull request as draft
October 8, 2026 19:32
The playwright/python image ships the browsers but not the Python package, so the agent failed to import it.
The numbers live in benchmarks/results/ for the docs to render (report.json holds every chart as a Plotly figure), and a short table in README.md shows how many browsers each server setup keeps interactive. The workflow now runs weekly and opens a pull request with the update.
The weekly run commits a load test only when a setup's capacity changes step, and the renderer only when a median moves by over 25% and 5 ms. The pull request lists what moved.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Load tests for streaming and callbacks, a headless-browser swarm for large runs, and CI that commits the numbers to the repo, with a short table in the README.
Changes
benchmarks/streaming/: how many browsers each server setup can stream to (Flask/gunicorn, FastAPI and Quart on uvicorn, 1 or 4 workers). Simulated browsers follow the renderer'sStreamClientand the count climbs until p95 frame latency or errors give out.benchmarks/callbacks/: the same sweep for plain callbacks, HTTP vs websocket, by backend (Flask HTTP only; FastAPI and Quart both).--kind asyncand--work-msadd per-call work.benchmarks/loadkit.py: the plumbing both share. It runs the server and the clients on separate CPUs, flags points where the clients ran out of CPU, and makes the server import the checkout under test, not the installed dash.benchmarks/swarm/: real headless Chromium users (Playwright), many per machine and many machines, against a deployed app. Users click an HTTP callback, a websocket callback or a stream, timed in the page from click to DOM update, so machines only share a start time, not clocks.benchmarks/README.md.benchmarks/publish.py+.github/workflows/benchmarks-publish.yml: one place for the numbers, in this repo.benchmarks/results/and the README table only when the numbers moved significantly: a load test's capacity changes step, or a renderer median moves by over 25% and 5 ms. The pull request lists what moved.benchmarks.yml) is unchanged.benchmarks/results/report.jsonhas every chart as a Plotly figure plus the tables, for the docs to render withdcc.Graph. The raw numbers and their history sit next to it.GITHUB_TOKENstarts no checks, so close and reopen it before merging.Numbers
Callbacks, sync, 1 worker, 8-core laptop:
With 4 workers no setup saturated at 4000 browsers; the swarm is how to find those limits. The multi-worker Redis streaming numbers need #4054.
Tests