Skip to content

docs: propose a public remote cache on Cloudflare - #716

Draft
fengmk2 wants to merge 7 commits into
mainfrom
rfc-cloudflare-remote-cache
Draft

fengmk2 wants to merge 7 commits into
mainfrom
rfc-cloudflare-remote-cache

Conversation

@fengmk2

@fengmk2 fengmk2 commented Sep 9, 2026

Copy link
Copy Markdown
Member

Motivation

Open-source contributors need to reuse task results without credentials. Maintainers need a low-cost service that accepts uploads from trusted GitHub Actions jobs.

Propose a self-hosted remote cache for vp run on Cloudflare Workers, D1, and R2, based on #713. Reads are public. Writes use GitHub OIDC and require a push job on the registered repository's main branch. vp cache push publishes selected results explicitly.

The RFC includes setup and storage diagrams, operational limits, and Free/Paid capacity estimates. Measurements from four published Vite frontends support the cache-size assumptions.

fengmk2 and others added 4 commits September 7, 2026 10:56
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
@github-actions

github-actions Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  -0.48%  [ -6.71% ..  +6.69%]  overhead  +285.17%
dynamic/access             change  +0.69%  [-11.56% .. +41.48%]  overhead   +12.58%
dynamic/access-relative    change  +0.05%  [ -2.26% .. +13.55%]  overhead   +54.03%
dynamic/access-contended   change  +1.19%  [ -0.38% ..  +3.01%]  overhead    +8.09%
static/launch              change  -0.96%  [ -7.84% ..  +6.45%]  overhead  +685.51%
static/access              change  +0.22%  [ -6.72% ..  +7.58%]  overhead  +669.97%
static/access-relative     change  -0.16%  [ -3.64% ..  +4.08%]  overhead  +959.75%
static/access-contended    change  -0.43%  [ -3.23% ..  +1.05%]  overhead +2402.72%

macos

dynamic/launch             change  -0.15%  [ -2.78% ..  +3.00%]  overhead  +213.45%
dynamic/access             change  +0.34%  [ -3.64% ..  +4.23%]  overhead   +10.63%
dynamic/access-relative    change  +0.00%  [ -1.88% ..  +1.64%]  overhead  +278.54%
dynamic/access-contended   change  +0.09%  [ -4.03% ..  +2.40%]  overhead    +3.40%

windows

dynamic/launch             change  -1.06%  [ -7.20% ..  +3.20%]  overhead   +24.25%
dynamic/access             change  +0.35%  [ -2.29% ..  +3.88%]  overhead    +1.37%
dynamic/access-relative    change  -0.11%  [-16.45% .. +12.34%]  overhead    +3.07%
dynamic/access-contended   change  -1.58%  [-13.26% .. +10.70%]  overhead   +16.71%

wan9chi and others added 2 commits September 10, 2026 21:46
Co-authored-by: GPT-6 <codex@openai.com>
Co-authored-by: GPT-6 <codex@openai.com>
fengmk2 added a commit that referenced this pull request Sep 12, 2026
Copy RFC #716 from c201f8e and adjust relative paths.

Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 <codex@openai.com>
fengmk2 added a commit that referenced this pull request Sep 14, 2026
Copy RFC #716 from c201f8e and adjust relative paths.

Co-authored-by: GPT-6 Codex <codex@openai.com>
wan9chi pushed a commit that referenced this pull request Sep 27, 2026
Copy RFC #716 from c201f8e and adjust relative paths.

Co-authored-by: GPT-6 Codex <codex@openai.com>
wan9chi pushed a commit that referenced this pull request Sep 27, 2026
Copy RFC #716 from c201f8e and adjust relative paths.

Co-authored-by: GPT-6 Codex <codex@openai.com>
wan9chi pushed a commit that referenced this pull request Sep 27, 2026
Copy RFC #716 from c201f8e and adjust relative paths.

Co-authored-by: GPT-6 Codex <codex@openai.com>
wan9chi pushed a commit that referenced this pull request Sep 28, 2026
Copy RFC #716 from c201f8e and adjust relative paths.

Co-authored-by: GPT-6 Codex <codex@openai.com>
wan9chi pushed a commit that referenced this pull request Oct 5, 2026
Copy RFC #716 from c201f8e and adjust relative paths.

Co-authored-by: GPT-6 Codex <codex@openai.com>
wan9chi added a commit that referenced this pull request Oct 5, 2026
## Motivation

The self-hosted remote cache server in #718, designed in #716, serves
reads to anyone but accepts a store only with a GitHub Actions OIDC
token. Other servers will need other credentials. For example, a private
cache behind Cloudflare Access could need headers on every request. This
adds one general hook so each kind of credentials is a separate
implementation, and the client doesn't need to know about any of them.

## Changes

- `vt_remote_cache::auth::Auth` supplies the headers for each request,
given its operation: fetch, download, or store. It can use the client's
HTTP client to get credentials, such as a token, and that client doesn't
follow redirects. If it fails, the request isn't sent, and the operation
fails with the new `Error::Auth` ("failed to authenticate").
- `Client::new(endpoint, auth)` takes the auth. `Anonymous` adds no
headers.
- Planning resolves how requests authenticate into `remote_cache.auth`,
next to the access mode and endpoint. The result holds everything needed
to build the credentials, so nothing reads envs after planning. Choosing
the auth from `cache.remote` config or envs later only changes this
step. For now, the only kind is `anonymous`.
- `vt` turns the resolved auth into a `vt_remote_cache` auth with
`build_auth`, a single `match`, and caches clients by endpoint and auth.

Requests don't change. Plan snapshots gain `"auth": {"kind":
"anonymous"}`. The next PR in this stack adds GitHub Actions OIDC as the
first auth with credentials.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
wan9chi added a commit that referenced this pull request Oct 5, 2026
…DC (#798)

## Motivation

The self-hosted remote cache server in #718, designed in #716, accepts
uploads only with a GitHub Actions OIDC token. The token's audience must
be the namespace endpoint, and it must come from a push job on the main
branch. `vp run` sends stores without credentials today, so that server
rejects every upload with 401.

## Changes

- Planning resolves `remote_cache.auth` to `github-oidc` when
`ACTIONS_ID_TOKEN_REQUEST_URL` and `ACTIONS_ID_TOKEN_REQUEST_TOKEN` are
set in the envs visible at the `vp run` level. That happens in jobs with
`permissions: id-token: write`; otherwise the auth stays `anonymous`.
- It holds the request URL, the request token, and the audience, which
is the endpoint without a trailing slash.
- The request token is a `Secret`, which debug output and serialized
plans redact.
- `build_auth` turns `github-oidc` into
`vt_remote_cache::auth::GithubOidc`, which adds `Authorization: Bearer
<token>` to stores only. Fetches and downloads stay anonymous.
  - It requests a token when the first store needs one.
- Later stores reuse the token until two minutes before its `exp`.
Cloudflare receives a store's whole body before the Worker checks the
token, so the token has to outlast the upload. A token without `exp` is
a malformed response.
  - Concurrent stores wait for the same request.
- A failed request is remembered, so later stores fail right away
without making more requests. Each task with a failed upload shows the
existing "Not uploaded to the remote cache" warning.
  - Neither token appears in debug output or errors.
- Its state is a single enum: ready with a request and an optional
cached token, or failed. A token can't stay cached after a failure.
- Tasks still receive the two env vars as untracked envs, as in #691, so
npm trusted publishing through `vp run` keeps working.

Stacked on #797, which adds the `Auth` hook and the resolved auth
config.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
wan9chi pushed a commit that referenced this pull request Oct 5, 2026
Copy RFC #716 from c201f8e and adjust relative paths.

Co-authored-by: GPT-6 Codex <codex@openai.com>
wan9chi pushed a commit that referenced this pull request Oct 5, 2026
Copy RFC #716 from c201f8e and adjust relative paths.

Co-authored-by: GPT-6 Codex <codex@openai.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants