Skip to content

feat: send first-party usage headers to the Kosli host - #1310

Merged
tooky merged 3 commits into
mainfrom
analytics-headers
Oct 28, 2026
Merged

feat: send first-party usage headers to the Kosli host#1310
tooky merged 3 commits into
mainfrom
analytics-headers

Conversation

@tooky

@tooky tooky commented Oct 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Requested by Steve Tooke · Slack thread

Before: the Kosli server sees only User-Agent: Kosli/<version> on a CLI request, so it cannot tell which command made it, which flags shaped it, which CI system ran it, or which requests came from the same run.

After: every request the CLI sends to the configured Kosli host also carries five first-party headers describing the command. Requests to any other host (registries, Bitbucket, GCP, presigned S3 uploads) carry none of them, and the User-Agent is unchanged.

Header Example Source
X-Kosli-Client cli/2.12.3 same version as the User-Agent
X-Kosli-Operation attest jira cobra command path without the leading kosli
X-Kosli-Invocation-Id 0b6c2b5e-... random UUID v4 per process, shared by every request of the run
X-Kosli-Cli-Flags commit,flow,name,trail sorted names of flags set on the command line, in KOSLI_* env vars or in the config file; capped at 1 KB, cut on a flag boundary
X-Kosli-Ci github WhichCI() mapped to github, gitlab, bitbucket, azure-devops, circleci, teamcity, codebuild, jenkins; otherwise other-ci when CI is true, else none

Why

Kosli's product analytics (Segment to Mixpanel, grouped by org) only see write events today, and can only tell "CLI" from "other" by the User-Agent. These headers let the server attribute each API call to a client, its version, the operation and one run or session, so we learn how customers actually use Kosli with the data they put in. A follow-up server change reads the headers and adds them to the events. Until then the server ignores them, and they are all optional, so older clients keep working and the server falls back to the User-Agent.

What the CLI headers will let us see:

  • Which commands each org runs, for example how often kosli assert artifact is used compared with kosli attest junit.
  • kosli assert snapshot told apart from kosli get snapshot, which call the same endpoint.
  • Which flags are in use, by name only, so we know which options matter and which nobody sets.
  • Which CI systems pipelines run Kosli in (GitHub, GitLab, Jenkins and so on).
  • All calls from one CLI run grouped by the invocation id, for example how many API calls a single kosli snapshot k8s makes.

Names only, never flag values or arguments, and the headers go only to the configured Kosli host.

Privacy

  • Flag names only, never flag values or positional arguments. Many flags carry credentials (--api-token, --github-token, --jira-api-token, --registry-password, ...).
  • The connection flags api-token, host, org, config-file and http-proxy are left out of the flag list entirely.
  • CI is the vendor name only, no repo, run id or URL.
  • Headers go only to the configured Kosli host: the request host and effective port must match --host. The Kosli host already receives the request, so nothing new leaves the machine for a third party.

Compatibility

The server ignores unknown headers until it starts reading them and will treat each one as optional, falling back to the User-Agent, so older CLIs keep working. Related: kosli-dev/server#7728.

Where it is wired

  • cmd/kosli/usageHeaders.go builds the header values from the cobra command.
  • The root PersistentPreRunE (cmd/kosli/root.go) hands them to the client once, after initialize() has applied env and config values to the flags.
  • requests.Client.SetUsageHeaders (internal/requests/requests.go) stores them, and usageHeaderTransport, a RoundTripper wrapped around the retry client's inner transport, applies them on every hop: added when the hop goes to the Kosli host, removed when it does not. That covers redirects from the Kosli host to another host, which http.Client would otherwise follow with the headers copied. Dry-run sends nothing.

Notes for review

  • Env and config-file values count as "in effect" because bindFlags sets them with Flags().Set, which marks them changed, so Flags().Visit sees them. CI-derived defaults such as --commit falling back to GITHUB_SHA are defaults, not set flags, so they are not listed.
  • In multi-host mode (comma-separated KOSLI_HOST) each host gets the headers with the same invocation id, since the id is per process.
  • github.com/google/uuid moves from an indirect to a direct dependency.

Related PRs

The header names are shared across clients: X-Kosli-Client, X-Kosli-Operation and X-Kosli-Invocation-Id. The CLI also sends X-Kosli-Cli-Flags and X-Kosli-Ci.

Checklist

  • Docs are autogenerated from CLI help: no help text changes
  • Helm chart (charts/k8s-reporter/) updated, if needed: not needed
  • Terraform provider and related changes updated, if needed: not needed (separate PR for the Terraform provider if wanted)

https://claude.ai/code/session_01N9LH8PXUvMmSKYK9Mr65we

Generated by Claude Code

@claude
feat: send first-party usage headers to the Kosli host
dd2d606 
The server only sees a User-Agent of Kosli/<version>, so it cannot tell
which CLI command made a request, which flags shaped it, which CI system
ran it, or which requests belong to the same run. Product usage
instrumentation needs those answers, so Steve Tooke asked for the clients
to describe themselves in first-party headers.

Every request to the configured Kosli host now carries X-Kosli-Client,
X-Kosli-Operation, X-Kosli-Invocation-Id, X-Kosli-Cli-Flags and
X-Kosli-Ci. They hold flag names only, never values or positional
arguments, because many flags carry credentials, and the connection flags
(api-token, host, org, config-file, http-proxy) are left out of the flag
list. The same HTTP client calls registries, VCS providers and storage
buckets, so the headers are only added when the request host matches the
Kosli host.

The User-Agent is unchanged. The server ignores these headers until it
reads them and will treat each one as optional, so older CLIs keep
working.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N9LH8PXUvMmSKYK9Mr65we
@github-actions github-actions Bot added go Pull requests that update go code feat labels Oct 28, 2026
@claude

claude Bot commented Oct 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude finished @tooky's task in 47s —— View job

Follow-up review of PR #1310

  • Fetch existing review comments and previous findings
  • Diff BEFORE (7ae43cf)..AFTER (4e03cc5) to scope the review
  • Evaluate status of each previous finding touched by new commits
  • Check for new bugs/regressions in the changed lines
  • Close resolved inline threads
  • Post summary comment

Scope: the diff between 7ae43cf and 4e03cc5 is a single one-line change in cmd/kosli/usageHeaders_test.go (adding require.NotEmpty(t, jiraHeaders, ...) before the Jira-header loop).

Previous findings:

  • Redirect-leak finding (internal/requests/requests.go) — not touched by this commit; already resolved in 7ae43cf.
  • Vacuous-Jira-assertion finding (cmd/kosli/usageHeaders_test.go:96) — resolved, exactly as suggested. Closed the inline thread.
  • Silent 1 KB flag-list cut (cmd/kosli/usageHeaders.go:73) — not touched by this commit; author deliberately left it as-is (non-blocking nit).

No new bugs/regressions found in the changed lines. Full summary posted as a PR comment.

claude[bot]
claude Bot reviewed Oct 28, 2026
View reviewed changes
Comment thread internal/requests/requests.go Outdated
@claude
fix: keep usage headers off redirects to other hosts
7ae43cf 
The usage headers were added once in Do, after checking the first
request's host. The retry client's inner http.Client follows redirects
and copies every non-auth header onto the next hop, so a 3xx from the
Kosli host to a storage bucket or other third party would have carried
the X-Kosli-* headers there, breaking the promise that only the Kosli
host receives them (review comment on #1310).

The headers are now applied by a RoundTripper wrapped around the retry
client's inner transport, which runs on every hop: it adds them when the
hop goes to the Kosli host and removes them when it does not. It sits
below the proxy configuration, so --http-proxy keeps working.

SetUsageHeaders also logs at debug level when --host has no host part,
so a user wondering why the headers are missing can see why.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N9LH8PXUvMmSKYK9Mr65we
@tooky Claude

tooky commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Redirect gap fixed in 7ae43cf (see the inline thread). On the minor points:

  • Host without a scheme: 7ae43cf adds a debug log line when --host has no host part and the headers are switched off.
  • Debug output: no change. The headers are now set per hop inside the transport, after the payload logging, and their values are fixed strings from code, so I left them out of the --debug output to keep it unchanged.
  • Multi-host: no change needed, as you confirmed.

Generated by Claude Code

@tooky
tooky marked this pull request as ready for review October 28, 2026 18:04
claude[bot]
claude Bot reviewed Oct 28, 2026
View reviewed changes
Comment thread cmd/kosli/usageHeaders_test.go
claude[bot]
claude Bot reviewed Oct 28, 2026
View reviewed changes
Comment thread cmd/kosli/usageHeaders.go
@claude
test: assert attest jira reaches the Jira server in header scoping test
4e03cc5 
The end-to-end usage headers test checks that no usage header is sent
to the Jira server by looping over the requests Jira received. If
attest jira ever stopped calling Jira (the issue-key regex no longer
matching, or the lookup being skipped), that loop would run zero times
and the "not sent to other hosts" check would pass without testing
anything. Requiring at least one Jira request keeps it meaningful.

Raised by the automated review on #1310.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N9LH8PXUvMmSKYK9Mr65we
claude[bot]
claude Bot reviewed Oct 28, 2026
View reviewed changes
Comment thread cmd/kosli/usageHeaders_test.go
_, err = uuid.Parse(invocationID)
require.NoError(t, err)

require.NotEmpty(t, jiraHeaders, "attest jira should have called the Jira server")

@claude claude Bot Oct 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved in 4e03cc5 — require.NotEmpty(t, jiraHeaders, ...) now guards the loop below, so the "not sent to other hosts" check can no longer pass vacuously.

@claude

claude Bot commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Follow-up review: commit 4e03cc5

Diff reviewed: 7ae43cfb..4e03cc53 (one commit, one line changed in cmd/kosli/usageHeaders_test.go).

Previous findings:

  • Redirects can leak headers off-host (internal/requests/requests.go) — not touched by this diff; already resolved in 7ae43cf per the prior round.
  • Jira-call assertion could pass vacuously (cmd/kosli/usageHeaders_test.go:96) — resolved. require.NotEmpty(t, jiraHeaders, ...) was added before the loop, exactly as suggested. Closed the inline thread.
  • Silent 1 KB flag-list cut (cmd/kosli/usageHeaders.go:73) — not touched by this diff; author declined the change (non-blocking nit), left as-is.

No new issues found in the changed lines; this commit is a one-line, low-risk test strengthening.

This was referenced Oct 28, 2026
Merged
Merged
@tooky
tooky merged commit 5d6c49e into main Oct 28, 2026
23 checks passed
@mbevc1
mbevc1 deleted the analytics-headers branch October 28, 2026 18:24
@BrewTestBot BrewTestBot mentioned this pull request Oct 28, 2026
Merged
dangrondahl pushed a commit to kosli-dev/mcp-server that referenced this pull request Oct 29, 2026
@tooky @claude
feat: send first-party analytics headers on Kosli API requests (#106)
1ad9e19 
<!-- claude-slack-attribution -->
_Requested by **Steve Tooke** · [Slack
thread](https://kosli-internal.slack.com/archives/D0ADDDBJGP8/p1793194642140959)_

**Before:** the Kosli server can tell a request came from the MCP server
(via the `User-Agent`), but not which tool or catalog action made it, or
which requests belong to the same MCP session.

**After:** every Kosli API request carries first-party analytics headers
naming the client, and requests from the execute tools also name the
tool, the action and the session, so product analytics can count MCP
usage per operation and per session.

| Header | Value | Sent on |
|---|---|---|
| `X-Kosli-Client` | `mcp/<version>` (from `src/version.ts`, the same
source as the `User-Agent`) | every Kosli API request |
| `X-Kosli-Operation` | `<tool> <action id>`, e.g. `execute_read_action
get_trail`, `execute_write_action create_control` | requests from
`execute_read_action` / `execute_write_action` |
| `X-Kosli-Invocation-Id` | UUID v4, generated once per `createServer`,
so one per MCP session | requests from `execute_read_action` /
`execute_write_action` |

`User-Agent: kosli-mcp-server/<version>` is unchanged. `search_actions`
searches the bundled catalog and makes no request, so it sends nothing.
The redirect download for evidence files goes to a pre-signed
object-store URL, not Kosli, so it keeps only the `User-Agent` as
before.

**Why:** Kosli's product analytics (Segment to Mixpanel, grouped by org)
only see write events today, and can only tell "CLI" from "other" by the
User-Agent. These headers let the server attribute each API call to a
client, its version, the operation and one run or session, so we learn
how customers actually use Kosli with the data they put in. A follow-up
server change reads the headers and adds them to the events. Until then
the server ignores them, and they are all optional, so older clients
keep working and the server falls back to the User-Agent.

What the MCP headers will let us see:

- Which catalog actions agents call, for example `execute_read_action
get_trail`.
- The mix of read and write actions agents make.
- Activity per MCP session, grouped by the invocation id.
- Which MCP server versions are in use. Today the server's User-Agent
matching counts versioned MCP traffic as "other"; the server follow-up
fixes that.

Tool and action names only, never arguments or param values, and the
headers go only to the Kosli API.

**Privacy:** only names go in these headers: the tool name and the
catalog action id, which is a public OpenAPI operationId. No tool
argument, param value, org, search query or prompt text is ever put in a
header. Tests send arguments containing a marker string through both
execute tools and assert it appears in no header other than
`Authorization`.

**Compatibility:** the server ignores these headers until it reads them,
and treats each as optional with a fallback to the `User-Agent`, so this
can ship first and older releases keep working. Related:
kosli-dev/server#7728 (server-side query events). There is no opt-out,
by decision.

**Changes**
- `src/client/kosli-client.ts`: `X-Kosli-Client` on every request;
`execute()` takes an optional `RequestOrigin` (`tool`, `invocationId`)
that adds the other two.
- `src/tools/execute-action.ts`: `ExecuteOptions.origin` passed through
to the client.
- `src/server.ts`: one `randomUUID()` per `createServer`; each execute
tool passes its own name.
- `CLAUDE.md`: documents the headers and the names-only rule next to the
`User-Agent` convention.
- Tests in `test/client/kosli-client.test.ts` and `test/server.test.ts`
(over the in-memory transport): header values, operation per tool, one
invocation id per session and a new one per server, no arguments in any
header, no request from `search_actions`.

`npm run build`, `npm run typecheck`, `npm test` (147 passed) and the
offline `npm run smoke` all pass locally.

**Related PRs**

- [kosli-dev/cli#1310](kosli-dev/cli#1310): the
same headers from the Kosli CLI, plus `X-Kosli-Cli-Flags` and
`X-Kosli-Ci`
-
[kosli-dev/terraform-provider-kosli#293](kosli-dev/terraform-provider-kosli#293):
the same headers from the Terraform provider
- [kosli-dev/server#7728](kosli-dev/server#7728)
(merged): adds query events to Segment and Mixpanel (Artifact Asserted,
Snapshot Queried, Search Performed, Trail Queried, Evaluation
Requested), grouped by org

The header names are shared across clients: `X-Kosli-Client`,
`X-Kosli-Operation` and `X-Kosli-Invocation-Id`. The CLI also sends
`X-Kosli-Cli-Flags` and `X-Kosli-Ci`.

---
_Generated by [Claude
Code](https://claude.ai/code/session_01N9LH8PXUvMmSKYK9Mr65we)_

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments
@mbevc1 mbevc1 mbevc1 approved these changes

Assignees

No one assigned

Labels

feat go Pull requests that update go code

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tooky @mbevc1 @claude