Skip to content

☂️ Replace protobuf resource specs with Go structs (3.0) #18525

Open
11 of 11 issues completed
Open

Description

@Automaat

Problem

Twelve resource specs are still defined in hand-written .proto files. They keep a second code generator alive next to policy-gen, a protoc toolchain with four plugins, and a dual serialization path in the core resource model that branches on proto.Message for every read, write, hash and compare. Every policy added since the targetRef rework uses a plain Go struct, and the core model already supports both, so the proto path now exists only for these twelve types. While it stays, core resources cannot expose the same OpenAPI schema as policies.

One of the twelve is already dead in 3.0 and gets removed rather than converted. Nothing writes ServiceInsight, its endpoints answer 404, and the insight resyncer exists only to delete the rows older control planes left behind.

The SaaS global runs a v2 and a v3 deployment against one database, but a virtual control plane is owned by a single version and moves across once, so no rows are read and written by both versions at the same time. The requirement is that 3.0 reads what 2.14 wrote, with a working path back.

One constraint the code does not record: #18330 showed that a stored enum value the control plane cannot parse takes out the dataplane list, the overview, the global insight and the delete path for a whole mesh, and it was closed in favor of a pre-upgrade readiness report rather than fixed. Rollback has no readiness report. Once ServiceInsight is gone, Dataplane is the only converted spec that carries enums at all, so that is the one place the constraint has to hold.

On Kubernetes these seven resources store their spec as an opaque object, so the API server neither prunes nor validates what is written. Every resource already defined as a Go struct instead carries a fully typed schema, which prunes unknown fields and rejects invalid values on write. Converting a spec therefore changes its custom resource definition as well, and a stored object that the new schema rejects becomes impossible to update rather than merely unreadable, which is the same failure #18330 reported moved to the API server. Keep the spec opaque on Kubernetes while converting, and decide separately and later whether to tighten it.

Expected outcome

  • No .proto file in the repository defines a resource spec
  • A 3.0 control plane reads every resource a 2.14 control plane wrote, asserted against data captured from a running 2.14 cluster rather than hand-written fixtures
  • A 2.14 control plane still parses resources a 3.0 control plane wrote, so a rolled-back virtual control plane keeps serving
  • Dataplane gains no new enum value in 3.0, so no stored dataplane becomes unreadable on rollback
  • make generate runs without protoc and without the proto-driven resource generator
  • Core resources carry a real OpenAPI schema in the REST API specification, generated the same way policies already are
  • The custom resource definitions keep accepting and storing what they accept today, so no upgrade rewrites or rejects a stored object
  • Envoy xDS, the KDS transport and MADS keep working unchanged

Out of scope

  • The KDS wire format and MADS stay on protobuf. Both reuse Envoy discovery messages, and MADS is a contract with external Prometheus scrapers
  • The unused protoc-gen-validate wiring, which is dead independently of this work

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage/acceptedThe issue was reviewed and is complete enough to start working on it

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions