Skip to content

Harden control plane manifests: securityContext, non-root images, narrower RBAC #2936

Open

Description

@mikedickey

We run Substrate on GKE and install it with Kustomize over manifests/ate-install/, reconciled by Flux. We hardened the control plane manifests before putting it in a shared cluster, and would like to send the changes upstream so every install method in the extensible install proposal inherits them.

We have run all of this on v0.5.0. I checked each item against main at 90de075, and each is still missing there. I'd send them as the three PRs below, each verified on kind first.

1. securityContext and non-root images

Several control plane containers set no securityContext: ate-api-server, ate-controller, both containers in atenet-router, and ext-proc in atenet-egress. ko builds every image on gcr.io/distroless/static-debian13:latest, which runs as root.

Proposed:

  • Switch defaultBaseImage in .ko.yaml to static-debian13:nonroot, pinned by digest. atelet keeps its base-debian13 override.
  • Set allowPrivilegeEscalation: false and capabilities.drop: [ALL] on every control plane container.
  • Set runAsNonRoot: true on ate-api-server, ate-controller, atenet-router, podcertificate-controller, ate-idp-server, and the objectstore-plugin sidecar in ate-api-server. The nonroot base runs as UID 65532, so these need no runAsUser.

Exceptions:

  • ate-api-server keeps NET_BIND_SERVICE because it binds :443. podcertificate-controller already does the same.
  • The envoy container in atenet-router also needs runAsUser: 65532, because the Envoy base image sets no user. atenet-egress already does this.
  • ate-controller can also take readOnlyRootFilesystem: true. It writes nothing to disk and runs no webhook server.
  • atelet and its objectstore-plugin sidecar stay root and get only allowPrivilegeEscalation: false.

The base image change and runAsNonRoot have to go in the same PR. With today's root base, runAsNonRoot fails every pod with CreateContainerConfigError. ate-idp-server is new since v0.5.0, so we have not run it non-root yet. I'll check it on kind before opening the PR.

2. ate-controller reads every Secret in the cluster

The marker in egressmitmtrust_controller.go generates a ClusterRole rule with get/list/watch on all Secrets. The controller reads exactly one Secret, egress-mitm-ca-pool, and managerOptions in main.go already field-selects the cache to it.

We moved the rule to the namespaced Role and added resourceNames: [egress-mitm-ca-pool]. list and watch still work with resourceNames here, because the apiserver treats an exact metadata.name field selector as the request name.

Upstream this would be a marker change (namespace= and resourceNames=) plus regenerating role.yaml.

3. podcertificate-controller runs as the default ServiceAccount

pod-certificate-controller.yaml sets serviceAccountName: default, and both podcert bindings name default. Any pod created in podcertificate-controller-system without a serviceAccountName gets sign and attest permissions for every Substrate signer.

Proposed: add a podcertificate-controller ServiceAccount, and use it on the Deployment and in both bindings.

Related: the controller has no readiness probe. If its RBAC is wrong, the pod stays Ready and other pods' podCertificate volumes never fill. I can open a separate issue for that if it's wanted.

Activity

  1. stshrive commented on Oct 20, 2026

    @stshrive
    Collaborator

    At first glance, I don't see any issue with using nonroot versions of the images or limiting the secrets read for the egress-mitm-ca-pool, however I think there are some API-level considerations that need to be reviewed for some of the suggested changes.

    FYI Julian Gutierrez Oschmann (@juli4n), Lior Lieberman (@LiorLieberman), Benjamin Elder (@BenTheElder), Taahir Ahmed (@ahmedtd), Greg Castle (@destijl)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/securitySecurity related issue/prkind/featureAn enhancement / feature request or implementation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions