Repository navigation
Re-read the QEMU integration test work #31
Description
Headed for the OCP demo on light review. Listed worst-first by how much
it would cost to have got wrong. All of it is additive and test-only
except OpenPRoT#541, which changes
services/orchestrator/driver and is merged to ocp-global-demo-wip as
bd062c9.
Merged to ocp-global-demo-wip on 2026-10-08, in this order: the harness
seed as d5fcaf5, OpenPRoT#533 as 8ba0756,
OpenPRoT#536 as 6f11f2f, OpenPRoT#537 as ca7e35f and
OpenPRoT#538 as 81da269. The rest still sit on
cq-demo-integration.
Read first, these are not test directories:
- services/mctp: Add the loopback transport OpenPRoT/openprot#537 services/mctp/transport-loopback, merged as
ca7e35f. Lives beside the shipped MCTP code and will be read as a
library, not as a test fixture. About 150 lines. Check the queue
ownership: the router takes the sender by value, so the caller
holds the queue and lends it. Its lib.rs was split into queue.rs
and sender.rs before the merge, so the 150 lines are three files
now. - target/ast10x0/harness/qemu_runner.py and defs.bzl. Every QEMU
test in the tree runs through this file. About 40 lines. Check
that seeding from a file cannot truncate: oversize is an error,
not a silent cut. This was harness: Seed a CS image from a file OpenPRoT/openprot#532, which was merged to demo-wip and
taken back out; it went in again on 2026-10-08 as d5fcaf5,
underneath tools: Add the flash image builder OpenPRoT/openprot#533, with no PR of its own.
Read after, test-only:
-
tools: Add the flash image builder OpenPRoT/openprot#533 tools/flash_image, merged as 8ba0756. Host
build tool. The layout goes through the real
Region/Slot/ImageLayout constructors on purpose, so a test image
cannot describe a layout the orchestrator would refuse. -
services/test: Add the mock BMC boot model OpenPRoT/openprot#536 services/test/mock-bmc, merged as 6f11f2f.
The boot model. Two behaviours only: a late boot is Boots with a
long delay, so lateness is a property of the deadline rather than
of the device. -
target/ast10x0: Add the mock BMC integration test OpenPRoT/openprot#538 target/ast10x0/tests/integration/mock_bmc,
merged as 81da269. The first scenario. The negative case is
automated now, as mock_bmc/device_hangs. -
target/ast10x0/tests/integration/pldm_update. A full DSP0267
update between an agent and a firmware device, three apps in one
image. No PR yet; it is commit 331321c on cq-demo-integration.
The firmware device stages into RAM and the agent's bytes are
checked against a known pattern, so the check to make is whether
anything here could pass without the whole sequence running. -
The boot scenario's RoT app now calls bring_up, so the state
machine, the platform driver, the device table and CheckpointWalk
are the shipped code rather than a hand-rolled sequence. No PR
yet; it is commit ef645ea on cq-demo-integration. Read the board
stubs: the verifier authenticates everything, and that is a
decision about the demo, not an oversight. -
The PLDM scenario's firmware device now reports to a real
orchestrator in a fourth app and waits for its answer. No PR yet;
it is commit 9bb3b7e on cq-demo-integration. Two gaps are in the
commit message on purpose: the driver's Updatable stages nothing,
and a refusal is recorded rather than enforced. Check that the
three-message protocol cannot let a step run before its answer. -
Wiring the two test boards to the verifier seam
driver: Wire IncrementalVerifier into the update pump OpenPRoT/openprot#541 added is 980d4de. It leaves two verdicts on
the same candidate, the device's and the RoT's, and the update
needs both. -
The PLDM update test failed about six runs in ten before dc7cc19.
The agent could reach the bus before the device claimed its
endpoint id, so discovery is retried now. Twenty consecutive suite
runs pass. Worth a look at whether the retry hides anything else. -
target/ast10x0/tests/integration/README.md says how to run each
scenario and how to make it fail. Read it first: it is the
shortest path into the rest of this list. -
The three negative checks are test targets now, not hand edits. No
PR yet; it is commit bf43011 on cq-demo-integration, and
scenario.bzl plus SCENARIO_CFGS is the part worth reading. Check
the claim each negative makes: it has to name the failure it
arranged, not just observe that the run did not finish. -
The firmware device stages into SPI NOR on CS1 now, and the runner
reads the backing file back after QEMU exits. No PR yet; commits
28c2063 and ea245b6 on cq-demo-integration. ea245b6 touches
qemu_runner.py and defs.bzl, so it is not test-only: every QEMU
test in the tree runs through that file. The claim to check is
that a guest printing PASS still fails the test when the flash
does not match, which was verified by flipping a byte. -
full_update, the whole demo script in one image: five apps, an
update over PLDM followed by the device being reset into it and
reporting ready a second time. Commit 632122b on
cq-demo-integration, which is pushed to the 9elements fork. Two
things to check. The RoT owns the sentinel here rather than the
firmware device, through the rot_owns_verdict flag, which should
appear in exactly one BUILD file and leave the other scenarios'
verdicts where they were. And the claim is the ordering, so look
at whether a stale ready report could stand in for the second
boot: the reset path clears the latch, and the run requires the
reset count to go up across the activation. -
full_update's own negatives, commit 74220c6 on
cq-demo-integration, pushed to the fork. device_hangs keeps the
device from coming up at all; device_stays_down lets the update
succeed and then keeps the device from returning. The claim to
check is that each passes on its own outcome and nothing else: a
run that commits has to fail both, and device_stays_down's pass
has to show the update accepted and activating in the log before
the checkpoint failure, or it is passing on a broken update rather
than on a device that never came back. -
Discovery, transfer error, cancel and offer-before-supervising scenarios OpenPRoT/openprot#557, merged as f604054.
Type 0 terminus discovery before offering an update,
transfer_error (agent errors on one data request),
cancel_mid_transfer (agent withdraws two chunks in), and
offer_before_supervising (offer arrives before supervision,
refused as Busy). The discovery moves the start-order retry to
GetPLDMTypes. The scenario RoT'saccept_updatereturns
Option<RejectReason>instead of bool, so the reason is decided
where the cause is known. Eleven scenarios, twenty-two targets,
all green. -
Rename mock BMC to mock downstream device OpenPRoT/openprot#559, merged as 0e2ff6c.
Renames mock-bmc to mock-downstream-device: crate, type and
scenario. The mock models the device on the other end of the reset
and ready lines, not the BMC.pldm/firmware_updatekeeps the old
name, where card B really is the update agent. -
ast10x0: Ask for one chunk twice in the update scenario OpenPRoT/openprot#560, merged as 4519639.
A twelfth scenario, pldm_update/retry_chunk. The agent answers one
RequestFirmwareData with RetryRequestFwData, the device waits out
FD_T2, asks for the same window again and the update completes.
Nothing else makes the device request a window twice, so the
re-delivery branch in its download callback had no caller. The test
device reads the kernel clock now; with FdOps::now returning a
constant its FD_T1 and FD_T2 windows never elapsed and no retry
could fire. Checked against the failure it exists for: with 08763db
reverted the run ends at "FD: the retry was asked for and the update
did not complete". The second commit drops the commit timeout from
the gap list, since Arm the commit watchdog from the machine's window OpenPRoT/openprot#554 was closed and nothing arms a deadline on
BootConfirmed. -
orchestrator: Confirm the eRoT's own update trial at the next boot OpenPRoT/openprot#565, merged as bffec2c.
mock_downstream_device/self_trial. The board gets a session and the
eRoT's own floor, the session starts at TrialPending with the trial
image running, and the run passes when the chain coming up has moved
it to Committed. What Ready proves is only that the chain under the
eRoT came up, which is the bar the 2026-10-12 decision settled on.
Checked against the failure it exists for: with the ConfirmSelfTrial
emit removed from Ready entry the run ends at "the chain came up and
the self trial was not confirmed". Only this scenario settles;
pldm_update and full_update still wire self_image: None. -
orchestrator: Move the SVN floor advance to the update agent's ask OpenPRoT/openprot#568, merged as 33b0e3e.
The SVN floor advance moves off the confirmed boot onto
Event::SvnCommitRequest, which the event loop injects when the update
agent sends UpdateSecurityRevision. No firmware device parks at
SvnCommitPending yet, so full_update and pldm_update dispatch that
event themselves, the way they already dispatch BootConfirmed.
GAPS.md says what a green run therefore does not say: the 0x22
callback is not in the pinned pldm-lib, and the event-loop glue that
reads the status and answers is unwritten.
What to look for rather than reading line by line: each test's green has
to mean something. For every scenario, ask what would still be true if
the test were lying, and whether something rules that out.
Merge commits go here as they land, since rebase merging rewrites the
SHAs and the PR numbers alone get harder to follow later.
Not a re-read, a change to make:
- services/orchestrator/driver, the update pump's verifier state.
update_verifieris anOptiononly so the session can be moved
out to poll it and moved back. The hole that leaves is real: a
bail-out between the two has to hand the verifier back by hand
(drop_session), and a path that forgets would leave the driver
unable to verify anything ever again. Putting the session in the
phase (UpdatePhase::Authenticating(V::Session)) removes the
Optionand the last defensive arm with it. The cost is that
UpdateJobbecomes generic over the board, which threads through
pending_updateand every method that touches it.
driver: Wire IncrementalVerifier into the update pump OpenPRoT/openprot#541 went as far as collapsingAuthenticating
intoStaged, so the phase and the verifier can no longer
disagree; nothing is wrong today.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTodo
Activity
chrysh commentedon Oct 8, 2026
OpenPRoT#546 merged to ocp-global-demo-wip as 9982f92 (measurement downstream wiring). OpenPRoT#547 landed via the same rebase-merge.
chrysh commentedon Oct 8, 2026
Merge SHAs for the integration-test stack, all landed 2026-10-08 on
ocp-global-demo-wip. The stack was rebase-merged, so these are the SHAs on thebranch, not the ones the PRs carried.
Foundations:
d5fcaf52harness: Seed a CS image from a file. This is the re-land;Revert "harness: Seed a CS image from a file" OpenPRoT/openprot#542 reverted the first one on 2026-10-05.
8ba07564tools: Add the flash image builder6f11f2f8services/test: Add the mock BMC boot modelca7e35f7services/mctp: Add the loopback transporte86f7654sm: End an update the requester withdrewOpenPRoT#538
81da269etarget/ast10x0: Add the mock BMC integration testOpenPRoT#550 target/ast10x0: Add the PLDM update integration test
ca789635Add the PLDM update integration test01d4aa64Run the real orchestrator in the boot scenariod4904212Put the orchestrator behind the PLDM updateb1dda205Give the test boards an update verifier508aa747Stop the PLDM update test failing at discoveryOpenPRoT#551 target/ast10x0: Stage the update into flash on CS1
a26114f9Describe how to run the integration testsa717d689Make the negative scenarios test targets9deba1b2Stage the update into flash on CS1fcec9bf3Check what the guest left in CS1382457ceSay what the flash check proves in the README7687d2bbCommand the firmware device over the shipped IPC seamf0290ecfFix and explain the memory map comments35cf1bb3Fix review findings in the PLDM update testsOpenPRoT#552 target/ast10x0: Add the full update scenario
ea24f013Write down what the integration tests do not cover5fb39f31Add the full update scenario5eb7a513Give the full update scenario its negatives763dce68Document untested parts of the DSP0267 Type 5 flow6aa5bfd7Reword test comments to stand on their ownOpenPRoT#556
bb1d6752Fix the two builds the merges broke. Not part of the stack; itcleans up after it.
State at
bb1d6752:bazel test --config=virt_ast10x0 //target/ast10x0/tests/integration/...is 16 of 16 green. Eight scenarios, eachwith a no-panics twin.
target/ast10x0/tests/integration/GAPS.mdlanded withOpenPRoT#552 and is the list of what a green run still does not say.
One thing for the re-read list. The eight
*_no_panics_testtargets all take1.7 seconds, which looks like a fixed cost rather than real work, and it is:
rust_binary_no_panics_testruns pigweed'spanic_detector --imageover thebuilt ELF on the host. It sets
PW_RUNNER_PASSTHROUGH=1so the QEMU wrapperdoes not start QEMU at all. These are static scans for reachable panic sites,
not runs, so half of that 16 is not exercising any scenario.
chrysh commentedon Oct 13, 2026
Merge SHAs for everything that landed on
ocp-global-demo-wipafter the2026-10-08 stack, through
e86ec87f. Same caveat as before: rebase-merged, sothese are the branch SHAs, not the ones the PRs carried.
Integration-test work, the part this issue is about:
5e736171Discover the terminus before offering an update69b32db6Error on one data request in the update scenario4ec9c039Withdraw an update mid-transferf604054dOffer an update to a RoT that supervises nothing559ad14eAsk for one chunk twice in the update scenario0e2ff6caRename the mock BMC to the mock downstream device45196390Drop the commit timeout from the gap listSupporting, outside the tests but reached by them:
08763dbepldm: Let the firmware device wait out its retry timer690c72eedocs: what the orchestrator cannot check during an updateCrypto client, unrelated to this issue but on the same branch:
b9d85828crypto/api: add the crypto IPC wire protocolbd2fda06crypto/client: add the crypto IPC client644b8636crypto: tighten the wire protocol API3e6d69cdcrypto_client: drop the duplicated transport paragraphHousekeeping: OpenPRoT#562
e86ec87fthird_party: Bump ruzstd to 0.9.1The suite is twelve scenarios and twenty-four targets now, up from eight and
sixteen. Four of the five new scenarios close gaps GAPS.md had listed as
uncovered: Type 0 discovery, a transfer error, a cancel mid-transfer, and an
offer arriving before the platform is supervised. The fifth, the retried chunk,
exposed OpenPRoT#555: a device told to come back for a window ended its session instead
of waiting out FD_T2, because the retry timer and a state mismatch both reported
FdInitiatorModeError.
What is left on the re-read list from the earlier comment is unchanged. The
no-panics observation still stands: of the twenty-four targets, twelve are QEMU
runs and twelve are host-side panic scans.
chrysh commentedon Oct 13, 2026
PR OpenPRoT#561 merged 2026-10-13, rebased as seven commits on
ocp-global-demo-wip:8d114688capabilities: Implement SelfUpdate for ()810e7deedriver: Give the board the eRoT's own update sessiona91fd3c6orchestrator: Add the eRoT itself as a component kind2dd30149sm: Pass over the eRoT in the chain walk3577376dtarget/ast10x0: Put the eRoT at chain position 04428f35dconfig: Require the eRoT at chain position 01327d39ctarget/ast10x0: Count the device table instead of restating itThis one touches the integration tests rather than adding scenarios: every
device table gains the eRoT at position 0, so the managed device is now
ComponentId(1)in all three images, and index 0 of each per-component arrayholds a value nothing reads. The suite stayed at 24 of 24 through the change.
Two things to look at on a re-read.
DeviceConfig::reset_signalnow returnsOption<&R>, which every board meets as a compile error. Andchain_ofgainedtwo const asserts requiring the eRoT at position 0 and nowhere else; the eRoT
rule lives only there, so a bare chain built through
Chain::try_from, whichis what the SM unit tests and the QEMU runtime test do, may still start with
any kind. That split is on purpose but it is the kind of thing worth checking
has not drifted.