Repository navigation
load_queue: an LR waits for committed stores to drain, as an AMO does - #74
Open
adambagley wants to merge 3 commits into
Open
adambagley wants to merge 3 commits into
adambagley wants to merge 3 commits into
Conversation
The decoded-bundle queue's shadow register gives dispatch its narrow control fields a cycle early, loaded from ID's next-edge values. btb_correct_non_jalr was one of them, so ID's branch-target compare, against a predicted target PD assembles in the same cycle, fed the shadow's next-state mux. In the X3's post-opt netlist of the L2 branch, that path (pd_redirect_pc_low_r to out_shadow_q) had +0.053 ns, and a small change in the load queue took it to -0.160 ns: synthesis maps the logic after the compare differently from one netlist to the next. Dispatch only stores the bit in the branch's RS entry, so it now reads it from the queue's packet, which holds the same value in every cycle, and the bit leaves id_dispatch_ctrl_t. The compare then feeds only the ID-to-EX register, which has +0.264 ns. Dispatch sees the same value in every cycle: CoreMark takes 238,683 and 236,187 ticks before and after, and the decoded_bundle_queue bench passes. X3 post-opt closes at WNS +0.011 ns.
cached_tier_adapter records each accepted read in its slot: the valid bit, the slot's join group, and the live, sent and open bits of the group it opens. The request comes from the router, whose accept follows the LQ's launch, which follows the L0 tag lookup, and the group vectors took a clock enable decoded from it. In the X3's post-opt netlist of the L2 branch these paths had +0.089 ns to the group-vector enables and +0.261 ns to the other bits. A small change in the load queue took the enables to -0.152 ns, and with the enables fixed, the next synthesis took the other bits to -0.176 ns. Free slots now sample their join group every cycle, as they already sample the address, so the request no longer reaches the group-vector enables; a free slot's group is never read. The router also presents its cached read enable as a select between two early terms, the live LQ launch being the select. With SPLIT_READ_REQ, which cpu_and_mem sets, the adapter forms each slot's next valid, live, sent and open bits in one LUT whose late inputs are that select and the slot's own line-port term: the response that answers or bypasses the slot, or the line request that sends its group. Simulation checks that the split request equals the enable, and the router's formal target proves it. The adapter checks in every cycle that the accept state follows its unsplit definition and that each LUT's INIT gives the same bit. The adapter bench passes with and without SPLIT_READ_REQ (a new cached_tier_adapter_split entry), and the router bench, the response-mux proofs, ddr_atomic_test, directed_atomics, dma_envelope_lock3, amo_irq_torture_sim and the TestRIG traces pass. Cycles are unchanged: ddr_bench's kernel lines and lq_stale_slot_probe's 3,732,562 cycles match. X3 post-opt closes at WNS +0.011 ns. The LQ-to-adapter paths now have +0.522 ns or more, and the L1D-response-to-adapter paths +0.228 ns or more.
An LR left the LQ at the ROB head without waiting for the store queue, and any SQ write launch to the reserved doubleword cancels the reservation. The SQ's conflict check needs overlapping bytes, so a store older than an LR.W, to the other word of its doubleword, did not hold the LR. Behind slower stores, such as stores to the device quadrant, which launch one at a time, the older store could still be waiting when the LR's response set the reservation, and its write then cancelled it. The SC failed although no store after the LR touched the doubleword. The specification allows that failure, but FROST's rule and the Sail model's cancel the reservation only for the hart's stores after the LR, and CI's random A-extension TestRIG run found the difference: Sail completed the SC and FROST failed it. An LR now leaves the LQ only at the ROB head once no committed store is left to write, like an AMO, in both the issue selector and the staged-entry gate. Every older store has retired by then, and the committed-empty status clears on the ROB's combinational commit pulse, so none can still be waiting; a younger store cannot commit before the LR retires. Every SQ write that cancels the reservation now comes from a store after the LR. An LR waits for the stores ahead of it to drain, and a faulting LR reports its fault after them, as trap entry already waits for that drain. The LQ bench checks that a head LR waits for committed-empty (92 tests in each of the three configurations), and the tomasulo_wrapper and coherence benches, directed_atomics, ddr_atomic_test, rv64_amo_test, amo_irq_torture_sim and dma_envelope_lock3 pass. The load-queue, AMO-compute and tomasulo_wrapper proofs and the other LQ proofs pass. A saved TestRIG trace, lrsc_older_store.S, fails on the previous RTL and passes now; the previous RTL fails with four or more device stores ahead of the older store and passes with three, and the trace uses six. CI's counterexample and the run that found it (200 A-extension tests, seed 2541768910, with that job's DDR timing) pass, as do 400 tests from seed 1170001 at DDR latency 43. An independent review of the drain argument found no counterexample. X3 post-opt closes at WNS +0.011 ns, as before. On the X3, every hardware-regression stage passes, Debian over NFS included.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An LR left the load queue without waiting for older committed stores. A committed store to the other word of the LR's doubleword, still queued behind slower device stores, then cleared the reservation when it wrote, and the SC failed. CI's random A-extension TestRIG run on #68 found it; the bug is older than #68. An LR now waits for committed stores to drain before it issues, as an AMO does.
A load-queue unit test and a saved TestRIG trace (
lrsc_older_store.S) cover it; the trace fails on the previous RTL.Stacked on #73, which should merge first.