Skip to content

Tool-calling runs get quadratically slower: completed tool calls are re-fingerprinted every turn #5382

Open

Description

@deadstrobe5

Describe the bug

Run time grows quadratically with the number of tool-calling turns. On every turn, _dedupe_processed_response_invocations (src/agents/run_internal/tool_planning.py) recomputes the canonical identity of every completed tool call in the run history: model_dump, argument normalisation and two SHA-256 digests per call. It also model_dumps every history item just to read its type. With one tool call per turn, the per-turn cost is O(history), so a run costs O(turns²).

With a fake model, one tool call per turn, a 300-turn run takes 2.69 s on main (17d994b). Of that, about 1.2 s is this re-fingerprinting.

Root cause and a possible fix

The history fingerprints are only needed for call IDs that can collide: IDs that are reused within the history, or that appear in the current response. Fingerprinting only those call IDs, skipping model_dump when the model's type already rules the item out, and not rebuilding ToolApprovalItems when legacy reconstruction is off keeps the duplicate-ID errors and all outputs unchanged. new_items and to_input_list() hash identically in both run and stream.

I have this change with a regression test on a fork branch, in case it is useful to a collaborator: deadstrobe5/openai-agents-python@main...avoid-history-fingerprinting-per-turn

Fake model, one tool call per turn, CPython 3.14, 5 interleaved runs per side, median (range):

turns main with the change
50 141 ms (138-178) 104 ms (102-106)
100 385 ms (379-415) 244 ms (240-261)
200 streamed 1300 ms (1293-1328) 766 ms (747-788)
300 2691 ms (2625-2796) 1481 ms (1436-1505)

Debug information

  • Agents SDK version: 0.24.0 (main at 17d994b)
  • Python version: 3.14
  • Operating system: macOS
  • Model and model provider: fake Model (no network)
  • Does the issue reproduce with the latest Agents SDK release? Yes
  • Does the issue occur consistently or intermittently? Consistently

Repro steps

import asyncio, sys, time, json
from agents import Agent, Runner, function_tool, ModelResponse, Usage, RunConfig
from agents.models.interface import Model
from openai.types.responses import ResponseFunctionToolCall, ResponseOutputMessage, ResponseOutputText

N = int(sys.argv[1]) if len(sys.argv) > 1 else 200
MODE = sys.argv[2] if len(sys.argv) > 2 else "run"

@function_tool
def lookup(q: str) -> str:
    return "result for " + q + " " + "x" * 200

class FakeModel(Model):
    def __init__(self, n):
        self.i = 0; self.n = n
    def _out(self):
        self.i += 1
        if self.i > self.n:
            return [ResponseOutputMessage(id=f"m{self.i}", type="message", role="assistant", status="completed",
                content=[ResponseOutputText(text="done", type="output_text", annotations=[], logprobs=[])])]
        return [ResponseOutputMessage(id=f"m{self.i}", type="message", role="assistant", status="completed",
                    content=[ResponseOutputText(text="thinking " * 20, type="output_text", annotations=[], logprobs=[])]),
                ResponseFunctionToolCall(id=f"f{self.i}", call_id=f"c{self.i}", type="function_call", name="lookup",
                    arguments=json.dumps({"q": f"query {self.i}"}))]
    async def get_response(self, *a, **k):
        return ModelResponse(output=self._out(), usage=Usage(), response_id=None)
    async def stream_response(self, *a, **k):
        from openai.types.responses import ResponseCompletedEvent, Response
        out = self._out()
        yield ResponseCompletedEvent(type="response.completed", sequence_number=0, response=Response(
            id="r", created_at=0, model="fake", object="response", output=out, tool_choice="auto", tools=[],
            parallel_tool_calls=True))

async def main():
    agent = Agent(name="a", instructions="x", model=FakeModel(N), tools=[lookup])
    t = time.perf_counter()
    if MODE == "stream":
        r = Runner.run_streamed(agent, "hi", max_turns=N + 5)
        async for _ in r.stream_events(): pass
    else:
        r = await Runner.run(agent, "hi", max_turns=N + 5)
    dt = time.perf_counter() - t
    print(f"{MODE} N={N} {dt*1000:.1f} ms items={len(r.new_items)} out={r.final_output!r}")
    return r

if __name__ == "__main__":
    asyncio.run(main())

python bench.py 300 or python bench.py 200 stream. Doubling the turns roughly quadruples the time spent in _dedupe_processed_response_invocations.

Expected behavior

Per-turn overhead that does not grow with the length of the run.

Activity

  1. zhgao commented on Oct 17, 2026

    @zhgao

    Hi, I've implemented a fix for this issue and it's ready on my fork: zhgao/openai-agents-python branch fix/issue-5382.

    Compare view: main...zhgao:fix/issue-5382

    The branch has a single commit ("Fix quadratic slowdown in tool-calling runs from repeated history fingerprinting") which:

    • Adds two cheap helpers, tool_invocation_type_and_call_id and tool_output_type_and_call_id, in src/agents/_tool_invocation.py (they read type/call_id attributes directly instead of calling model_dump)
    • Reworks _dedupe_processed_response_invocations in src/agents/run_internal/tool_planning.py so full identity fingerprints are only computed for call IDs that can collide (IDs reused within the history or appearing in the current response); everything else uses the cheap attribute reads, keeping the per-turn cost flat as the run grows. Duplicate-detection semantics are unchanged.
    • Adds tests/test_dedupe_history_fingerprinting.py with 3 regression tests (fingerprint calls stay O(turns) over a 24-turn scripted run; a reused call ID with a different fingerprint still raises ModelBehaviorError; an identical reused call ID is accepted).

    However, this repository restricts pull request creation to collaborators, so I'm unable to open the PR myself. Could a maintainer please open a PR from the fix/issue-5382 branch on my fork (zhgao/openai-agents-python), or grant me collaborator access so I can submit it directly? I'm happy to address any review feedback. Thank you!

  2. AbarnaaSree commented on Oct 18, 2026

    @AbarnaaSree

    Hi, I’d like to help validate and improve the fix for this issue. Since a solution is already available on a contributor’s fork, would it be okay if I review the proposed changes, verify that duplicate-call detection and streamed runs preserve existing behavior, and run additional benchmarks and regression tests? I’ll coordinate with the existing contributor to avoid duplicating work and focus on any remaining gaps before proposing further changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions