Skip to content

feat(llms): add OrcaRouter named provider - #371

Open
JinhaoSong322 wants to merge 1 commit into
InternLM:mainfrom
JinhaoSong322:add-orcarouter-provider
Open

JinhaoSong322 wants to merge 1 commit into
InternLM:mainfrom
JinhaoSong322:add-orcarouter-provider

Conversation

@JinhaoSong322

Copy link
Copy Markdown

Summary

Add OrcaRouter as a named LLM provider in Lagent, mirroring the existing GPTAPI / AsyncGPTAPI OpenAI-compatible wrappers.

OrcaRouter is a gateway that fronts open-weight models from many vendors through a single OpenAI-compatible endpoint. It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

What changed

  • lagent/llms/orcarouter.py (new): OrcaRouterAPI and AsyncOrcaRouterAPI, subclasses of GPTAPI / AsyncGPTAPI with OrcaRouter defaults:
    • base url https://api.orcarouter.ai/v1/chat/completions
    • default model orcarouter/auto (auto-routed open-weight models on the gateway)
    • reads ORCAROUTER_API_KEY (keys start with sk-orca-) instead of OPENAI_API_KEY
    • accepts gateway aliases (orcarouter/*) and vendor-qualified model names (vendor/model, e.g. deepseek/deepseek-v4-pro) via the generic Chat Completions payload
  • lagent/llms/__init__.py: export the new wrappers.
  • docs/en/get_started/quickstart.md: document the "Using OrcaRouter" section.
  • tests/test_llms/test_orcarouter.py (new): 11 unit tests covering defaults, env-key resolution, payload generation (gateway alias / vendor-qualified / json mode / unsupported model), and a mocked chat call.

Verification

  • flake8 clean on changed files; py_compile passes.
  • Unit tests: 11/11 passed.
  • Full suite: identical results to clean main (7 pre-existing environment/network-dependent failures in tests/test_actions/*; tests/test_agents/test_rewoo.py fails collection on clean main too due to a refactored-out llm_qa module — unrelated to this change).
  • L3 live test against https://api.orcarouter.ai/v1 with a real key, through the new provider path:
    • sync chat (orcarouter/auto) → ORCA-OK
    • async chat → ORCA-OK
    • vendor-qualified model deepseek/deepseek-v4-pro → VENDOR-OK
    • json mode payload generation verified

Example

import os
os.environ['ORCAROUTER_API_KEY'] = 'sk-orca-...'

from lagent.llms import OrcaRouterAPI
llm = OrcaRouterAPI(model_type='orcarouter/auto', retry=5, max_new_tokens=2048)
print(llm.chat([{'role': 'user', 'content': 'Hello!'}]))

Disclosure: I'm an engineer on the OrcaRouter team.

Add OrcaRouterAPI and AsyncOrcaRouterAPI wrappers that mirror the
existing GPTAPI/AsyncGPTAPI OpenAI-compatible wrappers with OrcaRouter
defaults:

- base url https://api.orcarouter.ai/v1/chat/completions
- default model orcarouter/auto (auto-routed open-weight models)
- reads ORCAROUTER_API_KEY (sk-orca- prefix) instead of OPENAI_API_KEY
- accepts gateway aliases (orcarouter/*) and vendor-qualified model
  names (vendor/model) through the generic Chat Completions payload
- registered in lagent.llms exports, documented in quickstart.md,
  covered by 11 unit tests

L3 live-tested against the real endpoint (chat + async + vendor-qualified
model + json mode all returned correct responses).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: JinhaoSong322 <jinhao.song@myflashcloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant