Skip to content

feat: add RunInfra as an OpenAI-compatible provider - #644

Open
jaberjaber23 wants to merge 2 commits into
charmbracelet:mainfrom
jaberjaber23:feat/add-runinfra-provider
Open

jaberjaber23 wants to merge 2 commits into
charmbracelet:mainfrom
jaberjaber23:feat/add-runinfra-provider

Conversation

@jaberjaber23

Copy link
Copy Markdown

Summary

Adds RunInfra (https://runinfra.ai) as an OpenAI-compatible inference provider so Crush users can select it directly.

RunInfra serves open-weight models behind an OpenAI-compatible endpoint at https://api.runinfra.ai/v1, authenticated with RUNINFRA_API_KEY. This adds the provider config (internal/providers/configs/runinfra.json) plus the matching pkg/catwalk constant and internal/providers registry entry, so go test ./... passes.

Models

Model id Context Image input Input $/1M Cached $/1M Output $/1M
DeepSeek V4.1 Flash deepseek-v4-1-flash 1,048,576 No 0.14 0.03 0.58
GLM 5.3 Flash glm-5-3-flash 1,048,576 Yes 0.11 0.03 0.45
Qwen3.8 27B qwen3-8-27b 262,144 Yes 0.10 0.01 0.40
Ornith 1.5 35B ornith-1-5-35b 262,144 Yes 0.10 0.01 0.40
Nemotron 3.5 Lightning 30B nemotron-3-5-lightning-30b 262,144 No 0.05 0.01 0.15

Defaults: large deepseek-v4-1-flash, small nemotron-3-5-lightning-30b.

Notes

  • Prices, context windows, and image-input support were taken from the public per-model pages (https://runinfra.ai/inference-api/<id>) and the pricing page, not from memory. Only deepseek-v4-1-flash and nemotron-3-5-lightning-30b are text-only on this serve; the other three accept image input.
  • The single advertised cached-input rate is placed in cost_per_1m_out_cached (cache read) with cost_per_1m_in_cached left at 0, following the convention documented in CRUSH.md.
  • The model id is the slug RunInfra's own code examples pass as the model parameter (e.g. deepseek-v4-1-flash).
  • Verified locally with go build ./... and go test ./...; the internal/providers tests confirm the provider is registered in KnownProviders() and that the default model ids exist.

Source: https://runinfra.ai/pricing

馃 Generated with Claude Code

RunInfra (https://runinfra.ai) serves open-weight models behind an
OpenAI-compatible endpoint at https://api.runinfra.ai/v1, keyed on
RUNINFRA_API_KEY. This registers the provider and its five hosted chat
models: DeepSeek V4.1 Flash, GLM 5.3 Flash, Qwen3.8 27B, Ornith 1.5
35B, and Nemotron 3.5 Lightning 30B.

Prices, context windows, and image-input support come from the public
model pages (https://runinfra.ai/inference-api) and the pricing page.
The single advertised cached-input rate goes in cost_per_1m_out_cached
(cache read), per CRUSH.md. Adds the matching pkg/catwalk constant and
registry entry so go test ./... stays green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The buyer-facing env var in RunInfra's docs and SDK snippets is
RUNINFRA_GATEWAY_KEY; RUNINFRA_API_KEY is an internal/deployment-kit secret.
Match the documented buyer var so a user's key resolves.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant