Edge AI & LLM Gateway
Run small models at the edge, or proxy any model provider through one API with caching, BYOK and rate-limits.
What it is
Edge AI runs a curated set of small (≤8 B) models at every PoP — embeddings, classifiers, small chat models — with sub-50 ms latency. LLM Gateway is a universal proxy: bring your own keys for OpenAI, Anthropic, etc., or use AIARCO credits; get a single API, response caching, per-tenant rate limits, and full request logs.
When to use it
- Cheap embeddings or classification at the edge.
- One API surface across model vendors — switch providers without changing app code.
- Cache identical prompts to cut bill in half.
Quickstart
asc edge ai embed --model edge-embed-small --text 'hello'
asc edge gateway route create openai-cheap --provider openai --model gpt-4o-mini --cache 300svec = client.edge.ai.embed(model='edge-embed-small', text='hello')
resp = client.edge.gateway.chat(route='openai-cheap', messages=[{'role':'user','content':'hi'}])const resp = await client.edge.gateway.chat({ route: 'openai-cheap', messages: [{ role: 'user', content: 'hi' }] });Limits & quotas
| Limit | Default | Burst | Notes |
|---|---|---|---|
| Edge AI neurons (in/out) | $0.011 / 1K | — | |
| Gateway proxied requests | $0.10 / M | — | Excludes upstream model cost |
| Gateway cached response | $0.02 / M | — | |
| BYOK | Supported | — | Per-route keys |
Pricing
See pricing. Pay-as-you-go, billed monthly via Stripe.
API surface
POST /v1/edge/ai/embedPOST /v1/edge/gateway/{route}/chat
Required scope(s): edge:write, gateway:write. See Scopes.
Security
All access is authenticated and scoped. See Auth & scopes and Network controls.