Private LLM gateway for teams

NeoGate - Simple and easy-to-use LLM API gateway

Put OpenAI, Anthropic, and other upstream model providers behind one API entry point controlled by your company. Issue project API keys to internal apps, keep upstream credentials in one place, and track usage and cost by team, project, model, and key.

View on GitHub Quick deploy

Rust: fast relay layer
Projects: keys and usage isolated
Docker: start single-node

LLM API gateway OpenAI proxy Anthropic API usage tracking billing

Enterprise usage scenarios

Built for teams that need to share model access without losing control.

NeoGate focuses on the daily operational problems that appear once multiple teams, internal services, or business projects start calling large model APIs.

One internal model API entry point

Give every application the same company-controlled gateway instead of asking teams to connect to each provider separately.

Project keys for teams and apps

Issue separate API keys for departments, projects, services, or automation jobs so permissions and usage stay isolated.

Centralized upstream credentials

Keep vendor keys in the gateway and manage OpenAI, Anthropic, and compatible upstream channels from the admin console.

Usage and cost by project

Record usage by user, project, API key, model, provider, and channel so finance and platform teams can analyze cost accurately.

Internal mode first

Run without requiring user balance while still recording cost, making it practical for internal analysis and chargeback.

Failover when upstream keys fail

Cool down failing upstream keys and continue routing through available credentials to reduce impact on business applications.

Enterprise request path

Business applications call project keys; NeoGate handles the control plane.

Internal applications call NeoGate with project API keys. The gateway authenticates the project, applies service-mode policy, selects an upstream channel and credential, then records usage for troubleshooting, cost analysis, and internal accounting.

1Internal app calls a project API key

2NeoGate checks project policy and service mode

3Gateway selects provider channel and upstream credential

4Response is relayed and usage is recorded

Deployment path

Start with a single-node gateway, then move to clustered mode when you need scale.

git clone https://github.com/neogate-io/NeoGate.git cd NeoGate docker compose up -d --build

Most teams can start with the standalone Docker Compose deployment: frontend Nginx, backend, and PostgreSQL together. When multi-replica traffic becomes a real requirement, switch to distributed mode with shared PostgreSQL and Redis.