Managing Channels in the AI Gateway

A "Channel" is the configuration for a vendor access point within the AI Gateway—it stores the vendor’s endpoint, actual API Key, available models, as well as pricing and quota information. The gateway uses these channels to route application requests to the appropriate vendor. This document explains how to add, configure, test, and manage channels.

Prerequisites

• ServBay is installed and running, and you are logged into your ServBay account (login required to add channels).
• The actual API Key from your target vendor is ready (not required for local vendors such as Ollama / LM Studio).
• To understand the gateway’s overall architecture, we recommend reading the AI Gateway Introduction first.

Adding a Channel

Go to AI Gateway → Channels and click Add to launch the setup wizard. The wizard consists of three steps.

Step 1: Choose a Vendor

Vendors are grouped by category; click a card to select:

• Mainstream: OpenAI, Anthropic, Google Gemini, Azure OpenAI, AWS Bedrock, OpenRouter.
• China: DeepSeek, Qwen, Zhipu GLM, Kimi, Doubao (Volcano), ERNIE, Hunyuan, MiniMax, Yi-1, Step-fn AI.
• Local: Ollama, LM Studio.
• Custom: OpenAI Compatible, Custom.

Once selected, the gateway will automatically fill in the vendor’s default Base URL.

Dual-region switching

Chinese vendors like Qwen, Zhipu GLM, Kimi, Doubao (Volcano), Hunyuan, MiniMax, Yi-1, Step-fn AI provide both China (domestic) and Global endpoints. For these vendors, you’ll see a "Region" selector (🇨🇳 China / 🌐 Global)—switching updates the Base URL accordingly.

Step 2: Fill in Configuration

• Channel Name (required) — Used to identify the channel in the list; customizable.
• Base URL (required) — The vendor's API endpoint. Most are auto-filled; Azure OpenAI and Custom require manual entry.
• API Key (optional) — The actual secret from the vendor. If left blank, only endpoint reachability can be tested, not the validity of the key; local vendors (Ollama / LM Studio) usually do not require this.
• Models — Two alternatives:
  • Automatic Discovery: Click to fetch available models from the vendor’s list API, then select models as tags (chips).
  • Manual Entry: Enter model names directly. Azure channels require you to enter the deployment name instead of the model name.
• Priority / Weight — Determines routing and load distribution when multiple channels serve the same model.

Azure OpenAI Note

For Azure channels, "Models" should be your Deployment Name as created in the Azure portal—not the underlying model name. The Base URL should also be set to your Azure resource endpoint.

Step 3: Confirm and Submit

Review your configuration summary and submit. Once successful, the new channel appears in the channel list, showing real-time health status.

Connectivity Testing

Each channel in the list can be tested for connectivity, with two dimensions:

• Endpoint Reachability — Checks whether the Base URL is accessible (network and address are correct).
• API Key Validity — Actually calls the vendor’s API to verify if the API Key is valid. Only tested if an API Key is provided.

The results show round-trip latency (ms), a status badge, and error details if any.

TIP

In the setup wizard, you cannot proceed if the endpoint is unreachable. If the endpoint is reachable but the API Key is invalid, you will receive a warning but can still continue (e.g., if you plan to update the key later).

Advanced Configuration

Expand advanced configuration while adding or editing a channel to manage costs and quotas:

• Rate Multiplier — Multiplies the vendor's official price to match your actual cost or resale price; defaults to 1.0.
• Billing Structure — Indicates how this channel is billed, e.g., pay-as-you-go, subscription, or package.
• Balance — Source can be a fixed value, OSS billing, or manual management. If using OSS billing, you can specify the billing type. Balance and last updated time are read-only in channel details.
• Subscription Expiry — Tracks expiry time for subscription/package-type channels.
• Quota Limits (VIP) — Set limits based on token count, request count, or amount and select the period (daily/weekly/monthly/custom). You can also enable "Allow Drift" to tolerate a certain overage. Quota limits require a paid plan.

Editing and Deleting Channels

• Edit — Open any channel in the list to modify its name, Base URL, API Key, models, and advanced settings.
• Delete — Once deleted, any virtual key relying on that channel can no longer route through it. Delete with caution.

Health Status

Both the channel list and the Overview page show each channel’s real-time health status (Normal / Degraded / Unavailable), helping you quickly identify invalid vendor configurations.

Frequently Asked Questions (FAQ)

• Q: Why am I prompted to log in when adding a channel?
  • A: The AI Gateway is a premium ServBay feature. You must log in to your ServBay account before adding channels or API keys. Simply follow the login prompts on the interface.
• Q: Why does it say I've reached the channel limit?
  • A: The number of channels you can create depends on your account plan. If you reach the limit, remove unused channels or upgrade your plan.
• Q: Why does automatic model discovery fail to fetch the model list?
  • A: First, make sure the Base URL is correct and the API Key is valid (use the connectivity test for "API Key Validity"). Some vendors require a valid key to return the model list. You can also switch to manual entry of model names.
• Q: Do I need to enter a key for local Ollama / LM Studio integration?
  • A: Usually not. Just make sure the local service is running and listening on the default port (Ollama 11434, LM Studio 1234).

Summary

Channels are the foundation for request routing in the AI Gateway. With the three-step wizard, you can quickly connect to nearly 20 vendors. Dual-region switching, automatic model discovery, and two-level connectivity testing ensure proper setup, while advanced options like pricing and quota controls empower you to manage costs in detail. Once channels are configured, you can create Virtual Keys for use by your applications and tools.