Prepare LLM Inference Config

Before creating an LLM inference service, determine whether to reuse existing configurations. If suitable Engine configurations and inference templates already exist, you can select them directly when creating the service. If you need to establish standard configurations for your team, first maintain the corresponding configuration objects.

​

Configure Object

Not every service creation requires creating a new Engine configuration or inference template. If an existing configuration meets the deployment requirements, you can proceed directly to the service creation process.

​

Configure Large Model Engine

1. Go to Inference > LLM Engine Config.
2. Select a Region and Cluster.
3. Click Create.
4. Enter the engine configuration JSON.
5. Click Submit.

Common fields are as follows: After creation, you can edit, clone, or delete existing Engine configurations as needed. Before deleting a configuration, ensure that no templates or services depend on it.

​

Prepare Inference Templates

If multiple services reuse the same model, engine, probe, monitoring, or advanced configuration, you can first create an LLM inference template. After the template is created, enable Use Recommended Template and select the template on the LLM inference service creation form. For information about creating, reusing, and managing templates, see Manage LLM Inference Templates.

​

Next Steps

• Manage LLM Inference Templates
• Create LLM Inference Services