Skip to main content
LLM Engine Config is used to register inference engine versions that can be selected by services and templates, and to define the runtime Image, Execute Type, and role display rules. If an existing engine configuration meets deployment requirements, you can select it directly when creating a service or template. Create or edit an engine configuration only when you need to add an engine version, adjust the runtime Image, or maintain role rules. Engine configurations maintain reusable engine definitions. When creating a service, you must still select an engine Version for the service and configure Engine Params, Execute Type, and role resources.

​
Create Engine Configuration

​
1. Go to the creation page

  1. Go to Inference > LLM Engine Config.
  2. Select a Region and Cluster.
  3. Click Create.

​
2. Configure the Engine

Enter the engine configuration JSON in the editor. The fields in the configuration collectively determine the engine versions available when creating a service, as well as the roles and configuration items displayed for different execute types. The execute type and role rules in the engine configuration must be consistent with the startup method of the corresponding image. When creating a service, use the roles actually displayed on the page for the current engine version. Do not infer roles solely based on the execute type name.

​
3. Submit Creation

Click Submit. After creation, this engine version can be selected for LLM inference services and inference templates.

​
Manage Engine Configurations

Go to Inference > LLM Engine Config, locate the target configuration, and perform the appropriate action.

​
Next Steps