> ## Documentation Index > Fetch the complete documentation index at: https://docs.scitix.ai/llms.txt > Use this file to discover all available pages before exploring further. # Advanced Operations and Maintenance for Inference Services > Learn about post-launch scaling, monitoring and alerts, priority and preemption, release changes, and testing for inference services, and find the relevant operation guides. This page summarizes advanced operations for inference services after launch, including scaling, monitoring and alerts, priority and preemption, release changes, and testing. Both LLM inference services and general services support direct and canary releases, but their configuration options and canary workflows differ. Refer to the documentation for the relevant service type. ## Scaling Inference services support manual and elastic scaling. Before scaling out, confirm that the resource pool has sufficient quota. Before scaling in, ensure that the remaining capacity can handle current traffic, persistent connections, and streaming requests. | Type | Suitable scenarios | Recommendations | | - | - | - | | Manual scaling | Business traffic changes are predictable, or capacity needs to be increased temporarily. | Use the **Resize** option in the service list to adjust the replica count, instance configuration, and resource pool by role. For LLM inference services, if the page shows an **Smart Router** configuration, also verify the routing-side replica count, instance configuration, and **Router Policy**. | | Scheduled scaling | Traffic peaks and troughs follow a clear pattern. | In the advanced settings for creating or updating a service, configure the schedule and target replica count, and allow time for resource provisioning and model warm-up. | | Metrics-based scaling | Traffic fluctuates significantly and needs to be handled automatically. | Configure rules based on the target role, metrics, threshold, collection window, cooldown period, and minimum and maximum replica counts. Start with conservative thresholds, observe the results, and adjust gradually. | Before scaling based on metrics, verify that the service's Prometheus metrics endpoint returns data: the metrics port must be a positive number, and the metrics path must begin with `/`. When configuring rules, carefully check the **collection interval**, **window**, **calculation method**, **role**, **Metrics**, **labels**, **threshold**, **duration**, **Pod aggregation**, **minimum replicas**, **maximum replicas**, **scaling step**, **cooldown period**, and **metric unavailability policy**. Missing metrics do not mean the load is `0`; follow the page's policy to skip the action, reuse the last value, or perform fallback handling. When scaling based on time, verify the scheduled time, target role, target replica count, time zone, cross-day rules, and minimum service capacity. Allow time for resource provisioning and model warm-up. The maximum replica count is only an adjustment limit; the platform does not reserve resources in advance based on it. If any role in a service uses the `variants` multi-instance group configuration, metrics-window autoscaling cannot be enabled for the service. After a manual adjustment, an automatic policy may change the target replica count again. Before troubleshooting at a fixed capacity or load testing, decide whether to suspend automatic policies. ## Monitoring and alerts Inference services can collect resource and engine metrics. Resource monitoring shows CPU, GPU, GPU memory, memory, and network usage. Engine monitoring shows request-, latency-, and throughput-related metrics for engines such as vLLM and SGLang. ![Resource monitoring for LLM inference services](https://mintcdn.com/scitix-adfc65c2/ZqwdmfntBUDyGPkJ/siverse/inference/media/llm-inference-service-resource-monitoring.png?fit=max\&auto=format\&n=ZqwdmfntBUDyGPkJ\&q=85\&s=a9583856f4813cf3e8c99273d6047f96) | Objective | What to check | Suggested action | | - | - | - | | Identify resource bottlenecks | CPU, GPU, GPU memory, memory, and network. | If resource usage remains near its limit and latency rises, consider scaling out or adjusting engine parameters. | | Identify engine bottlenecks | Engine metrics such as request volume, errors, latency, and throughput. | Review these with resource metrics to distinguish server-side bottlenecks from fluctuations in requests. | | Configure alerts | Service exceptions, error rates, delayed deletion, or custom metrics. | Subscribe to alerts related to inference services in the alert center, and configure notification policies according to service availability requirements. | Available service-exception alerts depend on the subscriptions offered in the console. To monitor custom metrics, first verify that the service exposes a metrics path and port. ## Priority and preemption When creating an LLM inference service, use **Advanced Config** to set **Priority** and **Preemptible**. **Priority** sets the service's scheduling priority; **Preemptible** determines whether the service can be preempted. For the configuration entry point and other advanced parameters, see [Create an LLM Inference Service](/siverse/inference/create-llm-inference-services.md#advanced-config). ## Release and testing | Objective | Reference documentation | | - | - | | Release an LLM inference service directly or as a canary, and roll back if the target version has an issue. | [Release and Roll Back LLM Inference Services](/siverse/inference/release-and-rollback-llm-inference-services.md) | | Release changes to a general service directly, or validate the target version with a portion of the traffic before a full release or rollback. | [Release and Roll Back General Services](/siverse/inference/release-and-rollback-general-inference-services.md) | | Verify that the service API and model capabilities meet expectations. | [Test LLM Inference Service Functionality](/siverse/inference/test-llm-inference-service-functionality.md) | | Evaluate service throughput, latency, and capacity limits. | [Load Test LLM Inference Services](/siverse/inference/load-test-llm-inference-services.md) | Capabilities such as intelligent routing, distributed KV cache, and custom metrics dashboards depend on the engine, cluster, and service configuration. Enable them according to the business goal only when the page provides the relevant settings. Until the available options and fields are confirmed, do not describe these capabilities as fixed procedures. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.