Skip to main content
When a development environment, training task, or inference service fails to start, remains queued for an extended period, or displays an insufficient resources message upon submission, you typically need to check the total resources, task usage, resource pool configuration, personal quota, and order status. This document provides a cross-module troubleshooting path to help you first determine whether resources are truly insufficient, and then identify the layer at which the shortage occurs.

​
Issue Symptoms

The following issues may occur:
  • After creating a development environment, training task, or inference service, its status remains Queued or Waiting for Scheduling for an extended period.
  • When submitting a task, a message indicates insufficient resources, insufficient quota, or an unavailable resource pool.
  • The resource dashboard shows that resource pool usage is close to the total quota.
  • Within the same resource pool, some users can submit tasks while others cannot.
  • Large-specification tasks cannot obtain contiguous resources in a dedicated resource pool for an extended period.

​
Troubleshoot Total Resource Quantity

First, verify whether capacity is generally constrained in the target region, cluster, resource pool, or instance specification.
  1. Go to Resources > Resource Dashboard.
  2. On the Resource Pool tab, select the target region, cluster, resource type, instance specification, and time range.
  3. View the total quota, usage, scheduling rate, and unavailable resources.
If usage continues to approach the total quota, the resource pool may be at full capacity. You can scale out the resource pool, purchase more instance resources, or wait for tasks to release resources. If many resources are unavailable, go to the resource pool details page to check the resource pool and node statuses and determine whether there are any resource issues.

​
Distinguish Between Remaining Quota and Schedulable Capacity

If the page shows that quota remains but the task is still queued, further check whether schedulable resources are available that meet the current task configuration. Remaining quota indicates whether permissions and quota are sufficient; it does not mean that a task can necessarily be scheduled immediately. A single replica typically needs to be placed on a node that meets its resource requirements. If idle resources are distributed across multiple nodes, a replica requiring greater resources on a single node may still fail to start even when the total available resources are sufficient. Schedulable capacity changes with node status, the schedule policy, and other tasks. Querying schedulable capacity is used only to determine the number of replicas that can be scheduled at the current time; it does not lock or reserve resources.

​
Resolving Display Issues with Schedulable Capacity

If the task creation page or resource-related pages display schedulable capacity, first determine whether the resources are truly unschedulable or whether reliable query results are temporarily unavailable. If total available resources appear sufficient but the schedulable capacity remains 0, a common reason is that a single replica cannot fit on any node. For example, if a single task replica requires 4 units of the selected specification, while each of two nodes has only 3 units available, the total available capacity is 6 units, but the replica still cannot be started. For role configuration methods, see Resource Configuration for Training. The preceding values are for demonstration purposes only; actual resource quantities depend on the selected specification. In this case, you can wait for a single node to release sufficient resources, or adjust the resource configuration for a single replica after confirming that the program can run properly.

​
Troubleshooting Task Occupancy

If total resources are constrained, further verify which tasks are occupying them.
  1. Go to Resources > Task Dashboard.
  2. On the Running Tasks tab, filter by Region, Cluster, Resource Pool, Workload Type, or user.
  3. View the resource usage and GPU/CPU resource utilization of running tasks.
If you identify tasks that have been running for an extended period with low utilization, go to the relevant task details or business module to determine whether they still need to run. Once you have confirmed that they are no longer needed, you can stop the tasks to release resources. To further review task execution, go to the task details page to view Monitor and Logs.

​
Troubleshoot resource pool configuration

If Resource Dashboard shows that idle resources are still available but the task cannot be started, check whether the resource pool itself allows the current task to use them. Go to Resources > Instance, open the details of the target resource pool, and verify the following: If the Task Type is not allowed by the resource pool, submission may be rejected during quota validation, with a message similar to:
If you confirm that you need to submit a task to this resource pool, adjust the resource pool’s workload type configuration.

​
Troubleshooting Exclusive Resource Pool Fragmentation

When using an exclusive resource pool, large-scale tasks may wait for a long time because they cannot obtain contiguous whole-node resources. Go to the dedicated resource pool details page to view the following information: Defragmentation may migrate or stop some workloads. Before proceeding, verify that the tasks can be migrated or stopped, and review the defragmentation results.

​
Troubleshoot Order and Quota Validity პერიოდ

If resources were previously available but tasks can no longer be submitted, check whether the associated order has expired.
  1. Go to Resources > Order.
  2. Check the status and expiration date of the instance resource order.
  3. If the order status is expired, renew or purchase the resources again first.
  4. After the order status returns to active, go back to the instance resources or resource pool to confirm whether the quota has been restored.
After an order expires, the associated quota becomes invalid, which may reduce the total quota of the resource pool or prevent tasks from continuing to use the corresponding resources.

​
Still Unable to Resolve

If the cause still cannot be identified after the above checks, retain the following information and contact the platform administrator or technical support:
  • Task name, task type, and creation time.
  • Region, cluster, resource pool, and instance specification.
  • Current task status, status details, or error messages.
  • Total quota, usage, and unavailable resources of the target resource pool in the Resource Dashboard.
  • Resource usage of related running tasks in the Task Dashboard.
  • Authorized users of the resource pool, personal quotas, and workload type configurations.
  • Status and expiration time of related resource orders.

​
FAQ

Remaining quota indicates how many resources the current account is still allowed to use; it does not mean that the resource pool necessarily has schedulable resources at this time. The task must still match the resource pool status, personal quota, workload type, single-node capacity, and schedule policy. If idle resources are distributed across multiple nodes, replicas that require greater single-node resources may also be unable to start.
  • If the effective query returns 0, no replicas are currently schedulable under the selected specifications and constraints. Continue checking the resource pool, quota, workload type, and scheduling events.
  • If — is displayed, the status is temporarily unavailable, or the request returns an error, the page has not yet obtained reliable results. You can refresh and try again. If the issue persists, retain the resource pool, instance specification, task configuration, and screenshots, and contact platform support.