Find, View, and Manage Training Tasks

After submitting a training task, you can filter and locate it in the task list. On the training task details page, you can view tabs such as Status, Info, Event, and Storage, and clone, resubmit, stop, delete, share, or manage labels for the task.

Find Training

The task list includes the Personal Tasks and Shared Tasks tabs. In Personal Tasks, you can typically view tasks, view logs, clone, resubmit, stop, delete, share, and add labels. In Shared Tasks, you can typically view details, view logs, or clone tasks.

Go to Training > Workloads.
Select a region and cluster.
Select the Personal Tasks or Shared Tasks tab.
As needed, enter a name, select a status, label, or project group, or click View All to expand the search scope. Tag filtering typically uses an AND relationship. When multiple tags are selected, the list filters for tasks that match all of the selected tags.
Click Search.

When viewing search results, focus on the Name, Owner, Project Group Name, Resource Pool, Status, Priority, Instance Spec, creation time, update time, and running time to determine task ownership, execution progress, resource usage, and subsequent handling. Immediately after submitting a task, if it does not appear in the list, search or refresh first. After confirming that the task has been created and recording its task ID, go to the training task details page to view the actual resources, status of each replica, events, logs, and output path, avoiding repeated submission of the same configuration.

Determine Task Status

After locating the task, use its status to determine whether to continue waiting, view details, troubleshoot the issue, or perform management actions such as cloning, resubmitting, stopping, or deleting it.

Status	Description	Recommended Action
Pending	The task has been submitted and is waiting to be scheduled or initialized.	View Event or wait for scheduling.
Queued	The task is in the queue, typically due to quota, priority, or scheduling conditions.	View Event, resource pool quota, priority, and node scheduling policies.
Partially Pending	Some Pods have been scheduled, while others are still pending.	View Status and Event to identify Pods that are not ready, or node- or resource-related causes.
Running	The task is running.	View Log and Monitoring.
Succeeded	The task completed successfully.	View the output or register the model.
Failed	The task failed to run.	View Log and Event, then resubmit or troubleshoot as needed.
Error	A platform- or system-side error has occurred.	View Event and contact the platform administrator if necessary.
Stopping	The task is being stopped.	Wait for the task to stop.
Stopped	The task has stopped.	Clone, resubmit, or delete the task.

Tasks with automatic fault tolerance enabled may display fault-tolerance-related substatuses. When an idle-time task or Spot task is preempted, it may enter the failed state. If the status information shows was preempted by another task, this usually indicates that the task was preempted by another task. Determine the specific cause based on Status and Event. Queue position represents only a snapshot of resource contention at the current moment and cannot be converted into a stable wait time. Changes in resource pool availability, resources released by other tasks, Priority, Guarantee, or node constraints may affect task ordering and startup conditions.

View Task Information by Target

After locating the task and opening the training task details page, first select the appropriate tab based on what you want to view. Recommended use by target:

Objective	Tab	Information to Review	Next Steps
Determine which stage the task is stuck in	Status	Lifecycle timeline, task status, Pod status, and status description.	If the task is stuck in the queuing or scheduling stage, continue reviewing Event and resource information. If it is stuck in the running stage, review Log and Monitoring.
Troubleshoot scheduling or startup failures	Status, Event	Pod status, Pod exit code, node information, status description, and event information.	Based on the event information, check resource quotas, image pulling, Volume mounting, startup commands, and node scheduling policies.
Verify the creation configuration and actual resources	Info	Basic information, Resource Information, actual resource pool, Environment Information, mounts, cmd, and advanced feature configurations.	If the configuration does not match expectations, clone the task, modify the configuration, and resubmit it. If the off-peak fallback option is enabled, refer to the actual resource pool shown in the details.
View ephemeral storage and node disks	Storage	Pod ephemeral storage usage, quota, utilization level, and status, as well as the disk utilization level and remaining writable space of the node where the Pod resides.	If training logs, checkpoints, intermediate data, or model outputs continue to grow, write them to a mounted Volume to prevent the Pod’s ephemeral storage from filling up and causing the task to be evicted. For details, see Best practices for ephemeral storage.

View Status Details

In the task list, click the name of the target task to open the training task details page.
Open the Status tab.

Status is used to view task execution stages and Pod status.

View Task Details

In the task list, click the name of the target task to go to the training task details page.
Open the Info tab.

Info is used to verify the task creation configuration.

View Events

Click the task name to go to the training task details page.
Open the Event tab.
Review events in chronological order to determine whether the task is stuck during scheduling, image pulling, Volume mounting, Pod startup, execution, or fault recovery.
If an event points to a Pod, node, resource pool, or mount target, return to the Status, Info, Log, or Monitoring tab for further troubleshooting.

Events are used to explain platform-side scheduling and lifecycle changes. Common troubleshooting targets include:

Troubleshooting Target	Events to Watch
Prolonged queuing	Insufficient resources, quota validation failures, matching node scheduling policies, or unavailable resource pools.
Startup failure	Image pull failures, Volume mount failures, Pod creation failures, or startup command exceptions.
Runtime interruption	Pod restarts, node exceptions, resource preemption, or fault-tolerance recovery actions.
Fault-tolerance recovery	Events related to automatic retries, in-place restarts, resubmission, or failed-node avoidance.

Events are useful for first identifying when an issue occurred. Logs are used to view container and training program output, while monitoring is used to assess resource usage and metric performance. For distributed tasks, if only some replicas are not ready, first locate the specific Pod or role in Status, then use Event to determine whether the issue is related to resources, nodes, images, mounts, or startup commands.

View Storage

In the task list, click the name of the target task to go to the training task details page.
Open the Storage tab.

Storage is used to view temporary storage for task Pods and disk information for the nodes on which they reside.

Pod ephemeral storage accounts for local temporary data such as container writable layers, emptyDir, and container logs. It does not include mounted datasets, model storage, or other PVC data.
The disk information of the node where the task Pod resides is used to view the disk watermark and remaining writable space on that node. The data displayed on the page may be subject to collection delays. Please refer to the page prompts.

If training logs, checkpoints, intermediate data, or model outputs continue to grow, write them to a mounted Volume to prevent the Pod’s ephemeral storage from filling up and causing the task to be evicted. For details, see Best practices for ephemeral storage.

Manage Training

After locating the target task in the task list or on the training task details page, you can perform management actions based on the task status and your permissions.

Action Target	Action	Notes
Re-create a task using an existing configuration	Clone	Loads an existing task configuration into the creation page. Modify it as needed and submit a new task. When cloning a shared task, if it includes a Volume that the current account is not authorized to use, select the storage again and verify the paths in the command.
Run an existing task again	Resubmit	Resubmits the task with the same configuration. This is suitable for troubleshooting intermittent failures or rerunning experiments. Resubmission supports both normal mode and Debug mode. Debug mode typically retains the original resource requirements but starts a new task with a troubleshooting command, making it easier to investigate issues on site.
Interrupt a queued or running task	Stop	Stopping a task releases its resources while retaining the task record for subsequent viewing, cloning, or resubmission. Before stopping a task, ensure that the training process can be interrupted and that any checkpoints, logs, or model outputs that must be retained have been written to a persistent Volume.
Remove records that are no longer needed	Delete	Deletion cannot be undone. If the page supports batch deletion, ensure that none of the selected tasks are needed and that any checkpoints, logs, or model outputs that must be retained have been written to a persistent Volume before performing the batch operation.
Allow other users to view or reuse a task	Share	Shared users can typically view details and logs, and clone tasks. For differences in task visibility, project group visibility, personal sharing, and storage permissions, see Manage Training Task Sharing and Project Group Visibility.
Categorize and filter tasks	Add Labels	Label validation rules are determined by the Label Management configuration.
Adjust visibility within a project group	Set Visibility	Public status affects only task visibility; it does not automatically grant read or write permissions for resource pools, Volumes, or directories. For details, see Manage Training Task Sharing and Project Group Visibility.

Get Started

Resources

Development & Training

Models & Inference

Operations & Governance

Best Practices

Developer Access

FAQ

Find, View, and Manage Training Tasks

Find Training

Determine Task Status