Skip to main content
​
Training TasksInference ServicesDevelopment EnvironmentsImage Governance
Updates to training tasks, inference services, development environments, and image governance

​
New Features

  • When automatic fault tolerance is enabled for training tasks of the PyTorchJob type, the Automatic Node Replacement Upon Failure switch is now available. For details, see Create Training Tasks.
  • The Image Governance list now includes the Active Workloads column. For details, see View Image Usage Statistics.

​
Enhancements

  • Optimized inference service event display by filtering kubelet health-check events.
  • Added ephemeral storage write-risk notices to the SSH/Web login page for development environments.

​
Bug Fixes

  • Fixed an issue with Batch Quota validation for inference services.
​
Resource DashboardTraining TasksEphemeral Storage
Updates to Resource Dashboard, training tasks, and ephemeral storage

​
New Features

​
Enhancements

  • Optimized backoff handling after training task scheduling failures, improving task-processing stability in exceptional scenarios.
  • Optimized port reclamation after SSH is enabled for VS Code Server. Port resources are now automatically reclaimed when the service is stopped.
  • Optimized general service API response speed.
  • Optimized alarm and retry handling after default volume creation failures.

​
Bug Fixes

  • Optimized the scheduling status display logic for nodes in exclusive resource pools, fixing an issue where other users saw a node as scheduling-disabled when it was temporarily blocklisted by a user within their personal scope.
  • Fixed an intermittent failure when updating training task Pod statuses.
  • Fixed an issue where the username was missing from stop events due to the training task event write order.
  • Fixed an issue where exceptions during development environment event queue processing caused periodic service replica restarts.
​
Resource ManagementInference ServicesTask DashboardSchedule Policy
Updates to resource management and scheduling, inference services, and Task Dashboard

​
New Features

  • Resource Management and Scheduling
    • Exclusive resource pools now support the Resource Pool Scheduling Policy. You can choose BinPack or Spread to control whether tasks are consolidated on a small number of nodes or distributed across different nodes. The default policy is BinPack. For details, see Create a Resource Pool.
    • Authorized users of resource pools can now be imported and exported in batches. You can use strings separated by English commas or line breaks to configure user quotas in batches. For details, see Manage Authorized Users and Personal Quotas.
    • The resource pool details page now includes the All Workloads tab, which lets you view all historical workload records submitted to the current resource pool by workload type. For details, see View Resource Pool Details.
    • On View Resource Usage, the user resource details page now includes P50, P90, and P95 charts for Idle task queue duration (minutes) and Idle task run duration (minutes).
    • Added a node scheduling policy settings page that supports filtering by scope, node name, creator, reason, and other criteria. For details, see Manage node scheduling policies.
    • View Task Resource Usage now supports viewing historical development environment, inference service, and general service records.
  • Inference Services
    • LLM inference services now support configuring priority and whether preemption is allowed. For details, see Advanced Config.
​
Resource ManagementTraining TasksInference Services
Updates to resource management and scheduling, training tasks, and inference services

​
New Features

  • Resource Management and Scheduling
    • View Task Resource Usage now supports displaying instance specifications, GPU models, and actual resource usage, as well as sorting by GPU count or CPU core count.
    • View Resource Usage now includes a resource pool GPU performance metrics dashboard:
      • Displays SM utilization, Tensor utilization, GPU memory utilization, NVLink transmit and receive rates, and historical trends.
      • Supports comparing metrics across multiple resource pools and filtering by instance specification and node.
    • View Resource Usage now includes a user details page:
      • Supports accessing details from the user usage ranking and viewing user resource usage over the past 1, 7, or 14 days and by resource pool.
      • Supports viewing historical trends in the average number of GPUs in use by resource pool, with separate SM utilization trends for guaranteed and idle resources.
      • When displaying a user’s task scale, task type, and resource pool usage distribution, supports switching between GPU-hour distributions by “resource pool × task type” and “GPU model × task type.”
      • Displays the names, types, statuses, GPU counts, GPU models, resource pools, SM utilization, GPU-hours used, queue and run durations, and submission, start, and end times of users’ historical tasks.
    • View Resource Usage now includes node disk usage and alarms:
      • Supports viewing disk usage, capacity, writable capacity, and Top Pod usage for nodes in exclusive resource pools, as well as the number and details of Pods evicted within the past 24 hours because node disk usage or Pod usage reached a warning threshold.
      • When a user’s Pod ephemeral storage usage reaches the 80% or 90% threshold, the platform sends an alarm notification to the user.
  • Inference Services

​
Enhancements

  • Optimized monitoring metrics for idle tasks in Resource Dashboard, aligning the runtime duration card before the first preemption with the gauge metric.
  • Optimized fuzzy query performance for training tasks, reducing overall query time from approximately 3 seconds to approximately 1 second.