Scale and Maintain Cluster Nodes

When existing nodes cannot meet the compute requirements of the workload, you can create a node pool or increase the number of nodes in an existing node pool. After nodes join the cluster, first verify that they are available, then perform scheduling maintenance, Label Management, or deletion.

​

Create a Node Pool to Expand Capacity

1. Log in to the GPU Cloud console and go to the Clusters page.
2. Click the target cluster name to go to the cluster workspace.
3. In the left navigation pane, go to the Nodes page and switch to the Node Pools tab.
4. Click Create Node Pool and configure the node pool.

   Parameter	Description
   Node Pool Name	Set the node pool name. The node HostName is automatically generated based on the node pool name and node IP address. For example, if the node pool name is nodepool and the node IP address is 10.10.10.1, the node HostName is nodepool-10-10-10-1.
   Instance Type	Select node specifications based on the workload requirements for CPU, GPU, and memory.
   Node Instance Number	Set the initial number of nodes. Determine the required capacity based on the task scale, current Balance, inventory, and quota.
   VPC	Select the Cluster Network.
   Install GPU Driver, GPU Driver Version	Select to install the GPU driver when nodes need to run GPU workloads.

5. Click Create.
6. Return to the Node Pool tab and verify that the node pool has entered the Running state.

​

Adjust an Existing Node Pool

You can adjust the number of nodes in a node pool or add existing CES instances to the node pool.

​

Adjusting the Node Quantity of a Node Pool

When the existing configuration of a node pool meets business requirements, you can directly adjust the node quantity without creating a new node pool.

1. On the Node Pool tab, locate the target node pool and click Scale.
2. In Scale-out Node Count, enter the adjusted total number of nodes.
   • Greater than the current number of nodes: Scale out the node pool.
   • Less than the current number of nodes: Scale in the node pool.
3. Click Scale In/Out.
4. After the operation is complete, switch to the Nodes tab and confirm that the Node Quantity and Status meet expectations.

​

Add Existing Instances to a Node Pool

If an available CES instance already exists, you can add the instance directly to the node pool.

1. On the Node Pool tab, locate the target node pool, click in the Actions column, and then select Add Node.
2. Select one or more instances to add to the node pool.
3. Click Add.
4. After the action is complete, switch to the Nodes tab, confirm that the instance has joined the cluster as a node, and check the node status.

​

Confirm That Newly Added Nodes Are Available

1. On the Nodes page, switch to the Nodes tab.
2. Locate the target node and verify its node status, node pool, role, scheduling status, and node check results.
3. To view the complete configuration, click View in the Actions column for the target node.

When a Pod cannot be scheduled, a node is in an abnormal state, or workload fluctuations occur, go to node monitoring and use metrics such as CPU, memory, GPU, and network to assess node resource utilization.

​

Maintaining Running Nodes

​

Adjust Scheduling Status and Labels

1. On the Nodes tab, select one or more nodes.
2. In the action area at the bottom, select an action.

   Actions	Applicable scenario
   Cordon	Prevents new Pods from being scheduled to the node before node maintenance or decommissioning.
   Uncordon	Restores node scheduling after maintenance is complete.
   Drain	Evicts Pods from the node before maintaining, replacing, or deleting it. Before proceeding, ensure that workloads can be migrated or recreated.
   Add label	Adds a label to the node for node filtering, affinity scheduling, or workload grouping.
   Remove label	Removes a node label. Before removing it, ensure that no workloads depend on the label for scheduling.

3. Confirm the action as prompted, and verify that the node status is as expected.

​

Delete Node

• Delete a single node: In the target node’s Actions column, click , select Delete, and then follow the prompts to confirm.
• Delete nodes in batches: Select multiple nodes, choose Delete in the action area at the bottom, and then follow the prompts to confirm.