View Training Task Logs

While the task is running or after it is completed, you can view user logs and system logs on the training task details page to troubleshoot issues related to training scripts, images, mounts, and Pod startup.

Click the task name to go to the training task details page.
Open the Log tab.
Select User Log or System Log.
Set a time range, or filter for a specific Pod.
View the logs.

When viewing logs, you can select the log type and Pod based on your troubleshooting target:

Troubleshooting goal	What to check
Training script errors, missing dependencies, or incorrect data paths	Check User Log and use the Pod, Container, and time range to locate the error.
Image pulling, Volume mounting, Pod startup, or system-side exceptions	Check System Log to identify the failure stage and the reason returned by the platform. If further investigation of scheduling or lifecycle issues is needed, return to the details page and check Event.
Only some replicas in a multi-Pod task are abnormal	First locate the abnormal Pod in Status, then check that Pod’s logs.
Distributed RayJob logs	User code logs are typically available through the Submitter Pod; Ray cluster logs can be viewed in the Head Pod or Worker Pod.
Verify whether the task has actually started training	Check the user logs for completed initialization, checkpoint loading, training-step progress, or business metric output.

Platform logs currently retain data only for the past 30 days. If the selected start time falls outside the log retention period, logs may not be displayed. The log viewer supports keyword search, viewing the last lines, automatic refresh, manual refresh, and log downloads. It can be used to locate errors and retain troubleshooting records.

Get Started

Resources

Development & Training

Models & Inference

Operations & Governance

Best Practices

Developer Access

FAQ

View Training Task Logs

​Related Documentation

Related Documentation