MLOps Academy · 课时

将训练作为 Kubernetes 任务运行

在集群上执行批量训练，直到任务完成

将训练作为 Kubernetes 任务运行 是 CoddyKit 上的免费 MLOps Academy 课时。这是第 4 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 MLOps Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 MLOps Academy 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

Training Is Not a Server

A model server runs forever, but a training run should start, finish, and stop. Kubernetes has a different object built for that: the Job. 🏁

A Job Runs to Completion

A Job creates one or more Pods and watches them until they exit successfully. Once training succeeds, the Job is done and frees its resources.

apiVersion: batch/v1
kind: Job
metadata:
  name: train-ranker
spec:
  template:
    spec:
      restartPolicy: Never

restartPolicy Must Not Be Always

A Job Pod must use restartPolicy Never or OnFailure. Always is for servers and Kubernetes rejects it for a Job that is meant to end.

Retries with backoffLimit

Training can fail on a flaky download. The backoffLimit sets how many times the Job retries a failing Pod before it gives up for good.

spec:
  backoffLimit: 4
  activeDeadlineSeconds: 3600

Cap Runtime to Avoid Runaways

Set activeDeadlineSeconds so a hung training run cannot burn an expensive GPU node all weekend. The Job is killed once that limit passes. ⏱️

Request the GPU You Need

A training Job uses the same resources block as a Deployment. Add nvidia.com/gpu under limits so the run lands on a GPU node.

Parallelism for Sweeps

Set completions and parallelism to run many Pods, perfect for a hyperparameter sweep where each Pod trains one configuration at once.

Logs and Artifacts Outlive the Pod

A finished Job Pod is gone, so write your model and metrics to durable storage like S3 or a volume, never to the Pod filesystem.

CronJob for Scheduled Retraining

Wrap a Job in a CronJob to retrain on a schedule, like nightly. It creates a fresh Job each time the cron expression fires.

apiVersion: batch/v1
kind: CronJob
spec:
  schedule: "0 2 * * *"

Clean Up Finished Jobs

Completed Jobs linger by default and clutter the cluster. Set ttlSecondsAfterFinished so Kubernetes deletes them automatically after a grace period.

Watch Status with kubectl

Check a run with kubectl get jobs and read output with kubectl logs. The status shows succeeded or failed counts at a glance.

kubectl get jobs
kubectl logs job/train-ranker

Quick Check

Why is a Job, not a Deployment, right for training?

Recap

Run finite training as a Job with restartPolicy Never, a backoffLimit, and a runtime cap, then schedule retraining with a CronJob. ✅

常见问题解答

「将训练作为 Kubernetes 任务运行」课时是免费的吗？

是的 — 「将训练作为 Kubernetes 任务运行」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 MLOps Academy 课程的其余内容，请升级到 CoddyKit PRO。 MLOps Academy 课程共包含 4 节课。

「将训练作为 Kubernetes 任务运行」这节课中我会学到什么？

在集群上执行批量训练，直到任务完成你通过在浏览器中直接运行的动手代码来练习 MLOps Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 MLOps Academy 需要有经验吗？

无需任何先前经验。CoddyKit 上的 MLOps Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 4 节课，共 4 节。

「将训练作为 Kubernetes 任务运行」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 MLOps Academy 课中编写并运行代码吗？

能。每节 MLOps Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 MLOps Academy