Skip to content

Job submission and cluster execution

CheckMAITE traditionally executes capabilities through capability.run(...), which blocks until the run finishes. For small local workloads that is fine. For long-running evaluations, it creates two problems:

  1. Interactivity — notebook users cannot keep working smoothly while a run is executing.
  2. Compute scaling — one local Python process is a poor fit for capabilities that need more CPU/GPU resources or cluster execution.

The job-submission subsystem addresses both problems:

  • it gives users a non-blocking job handle,
  • it lets the same API target local or distributed job-submission backends.

How it works

The shared job handle contract, lifecycle states, reference-first results, and error semantics.

Kubernetes-specific guidance for KubeRay placement, detached actors, autoscaling, and durability boundaries.

Why durable result writes are more subtle in distributed execution and what job submission expects from the configured store.

For backend settings, see Job backend configuration. For how each backend runs jobs and how workers are set up, see Ray job backend, Ray simple job backend, and Worker environments. To submit a job step by step, follow the Ray Simple Job Submission and Ray Job Submission tutorials.