Jobs Overview
Jobs are the fundamental unit of computation in Magpie Cloud. A job represents a script or command that runs in an isolated environment with specified compute resources. Jobs can be ephemeral, stateful, or persistent depending on your use case.
Job Types
Ephemeral Jobs (Default)
Ephemeral jobs run once and terminate automatically when the script completes. They start with a clean filesystem and do not persist any data after completion. Perfect for stateless tasks like batch processing, CI/CD pipelines, and data transformations.
Stateful Jobs
Stateful jobs preserve the /workspace directory between runs. Use stateful jobs when you need to cache data, store intermediate results, or maintain state across multiple executions. Enable with stateful=True.
Persistent VMs
Persistent VMs stay running until explicitly stopped. They provide long-lived compute environments for interactive development, services, and applications. Access them via SSH for interactive work. Enable with persistent=True.
jobs.create()
Create and start a new job with the specified configuration.
Parameters
| Parameter | Type | Description |
|---|---|---|
| name required | str | A unique name for the job. Used for identification and retrieval. |
| script required | str | The bash script or command to execute in the job environment. |
| vcpus optional | int | Number of virtual CPUs to allocate. Defaults to 1. |
| memory_mb optional | int | Memory allocation in megabytes. Defaults to 512 MB. |
| stateful optional | bool | If True, preserves the /workspace directory between runs. Defaults to False. |
| persistent optional | bool | If True, keeps the VM running until explicitly stopped. Defaults to False. |
| ipv6_lease optional | str | IPv6 address lease identifier for networking configuration. |
| env optional | dict | Environment variables to set in the job environment as key-value pairs. |
| timeout optional | int | Maximum execution time in seconds. Job will be stopped if exceeded. |
Returns
Returns a job object with properties including id, name, status, vcpus, memory_mb, and created_at.
Example
from magpie import Magpie
client = Magpie(api_key="your_api_key_here")
# Create an ephemeral job
job = client.jobs.create(
name="data-processor",
script="python process_data.py",
vcpus=4,
memory_mb=2048
)
print(f"Job ID: {job.id}")
print(f"Status: {job.status}")
# Create a stateful job with environment variables
stateful_job = client.jobs.create(
name="ml-training",
script="""
pip install torch
python train.py --epochs 100
""",
vcpus=8,
memory_mb=16384,
stateful=True,
env={
"MODEL_PATH": "/workspace/models",
"DATASET": "imagenet"
},
timeout=3600
)
# Create a persistent VM
persistent_vm = client.jobs.create(
name="dev-environment",
script="echo 'VM ready for development'",
vcpus=2,
memory_mb=4096,
persistent=True
) jobs.run_and_wait()
Create a job and wait for it to complete, polling for status updates at regular intervals.
Parameters
| Parameter | Type | Description |
|---|---|---|
| name required | str | A unique name for the job. |
| script required | str | The bash script or command to execute. |
| vcpus optional | int | Number of virtual CPUs to allocate. |
| memory_mb optional | int | Memory allocation in megabytes. |
| stateful optional | bool | If True, preserves the /workspace directory between runs. |
| env optional | dict | Environment variables as key-value pairs. |
| timeout optional | int | Maximum execution time in seconds. |
| poll_interval optional | int | Seconds to wait between status checks. Defaults to 5. |
Returns
Returns a completed job object with the final status (completed or failed) and all job details.
Example
from magpie import Magpie
client = Magpie(api_key="your_api_key_here")
# Run a job and wait for completion
job = client.jobs.run_and_wait(
name="batch-report",
script="""
echo 'Generating report...'
python generate_report.py
echo 'Report complete!'
""",
vcpus=2,
memory_mb=1024,
poll_interval=10
)
print(f"Job finished with status: {job.status}")
# Get the logs after completion
logs = client.jobs.get_logs(job.id)
print(logs) jobs.get()
Retrieve detailed information about a specific job by its ID.
Parameters
| Parameter | Type | Description |
|---|---|---|
| job_id required | str | The unique identifier of the job to retrieve. |
Returns
Returns a job object containing all job details including configuration, status, timestamps, and resource allocation.
Example
from magpie import Magpie
client = Magpie(api_key="your_api_key_here")
# Get job details
job = client.jobs.get("job_abc123xyz")
print(f"Job Name: {job.name}")
print(f"Status: {job.status}")
print(f"vCPUs: {job.vcpus}")
print(f"Memory: {job.memory_mb} MB")
print(f"Created: {job.created_at}")
print(f"Stateful: {job.stateful}")
print(f"Persistent: {job.persistent}") jobs.get_status()
Get the current status of a job. Returns one of five possible states.
Parameters
| Parameter | Type | Description |
|---|---|---|
| job_id required | str | The unique identifier of the job. |
Returns
Returns a string representing the current job status:
- pending - Job is queued and waiting to start
- running - Job is currently executing
- completed - Job finished successfully
- failed - Job terminated with an error
- stopped - Job was manually stopped
Example
from magpie import Magpie
import time
client = Magpie(api_key="your_api_key_here")
# Create a job
job = client.jobs.create(
name="long-task",
script="sleep 60 && echo 'Done!'",
vcpus=1,
memory_mb=512
)
# Poll for status
while True:
status = client.jobs.get_status(job.id)
print(f"Current status: {status}")
if status in ["completed", "failed", "stopped"]:
break
time.sleep(5) jobs.get_logs()
Retrieve the console output logs from a job's execution.
Parameters
| Parameter | Type | Description |
|---|---|---|
| job_id required | str | The unique identifier of the job. |
| tail optional | int | Number of lines to retrieve from the end of the logs. If not specified, returns all logs. |
Returns
Returns a string containing the job's console output. Includes both stdout and stderr streams.
Example
from magpie import Magpie
client = Magpie(api_key="your_api_key_here")
# Get all logs
logs = client.jobs.get_logs("job_abc123xyz")
print("Complete logs:")
print(logs)
# Get only the last 50 lines
recent_logs = client.jobs.get_logs("job_abc123xyz", tail=50)
print("\nLast 50 lines:")
print(recent_logs)
# Monitor logs for a running job
job = client.jobs.create(
name="log-generator",
script="""
for i in {1..10}; do
echo "Processing item $i"
sleep 2
done
""",
vcpus=1,
memory_mb=512
)
import time
time.sleep(5)
logs = client.jobs.get_logs(job.id)
print(f"Current output:\n{logs}") jobs.ssh()
Get SSH connection details for a persistent job, allowing interactive terminal access.
Parameters
| Parameter | Type | Description |
|---|---|---|
| job_id required | str | The unique identifier of the persistent job. |
Returns
Returns an object containing SSH connection details including host, port, username, and command (a ready-to-use SSH command string).
Example
from magpie import Magpie
client = Magpie(api_key="your_api_key_here")
# Create a persistent VM
vm = client.jobs.create(
name="interactive-dev",
script="echo 'VM ready'",
vcpus=4,
memory_mb=8192,
persistent=True
)
# Wait for VM to be ready
import time
while client.jobs.get_status(vm.id) != "running":
time.sleep(2)
# Get SSH connection details
ssh_info = client.jobs.ssh(vm.id)
print(f"Host: {ssh_info.host}")
print(f"Port: {ssh_info.port}")
print(f"Username: {ssh_info.username}")
print(f"\nConnect with:\n{ssh_info.command}")
# Example output:
# Connect with:
# ssh -p 2222 magpie@vm-abc123.magpie.cloud jobs.stop()
Stop a running job immediately. The job status will change to "stopped".
Parameters
| Parameter | Type | Description |
|---|---|---|
| job_id required | str | The unique identifier of the job to stop. |
Returns
Returns a confirmation object indicating the job has been stopped successfully.
Example
from magpie import Magpie
client = Magpie(api_key="your_api_key_here")
# Create a long-running job
job = client.jobs.create(
name="background-task",
script="while true; do echo 'Working...'; sleep 10; done",
vcpus=1,
memory_mb=512,
persistent=True
)
# Work with the job...
import time
time.sleep(30)
# Stop the job when done
result = client.jobs.stop(job.id)
print(f"Job stopped: {result}")
# Verify status
status = client.jobs.get_status(job.id)
print(f"Current status: {status}") # Output: stopped jobs.delete()
Permanently delete a job and all associated data including logs and workspace files.
Parameters
| Parameter | Type | Description |
|---|---|---|
| job_id required | str | The unique identifier of the job to delete. |
Returns
Returns a confirmation object indicating the job has been deleted successfully.
Example
from magpie import Magpie
client = Magpie(api_key="your_api_key_here")
# Create a temporary job
job = client.jobs.create(
name="temp-calculation",
script="echo 'Quick calculation'",
vcpus=1,
memory_mb=512
)
# Wait for completion
job = client.jobs.run_and_wait(
name="temp-calculation",
script="echo 'Quick calculation'",
vcpus=1,
memory_mb=512
)
# Get logs before deletion
logs = client.jobs.get_logs(job.id)
print(f"Final output: {logs}")
# Delete the job
result = client.jobs.delete(job.id)
print(f"Job deleted: {result}")
# Attempting to get the job now will raise an error
try:
client.jobs.get(job.id)
except Exception as e:
print(f"Error: {e}") # Job not found