Ray Cluster Workflow

Deploy Ray clusters on HPC resources with ACTIVATE.

Quick Start

1. Select your compute resource
2. Choose Ray Version (default: 2.40.0)
3. Enable Submit to Job Scheduler for multi-node clusters (SLURM/PBS)
4. Click Run

The workflow installs Ray, starts the cluster, and opens a session to the Ray Dashboard.

Execution Modes

Mode	Use Case
SSH Direct	Single-node cluster on login node (scheduler disabled)
SLURM	Multi-node cluster across SLURM-allocated nodes
PBS	Multi-node cluster across PBS-allocated nodes

Connecting to Your Cluster

After the workflow completes, access the Ray Dashboard via the session link:

Ray Dashboard Session is ready!
  Proxy URL: https://platform.parallel.works/me/session/...

Submit Jobs

From the Ray head node (same machine running Ray):

source ~/pw/activate-ray/ray_venv/bin/activate
cd ~/pw/activate-ray/examples

# Use absolute path for local files
ray job submit --address=http://localhost:8265 -- python $PWD/hello_ray.py

From the cluster controller (different machine than head node):

source ~/pw/activate-ray/ray_venv/bin/activate

# Use --working-dir to upload your code to Ray
ray job submit --address=http://localhost:8265 --working-dir . -- python hello_ray.py

From your PW user workspace (via tunnel on port 8888):

source ~/pw/activate-ray/ray_venv/bin/activate

# Submit job through the session tunnel (uploads code via --working-dir)
ray job submit --address=http://localhost:8888 --working-dir . -- python your_script.py

From your local machine (via SSH tunnel):

# Set up tunnel first
ssh -L 8888:localhost:8888 -o ProxyCommand="pw ssh --proxy-command %h" $USER@workspace

# Then submit jobs (uploads code via --working-dir)
ray job submit --address=http://localhost:8888 --working-dir . -- python your_script.py

Connect Programmatically

import ray

# From cluster head node
ray.init("ray://localhost:10001")

# Or from workspace/local machine via tunnel
ray.init("ray://localhost:8888")

@ray.remote
def compute(x):
    return x * 2

results = ray.get([compute.remote(i) for i in range(10)])
print(results)

Local Development

Test the scripts locally without the full workflow.

git clone https://github.com/parallelworks/activate-ray.git
cd activate-ray
bash scripts/setup.sh   # Installs Ray (uses uv to auto-install Python 3.12 if needed)
bash scripts/start.sh   # Starts Ray cluster

Access dashboard at http://localhost:8265

Documentation

• Configuration Reference - All input options and settings
• Development Guide - Local testing and development

References

• Ray Documentation
• Ray on SLURM
• ACTIVATE Documentation