Runaway

A simple tool to help you in deploying experimental campaigns to high-performance clusters.

Runaway is a command line-tool that allows you to launch a significant number of simultaneous executions of your code on a remote resource, with one single call. Whatever the scale and the resource, results are one command away from your laptop.

To download the latest binary:

$ curl https://gitlab.inria.fr/runaway/runaway/-/wikis/releases/0.4.0/x86_64_linux_gnu/runaway-cli

First install? Checkout the install section of the documentation.

md5: 8cb16bf052348784d1380f2851819062

Remote execution in one line

Runaway synchronizes your code with the remote cluster, allocates the necessary resources, executes and fetches data automatically.

Terminal

$> runaway exec localhost train.py -- --dry-run --epoch=1
  runaway: Loading host
  runaway: Reading arguments
  runaway: Reading ignore files
  runaway: Compress files
  runaway: Acquiring node on the host
  runaway: Transferring data
  runaway: Extracting data in remote folder
  runaway: Removing archive
  runaway: Executing script
  
  Training model
  
  ...
  
  runaway: Compressing data to be fetched
  runaway: Transferring data
  runaway: Extracting archive
  runaway: Cleaning data on remote
$> 

Experimental campaign made simple

Want to run a batch of experiments for your campaign? The batch subcommand is here to help. Only a file containing the different arguments is needed.

Terminal

$> runaway batch localhost train.py -A parameters.txt
  runaway: Loading host
  runaway: Reading arguments
  runaway: Reading ignore files
  runaway: Compress files
  runaway: Transferring data
  runaway: Starting execution with arguments "--lr=0.1"
  runaway: Starting execution with arguments "--lr=0.01"
  
  10eb724a-e9b4-441b-bbe2-e197a11d628d: Training model 
  da177f52-bb68-4799-8d82-6b511d557181: Training model
  
  ...
   
  runaway: Cleaning data on remote
$> 

Adapt to your workflow easily

Hyper-parameters optimization, rescheduling, campaign analysis... All use cases can not be solved by running batches of executions. Runaway allows you to harness its power with your own logic, through its scheduling command.

Terminal

$> runaway sched my-cluster train.py "./sched.py --reschedule"
  runaway: Loading host
  runaway: Reading arguments
  runaway: Reading ignore files
  runaway: Compress files
  runaway: Transferring data
  runaway: Querying the scheduler
  runaway: Starting execution with arguments "--lr=0.1 --no-augment "
 
  31908337-10bc-485a-806c-e3ef38cf6d1e: Training model
  ...
  
  runaway: Querying the scheduler
  runaway: Starting execution with arguments "--lr=0.01 "
  runaway: Starting execution with arguments "--lr=0.1 "
  runaway: Querying the scheduler
   

Simple

Runaway is a simple command-line tool, meant to follow the unix philosophy. In particular, it stays out of your code, and does not require a full rewrite of your experiments to be used.

Versatile

Instead of using a target-specific agent on the remote end, Runaway uses a template system, which can be adapted to a wide range of resources. University cluster, cloud resources, or shared computer in the office, all of them can be adapted writing a bunch of bash commands.

Scalable

Thanks to its asynchronous handling of executions, it can easily scale to manage a large amount of simultaneous executions. If you have a large campaign in mind, Runaway can handle it.