If you're new to Unweave, the best place to start is with the Unweave Playground. The Playground is essentially a portal to create and manage Jupyter Lab environments, completely serverlessly and with access to GPUs and persistent storage.
The Unweave CLI and Dashboard are still in development and have many rough edges. If you'd still like to use them, instead of the Playground, we'd love to hear what you think and any feedback you might have. You can reach out to us on Discord, Twitter, or email
The Unweave CLI gives you instant serverless cloud environments to run code from your laptop. All you need to do is prepend
unweave in front of the Python command you would normally run.
For example, if you normally run your training script by executing
python train.py at the terminal, you'll need to run
unweave python train.py instead. Unweave will automatically build and execute your code in the cloud in seconds.
We currently only support MacOS. You can install the CLI with Homebrew:
brew install unweave/unweave/unweave
To update to the latest version of Unweave, run:
brew upgrade unweave
You can get a full list of commands available to use Unweave by running
Login through the CLI
Login on the CLI by running the
unweave login command. This will prompt you to login through the browser using a temporary authorization code.
❯ unweave login ? Do you want to open the browser to login? (Y/n)
Create a new Unweave Project
You can either create a project through the Unweave web Dashboard or through the CLI.
You can create a new project from the Dashboard home page, or if you're already on an existing project page,click on the
+ in the project switcher menu at the top of the page:
Follow the prompts on the side panel and name you project
Link Project to Local Directory
If you create a new project through the Dashboard, you must tell the Unweave CLI where to find it on your local computer. To do this, let's first create a directory we'll use for our ML experiments:
mkdir ml-experiments cd ml-experiments
To link this folder to the project you created, run the
link command from your terminal. This will list all your Unweave projects. Navigate to and select the
ml-experiments project from the list.
❯ unweave link ? Select project [Use arrows to move, type to filter] > ml-experiments
To create a project from the CLI, change into the directory with your ML code and run the
unweave init command.
❯ unweave init ? Enter project name [ml-experiments]:
Press enter to choose the suggested default name (recommended), or enter a new name for your project. This will create a new Unweave project and link it to your current directory.
Execute your First Zepl
Currently, we only support Python projects. For demo purposes, we're going to create a fake Python script to simulate a typical training run. You can find a more realistic workflow in the MNIST example.
Create a new file
train.py with some placeholder content to simulate a training script:
from time import sleep print("Starting training...") for i in range(100): print("Running iteration %d" % i) sleep(1) print("Training complete!")
Try running this locally to make sure it works:
❯ python train.py Starting training... Running iteration 0 ...
Finally the fun part. To run this serverlessly with Unweave, prepend
unweave to the command you just ran:
unweave python train.py
This will build and execute your code in the cloud and spit out logs within a few seconds!
You can upload data to Unweave either through the Dashboard or the CLI. We're going to cover the CLI in this tutorial. For a more detailed guide on how the
uwstore works, check out the Unweave Store and MNIST example pages.
Every zepl running in Unweave automatically has access to the
./uwstore directory relative to the project root. Any data you upload to this directory will be available to use in your Python script. To test this, let's create a new file and save it to the
Downloads directory. We'll then use the
unweave store upload command to upload it to the Unweave Store.
# Create a file with some demo data echo "Hello World" > ~/Downloads/data.txt # Upload to data file to the uwstore unweave store upload ~/Downloads/data.txt
You can verify that the file was uploaded by navigating to the
Data tab of the project page on the Dashboard.
Or by running the
unweave store list command:
❯ unweave store list 2022-08-24 16:39:46 -0700 PDT 12 B /data.txt
We can now modify our
train.py script from above to use the data we just uploaded and write the output back to the
from time import sleep # <<<<<<< New Code >>>>>>> print("Reading data.txt file from ./uwstore/data.txt") with open("uwstore/data.txt", "r") as f: print("Contents: ", f.read()) # <<<<<<< End >>>>>>> print("Starting training...") for i in range(100): print("Running iteration %d" % i) sleep(1) print("Training complete!") # <<<<<<< New Code >>>>>>> print("Witing output.txt file to ./uwstore/output.txt") with open("uwstore/output.txt", "w") as f: print("Contents: ", f.write("Goodbye world")) # <<<<<<< End >>>>>>>
Finally, we can run the same command as before to execute the script:
unweave python train.py
You can verify that the output file was uploaded by running the
unweave store list command again:
❯ unweave store list 2022-08-24 16:39:46 -0700 PDT 12 B data.txt 2022-08-24 16:55:00 -0700 PDT 13 B output.txt
d8b15d23-afcf-4f3a-a1d7-b9952d6de27e string in the output file path is the ID of the zepl that produced this file. Unweave snapshots the output of each zepl automatically, allowing you to version each training run out-of-the-box.
You can download data from the
uwstore using the
unweave store download command. Using this command you can download a single object or an entire directory to your local
# Download a single object ❯ unweave store download output.txt output.txt saved to ./uwstore/output.txt # Download entire directory ❯ unweave store download output ..