Zepl stands for Zip Evaluate Print Loop. It derives from the popular interactive computing concept of a REPL. Just like a REPL, a zepl gives you an interactive environment to run and experiment with your code.
When doing data-science or machine learning, you often need to run your code in the cloud, (to access more CPUs/GPUs or storage for instance). Unweave lets you execute your code in a serverless cloud environment without setting up or configuring any additional resources. A single execution of your code with Unweave is called a zepl.
# Creates a new zepl unweave python train.py
When you type
unweave python train.py on your terminal, a number of things happen behind the scenes:
- The Unweave CLI checks to see if your current directory, or any of its parents are linked to an existing Unweave Project.
- It checks for the presence of a
requirements.txtfile in the current directory.
- It uploads your entire directory to the cloud, ignoring any paths listed in the
- The Unweave Orchestrator downloads your data from the
uwstoreto the base of your project. The uploaded code in step 3 combined with the data in the
uwstorefolder is called the ZeplContext.
- Unweave builds your environment and schedules computing infrastructure to execute the command you entered (
python train.pyin this case).
- Finally, it streams live logs from the zepl execution back to your local terminal.
The ZeplContext defines the environment within which your code runs. For now, we only support Python zepls and require the presence of a
requirements.txt file. If you have no Python dependencies to install, you should still provide an empty
Machine learning wouldn't cause much FML without the data. Unweave makes handling data trivially easy with the
uwstore. The data in the
uwstore is accessible to the code you run, at the path
./uwstore/ relative to base of your repository.
To be precise, this is the path linked to your project, which should usually be the base of your repo.
For instance, if you have two images
image_b.jpg in the
uwstore along with a python file
train.py, this is what your ZeplContext would look like when you execute
unweave python train.py
. ├── uwstore │ ├── image_a.jpg │ └── image_b.jpg └── train.py 1 directory, 3 files
In other words, you can load
image_b.jpg from your
train.py script when it executes.
# ./train.py print("Running train.py script") img_c = None with open("./uwstore/image_a.jpg", "r") as img_a: with open("./uwstore/image_b.jpg") as img_a: img_c = img_a + img_b with open("./uwstore/image_c.jpg", "w") as out: out.write(img_c)
Any files you save to the
./uwstore/ path are automatically synced back to your
uwstore, similar to how they would in Google Drive or Dropbox. Unweave will also snapshot the exact state of your
uwstore at the end of each zepl execution. This state is fully reproducible, even if you overwrite the files in your
uwstore in subsequent executions!
Our goal with zepls is for them to be general serverless computation environments. However, for the moment, only Python code execution is supported.