Core concepts

What is a Zepl?

Zepl stands for Zip Evaluate Print Loop. It derives from the popular interactive computing concept of a REPL. Just like a REPL, a zepl gives you an interactive environment to run and experiment with your code.

When doing data-science or machine learning, you often need to run your code in the cloud, (to access more CPUs/GPUs or storage for instance). Unweave lets you execute your code in a serverless cloud environment without setting up or configuring any additional resources. A single execution of your code with Unweave is called a zepl.

# Creates a new zepl
unweave python train.py

Zepl Context

When you type unweave python train.py on your terminal, a number of things happen behind the scenes:

  1. The Unweave CLI checks to see if your current directory, or any of its parents are linked to an existing Unweave Project.
  2. It checks for the presence of a requirements.txt file in the current directory.
  3. It uploads your entire directory to the cloud, ignoring any paths listed in the .gitignore file.
  4. The Unweave Orchestrator downloads your data from the uwstore to the base of your project. The uploaded code in step 3 combined with the data in the uwstore folder is called the ZeplContext.
  5. Unweave builds your environment and schedules computing infrastructure to execute the command you entered (python train.py in this case).
  6. Finally, it streams live logs from the zepl execution back to your local terminal.

The ZeplContext defines the environment within which your code runs. For now, we only support Python zepls and require the presence of a requirements.txt file. If you have no Python dependencies to install, you should still provide an empty requirements.txt file.


Data

Machine learning wouldn't cause much FML without the data. Unweave makes handling data trivially easy with the uwstore. The data in the uwstore is accessible to the code you run, at the path ./uwstore/ relative to base of your repository.

To be precise, this is the path linked to your project, which should usually be the base of your repo.

For instance, if you have two images image_a.jpg and image_b.jpg in the uwstore along with a python file train.py, this is what your ZeplContext would look like when you execute unweave python train.py

.
ā”œā”€ā”€ uwstore
ā”‚   ā”œā”€ā”€ image_a.jpg
ā”‚   ā””ā”€ā”€ image_b.jpg
ā””ā”€ā”€ train.py

1 directory, 3 files

In other words, you can load image_a.jpg and image_b.jpg from your train.py script when it executes.

# ./train.py

print("Running train.py script")

img_c = None

with open("./uwstore/image_a.jpg", "r") as img_a:
   with open("./uwstore/image_b.jpg") as img_a:
      img_c = img_a + img_b

with open("./uwstore/image_c.jpg", "w") as out:
   out.write(img_c)

Syncing Data

Any files you save to the ./uwstore/ path are automatically synced back to your uwstore, similar to how they would in Google Drive or Dropbox. Unweave will also snapshot the exact state of your uwstore at the end of each zepl execution. This state is fully reproducible, even if you overwrite the files in your uwstore in subsequent executions!

Supported Environments

Our goal with zepls is for them to be general serverless computation environments. However, for the moment, only Python code execution is supported.

Previous
CLI