Usage¶

To use resen, simply enter resen at the command line:

$ resen

This will open the resen tool:

    ___ ___ ___ ___ _  _
   | _ \ __/ __| __| \| |
   |   / _|\__ \ _|| .` |
   |_|_\___|___/___|_|\_|

Resen 2020.2.0 -- Reproducible Software Environment

[resen] >>>

Type help to see available commands:

[resen] >>> help

This will produce a list of resen commands you will use to manage your resen buckets:

Documented commands (type help <topic>):
========================================
EOF     exit    help    list  remove  status
create  export  import  quit  start   stop

To get more information about a specific command, enter help <command>.

Resen Workflow¶

To create, import, export, and remove buckets, we use Resen. Buckets are portable, system independent environments where code can be developed and run. Buckets can be shared between Windows, Linux, and macos systems and all analysis within the bucket will be run exactly the same. Resen buckets come preinstalled with a variety of common geospace software that can be used immediately in analysis.

The interface to a resen bucket is a jupyter lab server and access to the bucket is provided through a web browser. The user home directory is /home/jovyan. Any mounted storage directories are available in mount.

Below are instructions on how to use Resen to work with buckets. A typical workflow will involve: creation of a bucket, performing scientific data analysis inside the bucket, exporting the bucket and sharing it with colleagues. Collaborators can then import the bucket, perform additional analysis, and then export the bucket for publication in an open access citable repository, such as Zenodo.

Setup a New Bucket¶

Creating a new bucket is performed with the command:

[resen] >>> create

The create command queries the user for several pieces of information required to create a bucket. First it asks for the bucket name. Creating a bucket named amber:

Please enter a name for your bucket.
Valid names may not contain spaces and must start with a letter and be less than 20 characters long.``
>>> Enter bucket name: amber

Next, the user is asked to specify the version of resen-core to use:

Please choose a version of resen-core.
Available versions: 2019.1.0, 2020.1.0, 2020.2.0
>>> Select a version: 2020.2.0

Optionally, one may then specify a local directory to mount into the bucket at /home/jovyan/mount:

Local directories can be mounted to either /home/jovyan/work or /home/jovyan/mount/ in
a bucket. The /home/jovyan/work location is a workspace and /home/jovyan/mount/ is intended
for mounting in data. You will have rw privileges to everything mounted in work, but can
specify permissions as either r or rw for directories in mount. Code and data created in a
bucket can ONLY be accessed outside the bucket or after the bucket has been deleted if it is
saved in a mounted local directory.
>>> Mount storage to /home/jovyan/mount? (y/n): y
>>> Enter local path: /some/local/path

Finally, the user is asked if they want jupyterlab to be started:

>>> Start bucket and jupyterlab? (y/n): y

after which resen will begin creating the bucket. Example output for a new bucket named amber with jupyterlab started is:

...adding core...
...adding ports...
...adding mounts...
Bucket created successfully!
...starting jupyterlab...
Jupyter lab can be accessed in a browser at: http://localhost:9002/?token=e7a11fc1ea42a445807b4e24146b9908e1abff82bacbf6f2

Check the status of the bucket:

[resen] >>> status amber

amber
=====

Resen-core Version:  2020.2.0
Status:  running
Jupyter Token:  e7a11fc1ea42a445807b4e24146b9908e1abff82bacbf6f2
Jupyter Port:  9002
Jupyter lab URL: http://localhost:9002/?token=e7a11fc1ea42a445807b4e24146b9908e1abff82bacbf6f2

Storage:
Local                                   Bucket                                  Permissions
/some/local/path                        /home/jovyan/mount/path                 rw

Ports:
Local          Bucket
9002           9002

At this point, the bucket should have a name, an image, at least one port, and optionally one or more storage locations. Status should be running if the user decided to have jupyterlab started, otherwise the status will be None.

Work with a Bucket¶

Check what buckets are available with list:

[resen] >>> list
Bucket Name         Docker Image             Status
amber               2020.2.0                 running

If a bucket is running, it will consume system resources accordingly.

Stop the bucket amber:

[resen] >>> stop amber

The status of amber should now be exited:

[resen] >>> list
Bucket Name         Docker Image             Status
amber               2020.2.0                 exited

The bucket will still exist and can be restarted at any time, even after quitting and restarting resen.

Start the bucket amber that was just stopped:

[resen] >>> start amber

The status of amber should now be running:

[resen] >>> status
Bucket Name         Docker Image             Status
amber               2020.2.0                 running

Export bucket amber:
```
[resen] >>> export amber
```

The export command will ask a series of question. First, provide a name for the output *.tar file:
>>> Enter name for output tar file: /path/for/output/amber.tar
If desired, change the default name and tag for the exported image:
By default, the output image will be named "amber" and tagged "latest".
>>> Would you like to change the name and tag? (y/n): y
>>> Image name: custom_name
>>> Image tag: custom_tag
Specify if you want all mounted directories to be included in the exported bucket. Answering n to this query will allow you to see how large each mount is and specify which you would like to include. Consider excluding any mounts that are not necessary for the analysis to reduce the size of the output file:
The following local directories are mounted to the bucket (total 2212 MB):
/home/usr/mount1
/home/usr/mount2
/home/usr/mount3
>>> Would you like to include all of these in the exported bucket? (y/n): n
>>> Include /home/usr/mount1 [154.68095 MB]? (y/n): y
>>> Include /home/usr/mount2 [2005.28493 MB]? (y/n): y
>>> Include /home/usr/mount3 [53.59823 MB]? (y/n): y
Confirm that you want to continue with the export. The values shown should be considered a “high-side” approximation and may not be the actual final size:
This export could require up to 13337 MB of disk space to complete and will produce an output file up to 4600 MB.
>>> Are you sure you would like to continue? (y/n): y
Exporting bucket amber.  This will take several minutes.
If a full path is not provided for the output file name, the default location for the output file is whatever directory resen was started in. For example, if you start resen in ~\Desktop\MyStuff and respond to the first prompt with new_bucket, the output tar file will be ~\Desktop\MyStuff\new_bucket.tar.

Import a new bucket, amber2, from a tar file amber.tar:
```
[resen] >>> import
```

This command will also ask a series of questions. First provide a name for the imported bucket:
Please enter a name for your bucket.
Valid names may not contain spaces and must start with a letter and be less than 20 characters long.
>>> Enter bucket name: amber2
Specify the *.tar file to import the bucket from:
>>> Enter name for input tar file: /path/to/file/amber.tar
If desired, enter a custom image name and tag. If not provided, the name an image saved on export will be used:
>>> Would you like to keep the default name and tag for the imported image? (y/n): n
>>> Image name: amber2
>>> Image tag: new_tag
When a bucket that had mounts is imported, the mounted directories must be extracted and saved on the local machine. Resen will do this automatically, but you have the option to specify where these files should be saved instead of the default location:
The default directory to extract the bucket metadata and mounts to is /default/save/path/resen_amber2.
>>> Would you like to specify and alternate directory? (y/n): y
>>> Enter path to directory: /new_save_path
Aside from the existing mounts, you can add new mounts to a imported bucket. This is useful if you would like to repeat the analysis with a different dataset:
>>> Mount additional storage to the imported bucket? (y/n): y
>>> Enter local path: /new/local/path/new_mount
>>> Enter bucket path: /home/jovyan/mount/new_mount
>>> Enter permissions (r/rw): r
>>> Mount additional storage to /home/jovyan/mount? (y/n): n
Similar to create, you have the option to start jupyter lab immediately after the bucket is imported:
>>> Start bucket and jupyterlab? (y/n): y
...starting jupyterlab...
Jupyter lab can be accessed in a browser at: http://localhost:9003/?token=70532767bab0ddc4febe2790efaaf974961e961e78e6025a

Sudo-enabled buckets¶

When starting a bucket with resen, sudo is enabled for the jovyan user to allow special installation and configuration where root security privileges are needed. The password for running sudo commands with user jovyan is: ganimede.

Remove a Bucket¶

WARNING: This will permanently delete the bucket. Any work that was not saved in a mounted storage directory or downloaded from the bucket will be permanently lost.

The user can delete a bucket with the following command:

[resen] >>> remove amber

A bucket that is running needs to be stopped before being removed.