Server Usage

Installing Dependencies

There's nothing to do! The dependencies you need for the Imitation learning lab are already installed in the imitation_learning conda environment.

When you connect, run conda activate imitation_learning).

Connecting to the server over SSH

At the terminal, the following command will SSH into the server, enable x-forwarding, and also forward a port for Jupyter Notebook access:

ssh -L 8888:localhost:8888 -X  $USERNAME@$IP_ADDRESS

Once you are connected, run ls - you have an empty directory for your team already created in the home directory:

$ ls
[snip] team14 team23 team37 team52 team56 team65 [snip] 

After you are SSH-ed in, remember to cd into your team's directory and activate the imitation_learning environment.

Adding a shortcut for easy SSH access:

This step is optional

You can simplify the process of connecting to the server further by adding the following information to your local SSH config file:

Host 6a01
    User $USERNAME
    HostName $IP_ADDRESS
    ForwardX11 yes
    IdentityFile ~/.ssh/id_rsa
    LocalForward 8888 127.0.0.1:8888
    ControlPath ~/.ssh/controlmasters_%r@%h:%p
    ControlMaster auto

This sets up port-forwarding (for Jupyter), SSH multiplexing, and X-forwarding once and for all.

Afterwards you can SSH into the server just by typing:

$ ssh 6a01

at your local command prompt.

Convenient SCP

With the shortcut documented above, you can also SCP directories and files to the server easily, e.g., to copy to your team's directory:

$ scp -r local_directory_w_training_images 6a01:~/teamN/ # where N is your team ID

(Faster) SCP Copying:

scp (secure copy) is a tool used for copying files to or from a remote location over a network. It follows the same format as cp:

scp -r $SOURCE_LOCATION $DESTINATION_LOCATION

where -r specifies recursive usage (i.e., the ability to copy directories and subdirectories).

However, copying each .jpg training image from a source to a destination one-by-one is slow. You might instead want to create a single .zip or .tar.gz file of your directory. For instance, if you are on your local machine, and you want to zip up a folder called data_12_02_2019_8-34/ to copy over to a training laptop:

# format: tar -czvf $DESIRED_FILENAME $DIRECTORY
$ tar -czvf data_12_02_2019_8-34.tar.gz  data_12_02_2019_8-34/

Then, to scp the file over, if you are NOT using the config shortcut documented above:

# replace USERNAME and IPADDRESS
$ scp data_12_02_2019_8-34.tar.gz $USERNAME@IPADDRESS:~/teamN

If you are using the shortcut you can instead run the following command:

$ scp data_12_02_2019_8-34.tar.gz 6a01:~/teamN

Note that the tilde ~ after the : specifies a destination path (teamID) relative to $USERNAME's home directory.

The folder can be expanded on the remote laptop (after ssh-ing in) via:

$ cd ~/teamN
$ tar -xzvf data_12_02_2019_8-34.tar.gz 

No-password logins

If you would like to avoid being asked for a password each time, you can generate a local identity file:

https://help.github.com/en/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent

Then, use the following command to connect to the server from your laptop.

# only required once
$ ssh-copy-id 6a01

which will copy your public key to the server.

Opening Jupyter Notebooks over SSH

Once you have SSHed into the computer, you can start jupyter in your team's folder (include --no-browser so that Firefox does not try to load over SSH - which will be annoyingly slow)

$ cd teamN/
$ conda activate imitation_learning
$ jupyter notebook --no-browser

Jupyter will provide a URL for you to use in your local computer's browser. Copy and paste it into your browser, e.g.,

Or copy and paste one of these URLs:
   http://localhost:8888/?token=r4nd0mh3xstring

$ cd teamN/
$ tmux
$ conda activate imitation_learning
$ jupyter notebook --no-browser