Lab 07 — Jupyter on the cluster: OnDemand vs. SSH tunnel

Goal

Run a Jupyter notebook on Unity (so it has access to cluster data and compute resources) viewed in your laptop’s browser — two ways:

  1. OnDemand — Unity’s web portal that gives you a browser-based Jupyter session with zero setup.
  2. The SSH tunnel — start Jupyter headlessly on a compute node and forward the connection through SSH to your laptop’s browser. More steps, but full control and GPU access.

By the end you’ll have used both and developed an opinion about when to use which.

Reading

Pay particular attention to: - Section 1 (OnDemand), including the /notebook/... → /lab URL hack to upgrade Jupyter Notebook into JupyterLab - Section 2 (the headless approach) - Section 4 (SSH port forwarding demystified — the -L flag explained)

Learning objectives

  1. Launch a Jupyter session via Unity OnDemand and convert it to JupyterLab via the URL hack.
  2. Run a notebook against your eslab mamba env (registered as a kernel in Lab 5).
  3. Set up the headless Jupyter + SSH tunnel approach end-to-end.
  4. Articulate when OnDemand is the better choice and when the tunnel approach is.

Setup / prerequisites

  • Labs 01–05 complete — SSH working, mamba eslab env exists, eslab registered as a Jupyter kernel
  • (Recommended) Lab 04 complete — you can use livenode for the headless approach
  • A browser on your Mac (Safari, Chrome, Firefox — any modern one)

Tasks

1. Launch Jupyter via Unity OnDemand (10 min)

  1. Open https://ondemand.asc.ohio-state.edu in your browser.
  2. Log in with your OSU credentials + one Duo push.
  3. Interactive Apps → Jupyter Notebook.
  4. On the form:
    • Partition: batch (or your lab partition)
    • Number of hours: 1
    • Number of cores: 2
    • Memory: 8 GB (or so)
    • Python version: accept the default — we’ll switch kernels inside Jupyter
  5. Click Launch.

The session appears in My Interactive Sessions with a status. Wait until it’s “Running” (could be ~10s to a few minutes depending on queue).

  1. Click Connect to Jupyter. A new tab opens with the Jupyter Notebook home page.

2. Use the /lab URL hack to upgrade to JupyterLab (5 min)

The OnDemand form only offers “Jupyter Notebook,” but JupyterLab (the fancier interface with a file browser, table of contents, multi-pane layouts) runs in the same session. The trick:

  1. Look at the URL in your browser. It should look something like:

    https://ondemand.asc.ohio-state.edu/node/u500/12345/notebook/tree

  2. Replace /notebook/... with /lab:

    https://ondemand.asc.ohio-state.edu/node/u500/12345/lab

  3. Press Enter. JupyterLab loads in the same browser tab.

You can switch back and forth between /notebook and /lab freely — same kernel, same files, different UI.

3. Create a notebook with the eslab kernel (10 min)

In JupyterLab:

  1. File → New → Notebook
  2. Select Kernel: pick “Python (eslab)” from the list (registered in Lab 5).
  3. Rename the notebook to lab07_ondemand.ipynb (File → Rename Notebook).

In the notebook, run cells like:

import sys, platform, socket
print("Hostname:", socket.gethostname())
print("Python:", sys.version.split()[0])
print("Platform:", platform.platform())

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# A simple computation that uses your env
x = np.linspace(0, 4 * np.pi, 200)
y = np.sin(x) * np.exp(-x / 6)

fig, ax = plt.subplots(figsize=(6, 3))
ax.plot(x, y)
ax.set_title("Damped sine — on a Unity compute node")
ax.grid(True)
plt.show()

The plot appears inline. Save the notebook (Ctrl+S). Take a screenshot for your deliverables.

4. End the OnDemand session (3 min)

Back at OnDemand → My Interactive Sessions → Delete next to your session (or wait for the timer to expire).

5. Now do the headless / SSH-tunnel approach (15 min)

From a VS Code integrated terminal on Unity (or any SSH session):

# Option A: use the livenode pattern you set up in Lab 4
livenode

# Option B (manual): start a compute-node interactive session yourself
sinteractive -p batch --cpus-per-task=2 --mem=8G --time=02:00:00

Once you’re on a compute node — note the hostname (e.g. u500) by running hostname. You’ll need it for the tunnel.

Activate your env and start Jupyter:

mamba activate eslab
jupyter notebook --no-browser --port=8888

Jupyter prints a startup message ending with something like:

Or copy and paste this URL:
    http://localhost:8888/?token=abc123...long-hex-string

Copy that whole URL (or just the token). You’ll paste it into your laptop’s browser in Task 7.

Leave this terminal running. Jupyter only lives as long as this process does.

6. Add a Host block for your compute node (5 min — one-time per node)

If you haven’t already, edit ~/.ssh/config on your Mac (locally) and add a Host block for the compute node you got allocated. Replace u500 with your actual node name:

Host u500
    HostName u500.asc.ohio-state.edu
    User yourname.##
    IdentityFile ~/.ssh/buckai_key
    IdentitiesOnly yes
    HostKeyAlgorithms +ssh-rsa
    PubkeyAcceptedAlgorithms +ssh-rsa
    ProxyJump unity

You already have a generic mynode block from Lab 1 if you did the extension — adapt it to the actual node name.

7. Open the SSH tunnel from your Mac (5 min)

In a new Terminal window on your Mac (NOT inside VS Code, NOT inside another tmux session — a fresh one):

ssh -L 9099:127.0.0.1:8888 u500

(Replace u500 with the actual node name from Task 5.)

This:

  • SSHes you onto u500 (through the jumphost + unity chain set up in Lab 1)
  • Forwards your laptop’s localhost:9099 to u500’s localhost:8888 (where Jupyter is listening)

✅ Leave this terminal connected. The tunnel only exists while it’s alive.

8. Open the notebook in your browser (5 min)

In your Mac browser, visit:

http://127.0.0.1:9099

(That’s port 9099, your local port — not 8888.)

You’ll see a Jupyter login page asking for a token. Paste the token from Task 5 (the long hex string after ?token= in the URL Jupyter printed).

You’re now in JupyterLab/Notebook, served from a compute node via SSH tunnel.

Create a notebook lab07_tunnel.ipynb. Select the Python (eslab) kernel. Run a similar set of cells as Task 3, but include something that proves you’re on a different node:

import socket
print("Hostname:", socket.gethostname())
# Should match the node name you tunneled to (e.g., u500), not the OnDemand one

9. Clean up (3 min)

When you’re done: 1. Save and close the notebook in the browser 2. In the terminal where Jupyter is running: Ctrl+C twice (or q then yes) to stop the server 3. exit the sinteractive session (or it’ll time out naturally) 4. Close the SSH tunnel terminal (Ctrl+D or exit)

Deliverables

Save to lab07/ in your personal repo:

  1. lab07/ondemand_lab.png — screenshot of your OnDemand JupyterLab session (after the /lab URL switch) showing the “Python (eslab)” kernel name and the damped-sine plot.

  2. lab07/tunnel_lab.png — screenshot of the same notebook in your laptop’s browser served via the SSH tunnel, showing the URL http://127.0.0.1:9099 and the cluster hostname output proving it’s on a compute node.

  3. lab07/lab07_ondemand.ipynb and lab07/lab07_tunnel.ipynb — both notebooks.

  4. lab07/reflection.md — 5–7 sentences:

    • What does OnDemand make easier than the SSH-tunnel approach?
    • What does the SSH-tunnel approach let you do that OnDemand can’t? (Hint: think about GPUs and custom envs.)
    • Which would you use day-to-day for your research, and why?

Self-check

Common issues

❌ OnDemand “session is queued, waiting for resources”

Normal during busy hours. Wait, or try batch instead of a lab partition if you have a choice (usually batch has more nodes available, so shorter waits for small sessions).

❌ The /lab URL gives a 404

Check the URL pattern carefully. The path before /notebook must be preserved. Format:

https://ondemand.asc.ohio-state.edu/node/<node>/<port>/lab

❌ “Python (eslab)” kernel doesn’t appear

Lab 5’s python -m ipykernel install --user --name eslab --display-name "Python (eslab)" was skipped or failed. Run it again from a Unity terminal with eslab activated.

❌ Browser at http://127.0.0.1:9099 shows “site can’t be reached”

The SSH tunnel isn’t open. Confirm the ssh -L 9099:127.0.0.1:8888 u500 terminal is still running. If it died, restart it.

❌ Jupyter token-prompt page, but I don’t have the token

Run on the compute node:

jupyter notebook list

It prints the current server URL with the token included.

❌ I can connect, but Jupyter says “kernel won’t start” for Python (eslab)

The kernel spec points to a Python interpreter that doesn’t exist (maybe you renamed/removed the env). Re-register:

mamba activate eslab
python -m ipykernel install --user --name eslab --display-name "Python (eslab)" --force

Time estimate

  • Reading: ~25 min
  • Tasks (OnDemand path): ~25 min
  • Tasks (SSH-tunnel path): ~30 min (more if queue is slow)
  • Deliverables: ~15 min

Total: ~1.5–2 hours

Extensions (optional)

Set up Mac-side aliases for the tunnel

Add to your local ~/.bashrc (or ~/.zshrc on macOS):

alias jup='ssh -L 9099:127.0.0.1:8888 u500'    # adjust node name

Now opening the tunnel is just jup on your Mac. See Handbook §3.

Try the same trick with TensorBoard

Start TensorBoard on Unity (it doesn’t need a compute node — login node is fine for reading TFEvents files):

ssh unity
tensorboard --logdir /fs/project/<group>/runs --port 6006

Then from your Mac:

ssh -L 16006:127.0.0.1:6006 unity

Open http://127.0.0.1:16006 in your browser.

Use livenode to make Jupyter survive disconnects

If you ran Jupyter inside a livenode tmux session (Lab 4), you can detach (Ctrl+b d), close your laptop, come back tomorrow, and reattach — Jupyter is still running with all your variables intact. Just re-open the SSH tunnel from your laptop and refresh the browser.

What’s next?

With Jupyter under your belt, Lab 08 — Your first Slurm batch job moves into unattended batch processing — the workhorse of real HPC research.