Remote Servers and Scripting

Real VLSI work happens on remote servers and big compute farms, and the repetitive parts get automated with scripts. This final lesson ties the path to how the job actually runs.

Working on remote machines

ssh you@server.company.com     # log in to a remote Linux server
scp report.rpt you@server:~/   # copy a file to the server
du -sh runs/                   # how big is this directory?
df -h .                        # how much disk space is left here?

Keeping work alive: tmux and screen

If you start a long run over ssh and your connection drops, the run can die with it. tmux (or screen) creates a session that keeps running on the server even after you disconnect, so you can log back in and your run is still there.

tmux              # start a persistent session
# ... start your long run ...
# press Ctrl-b then d to detach (run keeps going)
tmux attach       # reconnect later, run is still alive

Compute farms

Companies run thousands of jobs on shared compute farms managed by a scheduler. Instead of running a tool directly, you submit it to the queue and the scheduler finds a free machine. The exact command varies (LSF uses bsub, Grid Engine uses qsub), but the idea is the same: submit, then check status.

Your first shell script

A shell script is just a file of commands. Put your steps in a file, make it executable, and run it.

#!/bin/bash
# count violations in every report under runs/
for f in runs/*/timing.rpt; do
  count=$(grep -c "VIOLATED" "$f")
  echo "$f : $count violations"
done