This example assumes we are running a job via the LoadLeveler batch system on the blackforest.ucar.edu complex and would like to transfer our data after completion on blackforest to chinookfe.ucar.edu.
First, we generate our key pair on the blackforest complex, running the "ssh-keygen" command, as we did in our local workstation example, see section 3.1:
[bf0915en:/home/blackforest/wombat] $ ssh-keygen -t dsa Generating public/private dsa key pair. Enter file in which to save the key (/home/blackforest/wombat/.ssh/id_dsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/blackforest/wombat/.ssh/id_dsa. Your public key has been saved in /home/blackforest/wombat/.ssh/id_dsa.pub. The key fingerprint is: 46:a2:d2:a5:c3:ff:dd:12:54:ad:8a:98:b2:a2:9b:5d wombat@bf0915en
We then copy our public key file, "id_dsa.pub" to our ".ssh" directory on chinookfe.ucar.edu as "authorized_keys2":
[bf0915en:/home/blackforest/wombat] $ scp .ssh/id_dsa.pub chinookfe.ucar.edu:.ssh/authorized_keys2 The authenticity of host 'chinookfe (128.117.215.218)' can't be established. DSA key fingerprint is 46:a2:d2:a5:c3:ff:dd:12:54:ad:8a:98:b2:a2:9b:5d Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'chinookfe,128.117.215.218' (DSA) to the list of known hosts. wombat@chinookfe's password: id_dsa.pub 100% |********************| 607 00:00
We now "cd" to our working directory "/home/blackforest/wombat/dscp" on blackforest:
[bf0915en:/home/blackforest/wombat] $ cd dscp [bf0915en:/home/blackforest/wombat/dscp] $
We now start an "ssh-agent" to handle storing our key data in memory on the node. We pipe the associated startup information to a file, called "agent-info", modify the file, to make it executable, and source it:
[bf0915en:/home/blackforest/wombat/dscp] $ ssh-agent | head -2 > agent-info [bf0915en:/home/blackforest/wombat/dscp] $ chmod 700 agent-info [bf0915en:/home/blackforest/wombat/dscp] $ . ./agent-info
Sourcing the "agent-info" file is done above using ". ./agent-info". This applies to users of "ksh", other users would need to type "source ./agent-info".
Sourcing the "agent-info" file should provide environment settings like:
[bf0915en:/home/blackforest/wombat/dscp] $ env | grep SSH SSH_AUTH_SOCK=/tmp/ssh-QUL47772/agent.47772 SSH_AGENT_PID=27674
We would then want to add our keys to memory, using "ssh-add":
[bf0915en:/home/blackforest/wombat/dscp] $ ssh-add /home/blackforest/wombat/.ssh/id_rsa: No such file or directory Enter passphrase for /home/blackforest/wombat/.ssh/id_dsa: Identity added: /home/blackforest/wombat/.ssh/id_dsa (/home/blackforest/wombat/.ssh/id_dsa) [bf0915en:/home/blackforest/wombat/dscp] $ ssh-add -l 1024 `long hex string' /home/blackforest/wombat/.ssh/id_dsa (DSA)
Once the keys are added to memory, we can submit our batch job via LoadLeveler. For efficient use of blackforest, we submit a two step LoadLeveler job. In this example, the first step is the parallel calculation, and the second (serial) step uses the interactive class to "scp" a file "mydata" to chinookfe. The LoadLeveler script looks like this:
#!/bin/csh #llscript: Submit using 'llsubmit llscript' on the IBM SP # @ job_name = scp_test # @ wall_clock_limit = 1200 # @ output = $(job_name).$(step_name).out # @ error = $(job_name).$(step_name).err # @ step_name = parallel_step # @ job_type = parallel # @ executable = /usr/bin/poe # @ arguments = /usr/bin/date # @ node = 8 # @ total_tasks = 8 # @ class = share # @ queue # @ step_name = copy_step # @ dependency = (parallel_step == 0) # @ job_type = serial # @ requirements = (Machine == "$(hostname)") # @ executable = ~/copy_step # @ class = interactive # @ queue
"llscript" in the last step/phase invokes a file called "copy_step", which is the serial "scp" portion of the batch job; it contains:
#!/usr/bin/ksh
#
echo 'copy step'
echo ' '
agent=./agent-info
if [ -x ${agent} ]
then
. ./${agent}
else
echo "Can't find or read agent file; exiting."
exit 1
fi
scp ./mydata chinookfe.ucar.edu:mydata
ssh-agent -k
rm -f ~/dscp/agent-info
The first section of "copy_step" sources the "agent-info" file we created when we started the "ssh-agent" process. Since "ssh-agent" detaches, we can log out of blackforest and allow the system to process our batch job as resources become available. The last three lines of the "copy_step" script basically execute "scp" to copy the file "mydata" to chinookfe.ucar.edu. It then kills the running "ssh-agent" process and removes the "agent-info" file.
NOTE: You should use passphrases which are unique compared to any existing passphrases you may have. In addition, you are strongly encouraged to kill the "ssh-agent" process via "ssh-agent -k" after completion of your batch work as the last step in the process.
If you have questions about this document, please contact SCD Customer Support. You can also reach us by telephone 24 hours a day, seven days a week at 303-497-1278. Additional contact methods: consult1@ucar.edu and during business hours in NCAR Mesa Lab Suite 39.