How to run jobs on the New York Blue @ Brookhaven National Lab (BNL)
Connection to the Machines
The connection to either of the machine: IBM BlueGene/L or IBM BlueGene/P is made by first connecting to a front node:
- Script to connect to the front node: Edit to your user name and place it on your desktop machine as a script (e.g. change to executable permissions with $chmod a+x and run directly or by placing on /usr/bin).
- Scripts to connect to the BlueGene/L and to the BlueGene/P: Edit to your user name and place them in the front node (once you have successfully connected) (e.g. change to executable permissions with $chmod a+x and run with $./bglssh or $./bgpssh.
Some Comments
- The BG/L gives problems on many compilers, including when running mpicc. I never had the same problem with the BG/P, so I run everything on the P.
- In the other hand, the BG/L has git installed where the BG/P does not, so I use L to version control.
Submitting Batch Jobs
- Instead of running directly (eg. using mpirun -np 16 ./program.x) you should submit it as (several) jobs. Here is an example of a script to submit this jobs.
- Insert this file with the executable and after editing conveniently, type $llsubmit run.scr .
- You should receive some message like this, where 21887 is the JOBNUMBER:
llsubmit: The job "fenpgpfs.bluegene.bnl.gov.21887" has been submitted.
- To verity the status of your job type $llq -b or $llq -b JOBNUMBER.
- To cancel a job type $llcancel JOBNUMBER.
- It is good practice to tail the output and error logs. There will be two files named JOBNUMBER.err and JOBNUMBER.out.
Transferring Files to your local Machine
Other Resources
Getting a Computer Account on New York Blue.
Sample LoadLeveler Batch Job Control File.
One-hop logins and file transfers using ssh tunneling.
Aaron Jackson's about Setting up Remote Access via SSH-Tunneling.
Aaron Jackson's about How to Install VisIt at BNL.
Aaron Jackson's about How to use VisIt with the Visualization Cluster at BNL.