AST 443: Notes on Computing

You will all soon have accounts on kirk, vulcan, uhura, and some of the workstations in the Astronomy Cluster. Of these, the names machines have IDL 8.7 installed.

Your username is first_initial+last_name.

Our system manager, Doug Swesty, has mailed you a password on your SBU email account. Please do not lose or share your password.

Accessing the Astronomy Cluster

We have limited seating in the Astronomy Computer lab, so to ensure social distancing, we recommend you log in remotely using an ssh client. ssh is available on most UNIX platforms (Mac OS runs unix). The syntax is ssh yourname@computername.astro.sunysb.edu. where yourname is your user name on the astronomy cluster. computername will be either uhura or vulcan. If you can ssh in, but cannot generate a graphics window, try using the -Y option in ssh (ssh -Y yourname@computername.astro.sunysb.edu).

To log in securely from a Windows platform, you will need to install putty, a free ssh client. Putty is available from a number of websites, including putty.org and chiark.greenend.org.uk. This site includes a separate link to pscp, an scp emulator. You will also need an X-server such as Cygwin/X, on your laptop. Refer to the Putty documentation for more information.

Please try to log into kirk.astro.sunysb.edu using your password.

Once you are able to log in, please change your password using the passwd command. Only change your password on kirk! - the change will propagate to the other machines on the cluster in a few minutes. Then you can logout and log into vulcan or uhura using your reset password. We will talk about password protocol and system security on Monday. If you have problems logging in, please let me know by e-mail, and we'll try to clear it up on Monday.

Doug Swesty has a 12 video series about how to use the Linux command line, and a 4 video series on editing with Emacs, on his YouTube channel.

Where to log in?
We have 12 IDL licenses for the cluster, so you cannot all be running IDL at the same time. And performance drops the more users are logged in. Once you are logged in, use the finger command to see who else is logged on.

vulcan and uhura are the most capable machines in the cluster, and should be capable of handling up to 6 users at a time. kirk can handle 2-3 users.

If you have heavy-duty computing to do, or want faster response, consider logging in at off-hours.

Modifying System Defaults

Once you have successfully logged in there are some things you need to do.

You will need to modify your .cshrc or .bashrc file, depending on what shell you are using. (To find out which shell you are running, type:
echo $0 ).
Use chsh to change shells. e.g., chsh /bin/tcsh changes your shell to the C shell.

If using the C shell, insert the following lines into your .cshrc file:

If using the bash shell, insert the following lines into your .bashrc file:

Useful (?) Information

UNIX commands

Navigation
When you log in, you are in your home directory /home/username where username is the name you used to log in with. Your actual home directory is on kirk; for various reasons we wipe the home disk on vulcan nightly. DO NOT WRITE FILES YOU INTEND TO KEEP ON VULCAN! Write files instead to /kirk/home/username. If you are logged into uhura, then no worries - the kirk and uhura home disks are the same.

It is adviable to organize your data into subdirectories in your home directory. Create them with mkdir. Use subdirectories to organize your files. Use the cd command to change directories. Don't know where you are? Type pwd.

UNIX is case-sensitive. ls and LS are two different commands.

There are many variants of UNIX. I use the C-shell. Type tcsh to use this shell. If you want to use the bash shell, or any other shell, you are on your own!

You can run any UNIX command in the background by adding an ampersand (&) to then end of the command line. For example, to run an editor in the background, so you can use the terminal window for other things, type emacs &.

Logging Out
When you are done, please log out.

Running IDL

IDL should be in your path. If the command which idl returns the string /usr/local/bin/idl, or something similar, you are in business. Otherwise, you will have to find the location of the IDL executable and add it to your path.

Start by typing idl. Need help? Type a question mark (followed by a return). This brings up the help manual (a pdf file). Read this at your leisure.

A better manual is Fanning's "IDL Programming Techniques". this is a large pdf file - please do not print it out.

There are a number of good web sites devoted to IDL programming. Among these are:

But before you go to any of the above-mentioned sites, read the following primer first.

Using IDL

A primer on IDL

IDL is a powerful language, with many intrinsic capabilities. It can be run interactively, line by line, from the terminal, or you can create programs (called procedures) which you can then run. The syntax is very similar to FORTRAN, so a user familiar with FORTRAN should have little difficulty coming up to speed in IDL (however, it takes years to become expert). Unlike Fortran or C, IDL code need not be compiled and linked. You can stop a procedure anywhere in the code, and examine intermediate results, change values, or do whatever you might think of doing. The best advice is, as often, to read the online manuals before you begin.

To run IDL, type idl.

Note that, unlike UNIX, IDL is not case-sensitive. However, file names are, because IDL must interact with the operating system to find files. The IDL commands that follow are capitalized to distinguish them from the text; they need not be.

IDL supports both procedures (stand-alone programs) and functions (procedures that return a value).The syntaxes are slightly different.

Note that both procedures and function must have .pro filename extensions.

IDL recognizes a lot of variable types, including

Once you start IDL, you should check your default directory. One way to find where you are is to type
cd,CURRENT=CURRENT & PRINT,CURRENT. If you are not where you want to be, you can use the cd command. cd is like the UNIX cd command, but it needs a string argument, as in cd,'new_dir'.

In addition to the intrinsic capabilities there exists a large and growing library of user-supplied routines. The most useful for our purposes is the Astronomy Users Library. These routines are in your IDL_PATH. A brief description of the contents can be found at the IDL Astronomy Users Library website.

For the most part, you will only need primitive IDL commands for displaying and measuring your data. The data files are generally in FITS format, which you can easily read in IDL by following these instructions. The data will then consist of a header H (a string array) and a data array D, which may be multi-dimensional.

Pointers on Using IDL

Pointers on IDL graphics

Acquaint yourself with the PLOT and TV commands. Use PLOT for line plots; use TV to display images.

The simplest graphics/display software in idl is written for single-plane, 256 color graphics. I don't know whether of not the Suns can deal with 24-bit color (true colors). If your plots come up black, or you can't get color tables to work, try DEVICE,DECOMPOSED=0. Feel free to experiment with colors.

Unpacking Dimensions

It is easy to extract M-dimensional data from N-dimensional data. For example, given a 2-dimensional array of spectra D (as you get from the IUE high dispersion data), you can extract the nth row with the command S=D[n,*], or the nth column with the command S=D[*,n]

Reading FITS Files

Read FITS files with READFITS. READFITS, a function, returns a parameter (the data array). To read a file, type
data = readfits("my_fits_file.fits",header)
Data is an array containg the data; header is a string array containing the FITS header. The fits file name is a string, and must be enclosed in quotes (unless it is a string variable). Use SXPAR to extract values from the fits header.

READFITS can read gzipped file (those with .fits.gz extensions): there is no need to uncompress the file first with gunzip.

More information about FITS files and reading FITS extensions is here.

Plotting and Measuring Data

For one-dimensional data (e.g., spectra), use the PLOT command. Set the plot scale and range with SET_XY.

Two-dimensional data can be displayed using the TV. Note that the window displays byte values, and PLOT does not do any byte-scaling. Use TVSCL,data or TV,BYTSCL(data) to scale your data appropriately. Note that, if you have a few pixels that are very much brighter than the others, you may not see much. This is because the full range is compressed into about 256 intensity levels. Use the < operator to truncate the image (e.g., TVSCL,data<500 will plot all pixels with values >500 as 500. Used in this manner, the operator does not overwrite the data.).

Using the Cursor to Measure Positions

The basic way to measure a value on the screen is with the CURSOR command. Type CURSOR,X,Y. Click the left mouse button at the point of interest, and then PRINT,X,Y to get the values back. On a 2-dimensional image, X and Y will be the pixel number, and the value at that point is IMAGE[X,Y].

In order to use the CURSOR command for images, you need to set up the plot limits properly (there may be more elegant methods, but this works!).

s=size(image)                  ;this returns a vector containing the size
                               ;of the image. s[0] is the number of dimensions,
                               ;which should be 2 for an image array
sx=s[1]                        ;size of X dimension
sy=s[2]                        ;size of Y dimension
window,xsize=sx,ysize=sy       ;create window the size of the image
!p.position=[0,0,1,1]          ;plot boundaries are window boundaries - there
                               ; are no margins
set_xy,0,sx-1,0,sy-1           ;set plot limits
!x.style=1 & !y.style=1        ; force exact plot boundaries
plot,indgen(10)                ;this dummy plot will define the plot limits
tvscl,image                    ; display image
x=sx/2 & y=sy/2                ; initialize to center of plot
cursor,x,y                     ; use cursor command to get position

Problems

On occasion, a procedure will encounter an error and stop. You will know this has happened when you see an error message followed by a traceback like
% Attempt to subscript K with  is out of range.
% Execution halted at:  DBHELP            192
  /data/gremlin/fwalter/astrolib/pro/database/dbhelp.pro
%                       $MAIN$
When this happens, you can either try to fix the problem, or bail out. The latter is generally recommended unless you know what you are doing. To bail out, type RETALL. This will close the procedure and return to the main level.

Python

TBD

Other programs under UNIX

Accessing Data

Shared data will be placed in world-readable location, to be determined. Use cp to copy the data to your own directory, or scp to copy to another computer.

Back to top