The PyRAF Tutorial

Contents

• Introduction
  • Python Prerequisites
  • IRAF Prerequisites
• Starting and Using PyRAF Interactively
  • New Capabilities in PyRAF
  • Differences from the CL and Unimplemented CL Features
• The EPAR Parameter Editor
  • Action Buttons
  • Menu Bar
  • Parameter Editing Panel
  • Status Line
• PyRAF Graphics and Image Display
  • Printing Graphics Hardcopy
  • Multiple Graphics Windows
  • Other Graphics Devices
• Running Tasks in Python Mode
  • The PyRAF Interpreter Environment
  • Example with Standard Python Syntax
  • Setting IRAF Task or Package Parameters
  • Other Ways of Running IRAF Tasks

Introduction

Python Prerequisites

In any case, do not spend too much time reading before actually starting to run PyRAF and use Python. The Python environment is an interactive one and one can learn quite a bit by just trying different things after a short bit of reading. In fact, you may prefer to try many of the examples given in this guide before reading even a single Python document. We will give enough information for you to get started in trying the examples. Our approach will be largely based on examples.

IRAF Prerequisites

Starting and Using PyRAF Interactively

% pyraf 
setting terminal type to xterm...
 
    NOAO Sun/IRAF Revision 2.11.2 Wed Aug 11 13:24:18 MST 1999
    This is the EXPORT version of Sun/IRAF V2.11 for SunOS 4 and Solaris 2.7
 
    Welcome to IRAF.  To list the available commands, type ? or ??.  To get
    detailed information about a command, type `help command'.   To  run  a
    command  or  load  a  package,  type  its name.   Type  `bye' to exit a
    package, or `logout' to get out of the CL.   Type `news'  to  find  out
    what is new in the version of the system you are using.   The following
    commands or packages are currently defined:
.
.
.
--> imheader dev$pix long+
dev$pix[512,512][short]: m51  B  600s
No bad pixels, min=-1., max=19936.
Line storage mode, physdim [512,512], length of user area 1621 s.u.
Created Mon 23:54:13 31-Mar-1997, Last modified Mon 23:54:14 31-Mar-1997
Pixel file "HDR$pix.pix" [ok]
'KPNO-IRAF'           /
'31-03-97'            /
IRAF-MAX=           1.993600E4  /  DATA MAX
IRAF-MIN=          -1.000000E0  /  DATA MIN
IRAF-BPX=                   16  /  DATA BITS/PIXEL
IRAFTYPE= 'SHORT   '            /  PIXEL TYPE
CCDPICNO=                   53  /  ORIGINAL CCD PICTURE NUMBER
.
.
.
HISTORY '24-04-87'
HISTORY 'KPNO-IRAF'           /
HISTORY '08-04-92'            /
--> imstat dev$pix
#               IMAGE      NPIX      MEAN    STDDEV       MIN       MAX
              dev$pix    261626     105.1     75.86       -1.     1000.
--> imcopy dev$pix mycopy.fits
dev$pix -> mycopy.fits
--> stsdas
 
 
      +------------------------------------------------------------+
      |       Space Telescope Science Data Analysis System         |    
      |          STSDAS Version 2.1, September 29, 1999            |
      |                                                            |
      |   Space Telescope Science Institute, Baltimore, Maryland   |
      |         For help, send e-mail to help@stsci.edu            |
      +------------------------------------------------------------+
 
 
stsdas/:
 analysis/      examples        hst_calib/      sobsolete/
 contrib/       fitsio/         playpen/        toolbox/
 describe       graphics/       problems
--> fitsio
--> catfits mycopy.fits
EXT#  FITSNAME      FILENAME              EXTVE DIMENS     BITPI OBJECT         
 
0     mycopy.fits                               512x512    16    m51  B  600s
--> imhead dev$pix long+ > devpix.header
-->

You can start up PyRAF from any directory; unlike the IRAF CL, you are not required to change to your IRAF home directory. PyRAF determines the location of your IRAF home directory by looking for your login.cl file, first in your current working directory and then in a directory named ~/iraf. So as long as your IRAF home directory is ~/iraf, you can start up PyRAF from any working directory. (You can start from other directories as well, but without access to login.cl your IRAF environment will be only partly initialized. We expect to add a startup configuration file, .pyrafrc, that allows you customize your initial PyRAF configuration including your IRAF home directory.)

The first time you run PyRAF, it creates a pyraf directory in your IRAF home directory. At the moment all it stores there is a directory named clcache, which is used to save translated versions of your own custom CL scripts. (The files in that directory have cryptic names that actually encode the contents of the corresponding CL scripts, thus allowing the translation to be used regardless of the CL script name.)

Note that the task syntax shown above is identical to that of the IRAF CL. But there is no escaping that you are really running in a Python environment. Should you make a mistake typing a task name, for example,

--> imstart dev$pix
  File "", line 1
    imstart dev$pix
              ^
SyntaxError: invalid syntax

--> =cl.menus
  File "", line 1
    =cl.menus
    ^
SyntaxError: invalid syntax

Aside from some noticeable delays (on startup, loading graphics modules, or in translating CL scripts not previously encountered), there should be little difference between running IRAF tasks in CL emulation mode and running them in the IRAF CL itself.

New Capabilities in PyRAF

Several capabilities in the PyRAF interpreter make it very convenient for interactive use. The up-arrow key can be used to recall previous commands (no need to type ehis!), and once recalled the left and right arrow keys can be used to edit it. The control-R key does pattern-matching on the history. Just type part of the command (not necessarily at the beginning of the line) and you'll see the matched command echoed on the command line. Type ^R again to see other matches. Hit return to re-execute a command, or other line-editing keys (left/right arrow, ^E, ^A, etc.) to edit the recalled command. There are many other ways to search and manipulate the history -- see the gnu readline documentation for more information.

The tab key can be used to complete commands, in a way familiar to users of tcsh and similar shells. At the start of the command line, type imhe<tab> and PyRAF fills in imheader. Then type part of a filename <tab> and PyRAF fills in the rest of the name (or fills in the unambiguous parts and prints a list of alternatives). This can be a great timesaver for those long HST filenames! You can also use tab to complete IRAF task keyword names (e.g., imheader lon<tab> fills in longheader, to which you can add =yes or something similar). And when using Python syntax (see below), tab can be used to complete Python variable names, object attributes, etc.

The function

saveToFile filename

restoreFromFile filename

Differences from the CL and Unimplemented CL Features

Some IRAF CL commands have the same names as Python commands; when you use them in PyRAF, you get the Python version. The ones most likely to be encountered by users are print and del. If you want to use the IRAF print command (which should rarely be needed), use clPrint instead. If you want the IRAF delete command, just type more of the command (either dele or delete will work).

Another similar conflict is that when an IRAF task name is identical to a reserved keyword in Python (to see a list, do import keyword; print keyword.kwlist), then it is necessary to prepend a 'PY' (yes, in capital letters) to the IRAF task name. Such conflicts should be relatively rare, but note that 'lambda' and 'in' are both Python keywords.

The PyRAF help command is a little different than the IRAF version. If given a string argument, it looks up the CL help and uses it if available. For other Python argument types, help gives information on the variable. E.g., help(module) gives information on the contents of a module. There are some optional arguments that are useful in Python programs (type help(help) for more information). We plan further enhancements of the help system in the near future.

If you need to access the standard IRAF help command without the additional PyRAF features, use system.help taskname options.

Note that the IRAF help pages are taken directly from IRAF and do not reflect the special characteristics of PyRAF. For example, if you say help while, you get help on the CL while loop rather than the Python while statement. The login message on startup also comes directly from IRAF and may mention features not available (or superseded) in PyRAF. We will eventually remove some of these inconsistencies, but for the moment users will occasionally encounter such conflicts.

There are a few features of the CL environment and CL scripts that are not yet implemented:

• Packages cannot be unloaded. Currently there is no way to unload a loaded IRAF package. The bye command exists but does not do anything; the keep command also does nothing (effectively all modifications to loaded tasks and IRAF environment variables are kept). This will be added in a future version, although we are also considering alternative ways to organize the namespace of tasks and packages that may be more "Pythonic".
• No GOTO statements in CL scripts. Python does not have a goto statement, so converting CL scripts that use goto's to Python is difficult. At the moment we have made no effort to do such a conversion, so CL scripts with GOTO's raise exceptions. GOTOs may not ever get implemented.
• Background execution is not available. Background execution in CL scripts is ignored. This will get added if there is sufficient demand.
• Error tracebacks in CL scripts do not print CL line numbers. When errors occur in CL scripts, the error message and traceback refer to the line number in the Python translation of the CL code, not to the original CL code. We plan to add the CL code itself to the error messages eventually. (If you want to see the Python equivalent to a CL task, use the getCode method -- e.g. print iraf.spy.getCode() to see the code for the spy task)..

The EPAR Parameter Editor

--> epar imcalc

An EPAR window consists of a menu bar, current package and task information, action buttons, the parameter editing panel, and a status box. If there are more parameters than can fit in the displayed window, they will appear in a scrolling region.

Action Buttons

• EXECUTE - Execute the task with parameter values currently displayed in the EPAR windows. (Several windows may be open at once if the task has PSET parameters -- see below.) If parameter values were changed and not saved (via the SAVE button), these new values are automatically verified and saved before the execution of the task. The EPAR window (and any child windows) is closed, and the EPAR session ends.
• SAVE - This button saves the parameter values associated with the current EPAR window. If the window is a child, the child EPAR window closes. If the window is the parent, the window closes and the EPAR session ends. The task is not executed.
• UNLEARN - This button resets all parameters in the current EPAR window to their system default values.
• ABORT - This button exits the current EPAR session without saving any modified parameter values.
• HELP - This button displays the IRAF help information for a task.

Menu Bar

The EPAR menu bar currently consists of only the File and Options menus. All of the File menu choices map directly to the action button functionality. The Options menu allows the user to choose the way help pages are displayed; the information can be directed to the user's web browser or to a pop-up window (the default).

Parameter Editing Panel

Different means are used to set different parameter types. Numeric and string parameters use ordinary entry boxes. Parameters with an enumerated list of allowed values use choice lists. Booleans are selected using radio buttons. PSETs are represented by a button; when clicked it brings up a new EPAR window. PSET windows and the parent parameter windows can be edited concurrently (you do not have to close the child window to make further changes in the parent window).

Parameters entered using entry boxes (strings and numbers) are checked for correctness when a carriage return or tab is typed. (Tab skips forward to the next field, and Shift-tab skips back to the previous field.) The parameter values are also checked when either the SAVE or EXECUTE button is clicked. Any resulting errors are either displayed in the status area at the bottom (upon validation after return or tab) or in a pop-up window (for SAVE/EXECUTE validation).

For string, numeric, or enumerated list parameters, the user can click the right-most mouse button within the entry box or choice list to generate a pop-up menu. The menu includes options to invoke a file browser, clear the entry box, or unlearn the specific parameter value. "Clear" removes the current value in the entry, making the parameter undefined. "Unlearn" restores the initial default value for this specific parameter only. The file browser pops up an independent window that allows the user to examine the directory structure and to choose a filename for the entry. Some items on the right-click pop-up menu may be disabled depending on the parameter type (e.g., the file browser cannot be used for numeric parameters.)

Status Line

Finally, the bottom portion of the EPAR GUI is a status line that displays help information for the action buttons and error messages generated when the parameter values are checked for validity.

PyRAF Graphics and Image Display

Currently the PyRAF built-in graphics kernel is based on OpenGL and Tkinter (other alternatives will probably be added in the future). Graphics windows are created from PyRAF directly. One can run a graphics task like any other. For example, typing

--> iraf.prow('dev$pix',256)

Interactive graphics capability is also available. For example, typing iraf.implot('dev$pix') will put the user into interactive graphics mode. The usual graphics keystroke (gcur) commands recognized by the task will work (e.g., lowercase letter commands such as c) and colon commands will work as they do in IRAF. Most CL-level (capital letter) keystroke commands have not yet been implemented; the following CL level commands are currently available:

• The arrow keys move the interactive cursor one pixel. Shift combined with the arrow keys moves the cursor 5 pixels.
• C prints the current cursor position on the status line.
• I immediately interrupts the task (this is the gcur equivalent to control-C).
• R redraws the plot with annotations removed (also available through the Edit->Undo All menu item.)
• T annotates the plot at the current cursor position, using a dialog box to enter the text.
• U undoes the last "edit" to a plot (annotations or cursor markers). This can be repeated until only the original plot remains. (Also available using Edit->Undo.)
• A colon (:) prompts on the status line for the rest of the colon command. Other input from interactive graphics tasks may also be done from the status line.
• The :.markcur directive is recognized. It toggles the cursor marking mode (:.markcur+ enables it, :.markcur- disables it). Currently this directive cannot be abbreviated.

It is likely much of the additional functionality of CL level gcur commands (zooming, etc.) will also find its way into menus or other GUI elements of the PyRAF graphics window. Where possible we also will try to make this available through the keystroke commands familiar to expert IRAF users.

PyRAF attempts to manipulate the window focus and the cursor location in a sensible way. For example, if you start an interactive graphics task, the mouse position and focus are automatically transferred to the graphics window. If the task does not appear to be responding to your keyboard input check to see that the window focus is on the window expecting input.

Printing Graphics Hardcopy

It is possible to generate hard copy of the plotted display by using the File->Print menu item or, in gcur mode, the equal-sign (=) key. PyRAF will use the current value of stdplot as the device to plot to for hardcopy. Inside scripts, a hardcopy can be printed by

--> from pyraf.gki import printPlot # only need this once per session
--> printPlot()

This could be used in a Python script that generates graphics using IRAF tasks. It is also possible to do other graphics manipulations in a script, e.g., changing the display page.

Multiple Graphics Windows

It is possible to display several graphics windows simultaneously. The Window->New menu item can create windows, and the Window menu can also be used to select an existing window to be the active graphics window. Windows can be destroyed using the File->Quit Window menu item or directly using the facilities of the desktop window manager (close boxes, frame menus, etc.)

It is also possible to create new windows from inside scripts. If you type:

--> from pyraf import gwm # only need this once per session
--> gwm.window("My Special Graphic")

Other Graphics Devices

To plot to standard IRAF graphics devices such as xterm or xgterm one can

--> set stdgraph = stgkern
--> iraf.stdgraph.device = "xgterm"

One can generate plots to other devices simply by setting stdgraph to the appropriate device name (e.g., imdr or stdplot). Only special IRAF-handled devices such as xgterm and xterm need to use the "magic" value stgkern for stdgraph.

IRAF tasks such as tv.display that use the standard image display servers (ximtool, saoimage, saotng) should work fine. Interactive image display tasks such as imexamine work as well (as long as you are using the PyRAF graphics window for plotting.) Graphics output to the image display (allowing plots to overlay the image) is currently supported only through the IRAF kernel, but a PyRAF built-in kernel is under development.

Running Tasks in Python Mode

Currently there are a number of ways of running tasks and setting parameters. The system is still under development, and depending on user feedback, we may decide to eliminate some of them. Below we identify our preferred methods, which we do not intend to eliminate and which we recommend for writing scripts.

The PyRAF Interpreter Environment

It is also possible to use PyRAF from a standard Python session, which is typically started by simply typing "python" at the Unix shell prompt. In that case the simple CL syntax for calling tasks is not available and tab-completion, logging, etc., are not active. For interactive use, the conveniences that come with the PyRAF interpreter are valuable and we expect that most users will use PyRAF in this mode.

One important thing to understand is that the alternate syntax supported by the PyRAF front end interpreter is provided purely for interactive convenience. When such input is logged, it is logged in its translated, Python form. Scripts should always use the normal Python form of the syntax. The advantage of this requirement is that such scripts need no preprocessing to be executed by Python, and so they can be freely mixed with any other Python programs. In summary, if one runs PyRAF in its default mode, the short-cut syntax can be used; but when PyRAF is being used from scripts or from the standard Python interpreter, one must use standard Python syntax (not CL-like syntax) to run IRAF tasks.

Even in Python mode, task and parameter names can be abbreviated and, for the most part, the minimum matching used by IRAF still applies. As described above, when an IRAF task name is identical to a reserved keyword in Python, it is necessary to prepend a 'PY' to the IRAF task name (i.e., use iraf.PYlambda, not iraf.lambda). In Python mode, when task parameters conflict with keywords, they must be similarly modified. The statement iraf.imcalc(in="filename") will generate a syntax error and must be changed either to iraf.imcalc(PYin="filename") or to iraf.imcalc(input="filename"). This keyword/parameter conflict is handled automatically in CL emulation mode.

Example with Standard Python Syntax

This example mirrors the sequence for the example given above in the CL emulation section. In the discussion that follows we explain and illustrate some variants.

% pyraf
--> iraf.imheader("dev$pix", long=yes) 
dev$pix[512,512][short]: m51  B  600s
No bad pixels, min=-1., max=19936.
Line storage mode, physdim [512,512], length of user area 1621 s.u.
Created Mon 23:54:13 31-Mar-1997, Last modified Mon 23:54:14 31-Mar-1997
Pixel file "HDR$pix.pix" [ok]
'KPNO-IRAF'           /
'31-03-97'            /
IRAF-MAX=           1.993600E4  /  DATA MAX
IRAF-MIN=          -1.000000E0  /  DATA MIN
IRAF-BPX=                   16  /  DATA BITS/PIXEL
IRAFTYPE= 'SHORT   '            /  PIXEL TYPE
CCDPICNO=                   53  /  ORIGINAL CCD PICTURE NUMBER
.
.
.
HISTORY '24-04-87'
HISTORY 'KPNO-IRAF'           /
HISTORY '08-04-92'            /
--> iraf.imstat('dev$pix')
#               IMAGE      NPIX      MEAN    STDDEV       MIN       MAX
              dev$pix    261626     105.1     75.86       -1.     1000.
--> iraf.imcopy("dev$pix","mycopy.fits")
--> iraf.stsdas()
 
 
      +------------------------------------------------------------+
      |       Space Telescope Science Data Analysis System         |    
      |          STSDAS Version 2.1, September 29, 1999            |
      |                                                            |
      |   Space Telescope Science Institute, Baltimore, Maryland   |
      |         For help, send e-mail to help@stsci.edu            |
      +------------------------------------------------------------+
 
 
stsdas/:
 analysis/      examples        hst_calib/      sobsolete/
 contrib/       fitsio/         playpen/        toolbox/
 describe       graphics/       problems

--> iraf.fitsio()
--> iraf.catfits("mycopy.fits")
EXT#  FITSNAME      FILENAME              EXTVE DIMENS     BITPI OBJECT         
 
0     mycopy.fits                               512x512    16    m51  B  600s   
--> iraf.imhead('dev$pix',long=yes,Stdout='devpix.header')

• The task or package name must be prefixed with iraf. because they come from the iraf module. In scripts, use

  from pyraf import iraf

  to load the iraf module. Note that the first time PyRAF is imported, the normal IRAF startup process is executed (which can take a while). We are working on techniques to do fast, lightweight startups for stand-alone Python scripts that use PyRAF. At the moment the only such approach is to startup in a directory with a custom version of login.cl that defines a minimal IRAF environment.

  It is also possible to import tasks and packages directly using

  from pyraf.iraf import stsdas, imcalc

  With this approach, packages are automatically loaded if necessary and tasks can be used without the iraf. prefix. Like the IRAF CL, packages must be loaded for tasks to be accessible.

• The task must be invoked as a function in Python syntax (i.e., parentheses are needed around the argument list). Note that parentheses are required even if the task has no arguments -- e.g., use iraf.fitsio(), not just iraf.fitsio.

• String arguments such as filenames must be quoted.

Another change is that boolean keywords cannot be set using appended + or - symbols. Instead, it is necessary to use the more verbose keyword=value form (e.g., long=yes in the example above). We have defined Python variables yes and no for convenience, but you can also simply say long=1 to set the (abbreviated) longheader keyword to true.

Emulating pipes in Python mode is also relatively simple. If a parameter Stdout=1 is passed to a task, the task output is captured and returned as a list of Python strings, with one string for each line of output. This list can then be processed using Python's sophisticated string utilities, or it can be passed to another task using the Stdin parameter:

--> s = iraf.imhead("dev$pix", long=yes, Stdout=1) 
--> print s[0]
dev$pix[512,512][short]: m51  B  600s
--> iraf.head(nl=3, Stdin=s)
dev$pix[512,512][short]: m51  B  600s
No bad pixels, min=-1., max=19936.
Line storage mode, physdim [512,512], length of user area 1621 s.u.

Stdin and Stdout can also be set to a filename or a Python filehandle object to redirect output to or from a file. Stderr is also available for redirection. Note the capital 'S' in these names -- it is used to eliminate possible conflicts with task parameter names, but in the future we may decide to simplify these names by eliminating the uppercase 'S'.

Setting IRAF Task or Package Parameters

--> iraf.imcopy("dev$pix","mycopy.fits")

--> iraf.imcopy(input="dev$pix",output="mycopy.fits")

But parameters can also be set by setting task attributes. For example:

--> iraf.imcopy.input = "dev$pix"
--> iraf.imcopy.output = "mycopy.fits"
--> iraf.imcopy()  # run the task with the new values

--> iraf.imcopy.inp = "dev$pix"
--> iraf.imcopy.o = "mycopy.fits"

--> iraf.imstat.binwidth = "33.0"
--> iraf.imstat.binwidth = "33"
--> iraf.imstat.bin      = 33.0
--> iraf.imstat.bin      = 33

--> iraf.imstat.bin = "cow"
Traceback (innermost last):
  File "", line 1, in ?
ValueError: Illegal float value 'cow' for parameter binwidth

One can list the parameters for a task using one of the following commands:

--> iraf.imcopy.lpar()
--> iraf.lpar(iraf.imcopy)      # Note there are no quotation marks
--> iraf.lpar('imcopy')

The first approach is preferred since it best shows the structure. For those who have encountered object-oriented programming, iraf.imcopy is an 'IRAF task object' that has a method named lpar that lists task parameters. On the other hand, iraf.lpar is a function (in the iraf module) that takes either an IRAF task object or a string name of a task as a parameter. It finds the task object and invokes the .lpar() method.

One can start the EPAR utility for the task using a parallel set of commands:

--> iraf.imcopy.epar()
--> iraf.epar(iraf.imcopy)
--> iraf.epar('imcopy')

Tasks appear as attributes of packages, with nested packages also found. For example, if you load the stsdas package and the restore subpackage, then the mem task can be accessed through several different means: iraf.mem, iraf.stsdas.mem, or iraf.restore.mem will all work. Ordinarily the simple iraf.mem is used, but if tasks with the same name appear in different packages, it may be necessary to add a package name to ensure the proper version of the task is found.

Other Ways of Running IRAF Tasks

One way of reducing the typing burden (interactively or in scripts, though perhaps it isn't such a good idea for scripts) is to define an alias for the iraf module after it is loaded. One can simply type:

--> i = iraf
--> i.imcopy('dev$pix','mycopy.fits')
--> i.imstat('mycopy.fits')

--> i = 1
--> i.imcopy('dev$pix','mycopy.fits')

  AttributeError: 'int' object has no attribute 'imcopy'

There are a few command-line options available for PyRAF:

-v Set verbosity (repeated 'v' increases the level; mainly useful
   for system debugging)
-h List the options available
-i Do not start the special PyRAF interpreter, just run a standard
   Python interactive session
-m Start the PyRAF interpreter (default)

	PyRAF interpreter	Python default interpreter
prompt	-->	>>>
print interpreter help	.help	n/a
exit interpreter	.exit	EOF (control-D) or sys.exit()
start logging input	.logfile filename	n/a
append to log file	.logfile filename append	n/a
stop logging input	.logfile	n/a
run system command	! command	os.system('command')
start a subshell	!!	os.system('/bin/sh')

Perry Greenfield & Rick White Questions? Contact help@stsci.edu 2001 December 6