# Tutorial

This page was created from the LaTeX source in book/tutorial/ using latex2wiki

# Introduction

Welcome to a brief introduction to Madagascar. The purpose of this document is to teach new users how to use the powerful tools in Madagascar to: process data, produce documents and build your own Madagascar programs. To gradually introduce you to Madagascar, we have created a series of tutorials that are targeted to distinct audiences and designed to make you an experienced Madagascar user in a short-time period. The tutorials are divided by interest into three main categories:

• Users learn about Madagascar, how to use the processing programs, and build scripts.
• Authors learn how to build reproducible documents using Madagascar.

Each tutorial is designed to be completed in a short period of time. Additionally, each tutorial has hands-on examples that you should be able to reproduce on your computer as you go along with the tutorials. Most tutorials will use scripts that you can edit, modify, or play with to further gain experience and understanding. By the end of the tutorial series, you should be able to use all of the tools inside of Madagascar. Please note that this tutorial series does not explicitly show you how to process certain types of data, or how to perform common data processing operations (e.g. CMP semblance picking, time migration,etc.). Additional tutorials on those specific subjects will be added over time. The purpose of this document is simply to familiarize you with the Madagascar framework in a general sense. Before you go on, here are some notes on notation:

• important names, or program names are usually bold in the text. For example: sfwindow
• code snippets are always in the following formatting:
 sfwindow

# Users

The Users tutorials demonstrate how to use the Madagascar framework to create, process and visualize data, and to create reproducible scripts for processing data. The main goals of the Users tutorials are to learn about:

• the RSF file format,
• the command line interface,
• how to interact with files on the command line,
• commonly used programs,
• how to make plots in Madagascar,
• how to make reproducible scripts,
• how to use SCons and Python,
• how to visualize your data.

By the end of this tutorial group you should be able to fully use all of Madagascar's built-in tools for data processing and scripting. By using these tools, you'll be able to process data ranging from tens of megabytes to tens of terabytes in a reproducible fashion.

To begin, let's talk about the core principles of Madagascar, and the RSF file format.

The hierarchy of Madagascar. Fundamentally, everything builds off of the RSF file format. As you go up the chain the complexity level may increase, but the capabilities of the processing package increase as well.

### RSF file format

As previously mentioned, the lowest level of Madagascar is the RSF file format, which is the format used to exchange information between Madagascar programs. Conceptually, the RSF file format is one of the easiest to understand, as RSF files are simply regularly sampled hypercubes of information. For reference, a hypercube is a hyper dimensional cube (or array) that can best be visualized as an ${\displaystyle N}$-dimensional array, where ${\displaystyle N}$ is between 1 and 9 in Madagascar.

RSF hypercubes are defined by two files, the header file and the binary file. The header file contains information about the dimensionality of the hypercube as well as the data contained within the hypercube. Information contained in the header file may include the following:

• number of elements on all axes,
• the origin of the axes,
• the sampling interval of elements on the axes,
• the type of elements in the axes (i.e. float, integer),
• the size of the elements (e.g. single or double precision),
• and the location of the actual binary file.

Since we often want to view this information about files without deciphering it, we store the header file as an ASCII text file in the local directory, usually with the suffix .rsf . At any time, you can view or edit the contents of the header files using a text editor such as gedit, VIM, or Emacs.

The binary file is a file stored remotely (i.e. in a separate directory) that contains the actual hypercube data. Because the hypercube data can be very large (${\displaystyle 10s}$ of GB or TB) we usually store the binary files in a remote directory with the suffix .rsf@ . The remote directory is specified by the user using the DATAPATH environmental variable. The advantage to doing this, is that we can store the large binary data file on a fast remote filesystem if we want, and we can avoid working in local directories.

Cartoon of the RSF file format. The header file points to the binary file, which can be separate from one another. The header file, which is text, is very small compared to the binary file.

For more complete documentation on the RSF file format see the following links:

## Command line interface

Madagascar was designed initially to be used from the command line. Programmers create Madagascar programs (prefixed with the sf designation) that read and write Madagascar files. These programs are designed to be as general as possible, so that each one could operate on any dataset in RSF format, provided you also supply the correct parameters and have the right type of data for the program.

### Using programs

Madagascar programs follow the standard UNIX conventions for reading and writing RSF files to and from stdin and stdout . This is also the convention that Seismic Unix uses, so it should be familiar to SU users. For example: the program sfattr allows us to get attributes about an RSF file (mean, standard deviation, min, max, etc.) To get the attributes for a pre-computed file, we might use

sfattr < file.rsf
rms =       1.41316
mean =      0.999667
2-norm =       357.503
variance =      0.997693
std dev =      0.998846
max =       5.05567 at 36 8 27
min =      -3.59936 at 18 9 6
nonzero samples = 64000
total samples = 64000


We demonstrate how to read and write RSF files using sfwindow which is a program that allows us to select portions of an RSF file. When sfwindow is used without any additional parameters, we are able to make a copy of a file with a different filename. For example:

 sfwindow < file.rsf > file2.rsf


gives us two files, file.rsf and file2.rsf which are identical but not the same file. If your intention is simply to copy a file, you can also use sfcp . In addition to specifying files to read in and out on the command line we can specify the parameters for each program that are necessary for it to run, or to produce the desired result. The general format for specifying parameters on the command line is key=val , where key is the name of the parameter that you want to set, and val is the value of the parameter. There are four (4) types of values that are acceptable: integers, floating point numbers, booleans, or strings. Going back to the window program, we can specify the number of traces or pieces of the file that we want to keep like:

sfwindow < file.rsf n1=10 > file-win.rsf


### Self-documentation

Of course, we can specify as many parameters as we'd like on the command line. To figure out which parameters are needed for a specific program, just type the name of the program with no input files our output files on the command line to bring up a program's self-documentation. For example, sfwindow 's self documentation is:

NAME
sfwindow
DESCRIPTION
Window a portion of a dataset.
SYNOPSIS
sfwindow < in.rsf > out.rsf verb=n squeeze=y
j#=(1,...) d#=(d1,d2,...) f#=(0,...) min#=(o1,o2,,...)
n#=(0,...) max#=(o1+(n1-1)*d1,o2+(n1-1)*d2,,...)
PARAMETERS
float   d#=(d1,d2,...)  sampling in #-th dimension
largeint f#=(0,...)     window start in #-th dimension
int     j#=(1,...)      jump in #-th dimension
float   max#=(o1+(n1-1)*d1,o2+(n1-1)*d2,,...)
maximum in #-th dimension
float   min#=(o1,o2,,...)       minimum in #-th dimension
largeint n#=(0,...)     window size in #-th dimension
bool    squeeze=y [y/n]
if y, squeeze dimensions equal to 1 to the end
bool    verb=n [y/n]    Verbosity flag
USED IN
bei/dpmv/krchdmo
bei/dpmv/matt
bei/dwnc/sigmoid
bei/fdm/kjartjac
bei/fld/cube
bei/fld/shotmovie
bei/fld/synmarine
bei/fld/yc
bei/ft1/autocor
bei/ft1/ft2d
bei/krch/sep73
bei/sg/denmark
bei/sg/toldi
bei/trimo/mod
bei/trimo/subsamp
bei/vela/stretch
bei/vela/vscan
bei/wvs/vscan
cwp/geo2006TimeShiftImagingCondition/flat
cwp/geo2006TimeShiftImagingCondition/icomp
cwp/geo2006TimeShiftImagingCondition/zicig
cwp/geo2007StereographicImagingCondition/flat4
cwp/geo2007StereographicImagingCondition/gaus1
167 more examples listed in:
SOURCE
system/main/window.c
DOCUMENTATION
VERSION
1.3-svn


The self-documentation tells us the function of the program, as well as the parameters that are available to be specified. The parameter format is type - name=default value [options] and then a short description of the parameter. File parameters request a name of a file. For example:


file=file.rsf


Note: strings with spaces must be enclosed in quotation marks (e.g. 'my value') so that the Unix shell could interpret them correctly.

### Piping

Sometimes we want to chain multiple commands together without writing intermediate Madagascar RSF files in the process. We can avoid making intermediate files by using another standard UNIX construct, pipes. Pipes allow us to connect the standard output from one Madagascar program to the standard input to another program without first writing to a file. For example we could do the following without pipes:

sfwindow < file.rsf > file-win.rsf
sftransp < file-win.rsf > file2.rsf


Or we could do the equivalent using pipes on one line:

sfwindow < file.rsf | sftransp > file2.rsf


Pipes simply make these statements more compact, and allow us to reduce the number of files that we need to save to disk. Piping is very powerful, because there is no limit to the number of statements that you can pipe together. For example:

sfwindow < file.rsf | sftransp | sfnoise var=1 > file2.rsf


If you're using multiple programs, and do not want to save the intermediate files, then pipes will greatly reduce the number of files that you have to keep track of.

### Interacting with files from the command line

Ultimately though, 95% of your time using Madagascar on the command line will be to inspect and view files that are output by your scripts. Some of the key commands that are used to interact with files on the command line are:

• sfin , used to get header information,
• sfattr , used to get file attributes,
• sfwindow , used to select portions of RSF files,
• and sftransp , used to reorder files.

Here are detailed usage examples and explanations of what the above programs do: sfin is one of the most used program on the command line, because most often we simply need to check the dimensionality of our files to make sure that we have them in the correct order.

sfin file.rsf

file.rsf:
in="/var/tmp/file.rsf@"
esize=4 type=float form=native
n1=100         d1=0.004       o1=0          label1="Time" unit1="s"
n2=34          d2=0.1         o2=0          label2="Distance" unit2="km"
3400 elements 13600 bytes


sfattr is also commonly used from the command line to check files for correct values. Most often, we use sfattr to ensure that files are not filled with zeros, or with NaN's after a long computation, or to make sure that the values are reasonable. sfattr can be used to obtain basic statistics about the files as well.

sfattr < file.rsf

*******************************************
rms =             1
mean =             1
2-norm =       58.3095
variance =             0
std dev =             0
max =             1 at 1 1
min =             1 at 1 1
nonzero samples = 3400
total samples = 3400
*******************************************


sfwindow is used to select subsets of the data contained in an RSF file for computation elsewhere. Typically, you specify the data subset you want to keep using, the n, j, and f parameters which specify the number of indices in the arrays to keep, the jump in indices, and the first index to keep from the file in the respective dimension. For example if we want to keep the 15th-30th time samples from the first axis in file.rsf, we might use the following command:

sfwindow < file.rsf f1=15 n1=15 j1=1 > file-win.rsf


sftransp is used to reorder RSF files for other programs to be used. For example:

sftransp < file.rsf plane=12 > file-transposed.rsf


swaps the first and second axes, so that now the first axis is distance and the second axis is time. For more information about commonly used Madagascar programs please see the Guide to Madagascar programs.

## Plotting

VPLOT provides a method for making plots that are small in size, aesthetically pleasing, and easily compatible with Latex for rapid creation of production quality images in Madagascar.

### VPLOT

The VPLOT file format (.vpl suffix) is a self-contained binary data format that describes in vector format how to draw a plot on the screen using an interpreter. Since VPLOT is not a standard imaging format, VPLOT files must be viewed with third-party interpreters which we call pens. Each pen interfaces VPLOT with a third-party graphing library such as X11, plplot, opengl, and others. This flexibility makes VPLOT files almost as portable as standard image formats such as: jpeg, png, and gif. Unlike rasterized formats, VPLOT files can be scaled to any size without losing image quality.

### Creating plots

To generate VPLOT files, we need to pass our computed RSF files through VPLOT filters, that convert RSF data files to VPLOT files. The VPLOT filters are named by the type of plot that they produce. Table~(table:plotting) lists all of the available VPLOT filters. �egin{table} �egin{tabularx}{ extwidth}{|l|X| } \hline sfgraph & create line plots, or scatter plots

sfgrey & create raster plots or 2D image plots

sfgrey3 & create 3D image plots of panels (or slices) of a 3D cube

sfbox & make box-line plots

sfcontour & make contour plots

sfcontour3 & make contour plots of 3D surfaces

sfplotrays & make plots of rays

\hline \end{tabularx} \caption{List of available plotting programs in Madagascar}

(table:plotting)


\end{table} To actually create a plot, we can use the plotting programs on the command line in the same fashion that we would use a Madagascar program:

sfspike n1=100 | sfnoise > junk.rsf
sfgraph < junk.rsf title="noise" > junk.vpl


### Visualizing plots

In this example, we create a single trace full of noise and then send it to sfgraph to produce a single VPLOT file, junk.vpl. As you may have noticed, this only creates the file which is useful for saving the plot,does not allow us to visualize the data. To visualize the data we need to use a pen , which tells your machine how to actually draw the plot. A typical Madagascar installation will have multiple pens available for you to use. By default, you should use sfpen which will pick the best pen available for you. You can use sfpen to visualize your plots in the following manner:

sfpen < junk.vpl


This will pop up a screen on your window with the plot shown. Depending on which pen you are using you may be able to interact with the pen interface to control various parameters of the plot as shown by the buttons at the top of the screen. Depending on the pen that you are using, there may be keyboard shortcuts to many of the buttons. NOTE: oglpen uses a mouse interface that can be accessed by right-clicking on the plot.

### Converting VPLOT to other formats

If you want to build reports or documents using other programs, or want to send your images to someone who does not have Madagascar you will need to convert your VPLOT files to other image formats for transfer. To convert a VPLOT plot to another format use the tool vpconvert . vpconvert allows you to convert VPLOT files to any of the following formats, provided that you have the appropriate third-party libraries installed:

• avi,
• eps,
• gif,
• jpeg/jpg,
• mpeg/mpg (movie format),
• pdf,
• png,
• ppm,
• ps,
• svg,
• and tif.

Here's an example of how to use vpconvert :

vpconvert junk.vpl format=jpeg


NOTE: details on how to install these third-party libraries are not included with the Madagascar library, and we provide no support on installing them. Most users will be able to install them using either package management software (on Linux and Mac) or pre-compiled binaries.

\documentclass[12pt]{geophysics} \usepackage{hyperref} \usepackage{comment} \usepackage{verbatim} \usepackage{moreverb} \usepackage{tabularx} \setfigdir{Fig}

Madagascar's command line interface allows us to create, manipulate and plot files. Often though, we want to create scripts that will perform these operations automatically for us. Scripting is especially important when processing large amounts of data, or performing a complex chain of file manipulations in order to ensure that operations are completed in the right order or with proper parameters. In Madagascar, there are two main ways of creating scripts: shell scripts, and Python scripts using SCons.

### Shell scripting

Shell scripting is the first option for creating scripts. In shell scripting we simply copy the command lines that we would use and paste them into a file that is recognized by either BASH, C-shell or another shell of your choosing. Shell scripting may be familiar to users of other packages as SU, because it is the primary method of scripting in other packages. An example of a Madagascar shell script is shown below:

#!/bin/bash
sfspike n1=100 > junk.rsf
< junk.rsf sfgraph > junk.vpl


For simple processing flows, shell scripting is a quick and easy approach to saving your processing flow. However, shell scripts quickly become unmanageable when more complicated processing flows are used. Additionally, shell scripts have no way to avoid repeating commands that have already been successfully run, which causes shell scripts to spend a significant amount of time duplicating already completed work. Because of these issues, Madagascar's main scripting option is to use a build manager called SCons for scripting instead of shell scripts.

### SCons

SCons is a build manager written in 100\

### SConstructs and commands

SCons scripts are referred to as SConstructs. In order to use SCons, you must create an SConstruct in the local directory where you want to work. Since SCons is written in Python, an SConstruct is simply a text file written using Python syntax. If you don't know Python, you can still use SCons, because the syntax is simple. First, a primer on Python syntax. In SConstructs we are going to deal with Python functions and strings. Python functions are simple, and should be familiar to anyone who has used a programming language. For example, calling a Python function, foo, looks like:

One argument - foo(1).
Many arguments - foo(1,2,3,4,5,123)


Python functions can take many arguments, and arguments of different types, but the syntax is the same. Python strings are also very similar to other programming languages. In Python anything inside double quotes is automatically considered to be a string, and not a variable declaration. However, Python also supports a few notations to indicate strings (or long strings) as shown below:

"this is a string"
'this is a string'
"""this is a string"""
"'this is a string"'


Somtimes in Python you will need to nest a string within a string. To do use one of the string representations for the outer string, and use a different one for the inner string. For example:

"""sfgraph title="junk plot" """  OR
"' sfgraph title="junk plot" "'   OR
'   sfgraph title="junk plot" '


Fundamentally, SConstructs are composed of only three commands: Flow , Plot and Result . The first command is Flow . A Flow creates a relationship between the input file, the output file, and the processing command used to create the output file from the input file. The syntax for a Flow is:

Flow(output file,input file,command)


where, target and source are file names (strings), and command is a string that contains the Madagascar program to be used, along with the command line parameters needed for that program to run. For example:

Flow("spike1","spike","sfadd scale=4.0") ,


creates a dependency relationship between the output file 'spike1' and the input file 'spike'. The dependency indicates that 'spike1' cannot be created without 'spike' and that if 'spike' changes then 'spike1' also changes. The relationship in this case is that 'spike1' should be 'spike' scaled by four times. The equivalent command on the command line would be:

< spike.rsf sfadd scale=4.0 > spike1.rsf


Note: the file names of the input and output files do not need to include '.rsf' on the end of the files as SCons automatically adds the suffix to all of our file names. Now that we can create relationships between files using Flow s, we can create an SConstruct to do all of our data processing using SCons. However, we often want to visualize the results of our processing in order to quality control the results. In order to create Plot s (or other visualizations) in Madagascar we have two additional SCons commands: Plot and Result . Plot and Result tell SCons how to use Madagascar's plotting programs to create visualizations of files on the fly after they have been computed. The syntax for both Plot and Result is as follows:

Plot(input file, command)
OR
Result(input file, command)


In both cases, the Plot or Result command tells SCons to build a VPLOT file with same file name as the input file (with the .vpl suffix instead) from the input file using the command provided. For example, if we want to make a graph of a file we could use:

Plot("spike1","sfgraph pclip=100")


Behind the scenes, SCons establishes that we want to use "spike1.rsf" to create a Plot called "spike1.vpl" using sfgraph. The equivalent command line operation is:

< spike1.rsf sfgraph pclip=100 > spike1.vpl


Result can be used in the same way as Plot , illustrated above. At this point, you might be asking yourself, what's the difference between Plot and Result ? The answer is that Plot creates all VPLOT files in the local directory, whereas Result creates its VPLOT files in a subdirectory called Fig. Fig is a location used to store Plot s that we want to later use when creating papers using Latex. By default, you should use Plot when creating visualizations. Only use Result when you want to save something to be used in a paper. Note: since the VPLOTs from Plot and Result are placed in different locations you can use both Plot and Result for a single RSF file, but create two different plots for the same file.

### Creating an SConstruct

Now that we have the three SConstruct commands, we can write our first SConstruct. To do so, open the SConstruct file in your favorite text editor. Before we can create any Flow , Plot or Result statements we have to add a first statement to the SConstruct. Enter the following statement (verbatim) into your new SConstruct:

from rsf.proj import *


This statement tells SCons where to find the Flow , Plot and Result commands, and must be included in every SConstruct. After that statement has been entered, you can enter as many Flow , Plot and Result commands as you wish making sure to use proper syntax. It's helpful to use a text editor that has Python syntax highlighting, as that will help you find and remedy strings that are not closed. You can also create Python comments using the ${\displaystyle \#}$ mark to indicate the beginning of a comment to help document your SConstructs.

Lastly, you must include the following statement at the end of your SConstruct:

End()


This statement tells SCons that the script is done and that it should not look for anything else in the script. Make sure to include this statement as the very last item in every SConstruct, otherwise you will get interesting error statements during execution. Here's a sample SConstruct:

from rsf.proj import *  # Remember, this statement comes first... ALWAYS

'''Flow''' ("spike",None,"sfspike n1=100 k1=50")
'''Flow''' ("noise","spike1","sfnoise")

'''Plot''' ("spike",'sfgraph title="spike" ' ) # Note string nesting
'''Plot''' ("spike1",'sfgraph title="spike1" ')
'''Plot''' ("noise",'sfgraph title="noisy" ')

'''Result''' ("noise",'sfgraph title="noisy" pclip=75 ')

End() # Remember, this always ends the script.


Note: you do not have to order your Flow , Plot and Result commands as shown above. You can mix Flow , Plot and Result in any order. SCons automatically establishes the relationships between related files and commands.

### Executing SCons

Now that you have an SConstruct, you can start processing data the Madagascar way. To do so open a terminal and navigate to the local directory where your SConstruct is located. To execute your SConstruct simply run: scons . When you run SCons, it will check to make sure that all necessary dependencies are found and that all of your commands inside the SConstruct are valid. If not, SCons will return an error message showing you where you made a mistake. If everything is OK, then SCons will begin creating your files in the local directory and will output its progress as it executes the Madagascar programs on the command line. For example, running the sample SConstruct from above should show something similar to the following output:

scons: Reading SConscript files ...
scons: Building targets ...
/opt/rsf/bin/sfspike n1=100 k1=50 > spike.rsf
< spike.rsf /opt/rsf/bin/sfadd scale=4.0 > spike1.rsf
< spike1.rsf /opt/rsf/bin/sfnoise > noise.rsf
< spike.rsf /opt/rsf/bin/sfgraph title="spike" > spike.vpl
< spike1.rsf /opt/rsf/bin/sfgraph title="spike1" > spike1.vpl
< noise.rsf /opt/rsf/bin/sfgraph title="noisy" > noise.vpl
< noise.rsf /opt/rsf/bin/sfgraph title="noisy" pclip=75 > Fig/noise.vpl
scons: done building targets.


If the execution of scons ends with, "scons: done building targets" then the script completed successfully. Check your local directory for your output files.

### Common errors in SConstructs

There are two common errors that most users will experience when executing SConstructs: missing dependency errors, and misconfiguration errors. We'll demonstrate both of these errors to help new users troubleshoot them below. The first error is caused by missing a dependency in the SConstruct. To introduce this error into your SConstruct modify the sample SConstruct from above to the following:

from rsf.proj import *  # Remember, this statement comes first... ALWAYS

'''Flow''' ("noise","spike1","sfnoise")

'''Plot''' ("spike",'sfgraph title="spike" ' ) # Note string nesting
'''Plot''' ("spike1",'sfgraph title="spike1" ')
'''Plot''' ("noise",'sfgraph title="noisy" ')

'''Result''' ("noise",'sfgraph title="noisy" pclip=75 ')

End() # Remember, this always ends the script.


Now when you run scons , you should get an error message:

scons: Reading SConscript files ...
scons: Building targets ...
scons: *** [spike1.rsf] Source spike.rsf' not found, needed by target spike1.rsf'.
scons: building terminated because of errors.


In this case, SCons tells you exactly which file is missing and which output file is missing one of its dependencies. To solve this problem, add the Flow to create 'spike' to the SConstruct. If your SConstruct uses a file that is not created inside the SConstruct, and is complaining about a missing dependency, then make sure the file you are looking for is in a location that the SConstruct can access. The second error, is caused by having a misconfigured command. To demonstrate this type of error change your SConstruct to:

from rsf.proj import *  # Remember, this statement comes first... ALWAYS

'''Flow''' ("spike",None,"sfspike n1=100 k1=50")
'''Flow''' ("spike1","spike","sfasd scale=4.0")
'''Flow''' ("noise","spike1","sfnoise")

'''Plot''' ("spike",'sfgraph title="spike" ' ) # Note string nesting
'''Plot''' ("spike1",'sfgraph title="spike1" ')
'''Plot''' ("noise",'sfgraph title="noisy" ')

'''Result''' ("noise",'sfgraph title="noisy" pclip=75 ')

End() # Remember, this always ends the script.


In this case, we've introduced a typo into one of our commands. When running scons, the result is:

scons: Reading SConscript files ...
scons: Building targets ...
/opt/rsf/bin/sfspike n1=100 k1=50 > spike.rsf
< spike.rsf sfasd scale=4.0 > spike1.rsf
scons: *** [spike1.rsf] Error 127
scons: building terminated because of errors.


Again, the error message is pretty descriptive and could help you track down the error relatively quickly. It's important to note that in this case, the error was caught after SCons tried to execute the command and failed to do so, whereas the dependency error was caught before any commands were executed. This means that this typo error would kill a script that's been running a long time without any issues up till that point. Fortunately, SCons would restart the script at the point of failure thereby saving you all the additional time to recompute everything before this point in the script. These are only two of the most common errors that new users will see. For additional information about debugging SConstructs, or for exceptionally strange errors please consult the RSF users mailing list.

Here are some additional SCons commands that are useful to know.

#### Viewing Plots

If you're running an SConstruct and want to view the plots that it creates as it creates them, then you can use: scons view to force SCons to show the plots interactively. Doing so allows you to view your output at runtime and stop the SConstruct as it's running if you don't like what you see.

#### Viewing only one plot

If you want to view the plot associated with only a single file you can tell SCons to view that plot by using the command: scons file.view where file is the filename of the item that you want to view. For example: scons spike.view would show the plot of the spike.rsf file. The advantage to using this command is that SCons will only build the dependencies needed to view that specific plot, which can save you a lot of time in longer processing scripts.

#### Building specific targets

If you want to build a specific RSF file, or a few specific RSF files you can simply specify them on the command line when you execute SCons as follows: scons file.rsf file2.rsf file3.rsf ... . The advantage to doing this, is that SCons will only build those files and their dependencies instead of running through the entire script.

#### Cleaning up

After we've executed our SConstruct and viewed the results of the processings flows, we might decide that we want to clean up our project and go work on something else. Normally, one would have to delete all of the files and then move on, but SCons automates this process using the command: scons -c . Simply execute it in the local directory and SCons will remove all of the built RSF files, and VPLOT files. SCons will not remove files that it does not have rules for.

#### Dry-runs

Sometimes we want to test an SConstruct through the end of the script without waiting for it to run completely. To do so, we can use scons -n which tells SCons to simulate an actual execution. During a dry-run SCons will check all of the commands, and dependencies to ensure that they exist and are properly configured. If not, then SCons will tell you which commands are misconfigured. This command can save you hours of debugging! Use it liberally.

#### Parallel executions

For long scripts we can use parallel execution to run multiple processing commands at the same time. Parallel execution requires that the commands can function independently of one another (i.e. there are commands that do not depend on each other's output). To use parallel execution use: pscons which launches the maximum number of parallel jobs that your computer can handle at once. Parallel execution can greatly speed up your processing flows if SCons can parallelize it. In the worst case, SCons won't be able to parallelize your script and it will run at the same speed as if you just used scons .

### Final thoughts

At this point you should have a solid understanding of how to use SCons and create SConstructs to script your Madagascar processing flows. Admittedly, SCons is somewhat complicated and difficult to understand at first, but don't give up. By using SCons, you are able to create powerful Madagascar scripts that are completely reproducible and enjoy the benefits of using a powerful build management system. However, we have not shown you the most useful aspects of SCons, which are demonstrated in the next tutorial, where we show how to integrate Python with SCons, thereby making processing flows that are very powerful.

\documentclass[12pt]{geophysics} \usepackage{hyperref} \usepackage{comment} \usepackage{verbatim} \usepackage{moreverb} \usepackage{tabularx} \setfigdir{Fig}

Now that you have a grasp of how to use SCons to put together simple processing flows, we're going to show you how to abuse SCons to make more advanced processing flows that can handle multiple input and output files properly. Additionally, we're going to demonstrate some SCons tricks that make your life easier, and allow you to work faster, and smarter.

### Multiple input files

Many Madagascar programs require multiple input files and/or output multiple files. In order for SCons to properly recognize that these additional files are dependencies for a specific output file we have to change the syntax that we use for Flow , Plot and Result statements. To do so, we'll need to use Python lists to help us keep everything together when using our SCons commands. We first discuss the case where we need multiple input files. An example of a Madagascar program that requires multiple input files is sfcat . For reference, sfcat is used to concatenate multiple files together, essentially a file manipulation program. For example, we might use sfcat on the command line in the following fashion: �egin{verbatimtab}[4] < file.rsf sfcat axis=2 file1.rsf file2.rsf file3.rsf file4.rsf > catted.rsf \end{verbatimtab} To replicate this behavior using SCons we need to tell our Flow statements about the presence of multiple input files. Important: if we do not indicate to SCons that we have multiple input files then the dependency chain will not be correct and we cannot guarantee our results are correct . We can easily tell SCons of the presence of multiple input files by using a Python list as our input file, instead of a string: �egin{verbatimtab}[4] Flow('catted',['file','file1','file2','file3','file4'], 'sfcat axis=2 $\displaystyle {SOURCES[1:-1]}') \end{verbatimtab} As you may have noticed, there are two new items in this '''Flow''' statement, but let's start by discussing only the list of file names: '''['file','file1','file2','file3','file4']''' . The list of file names is simply a Python list of strings that contains each of the names of the files that we want to use in this Flow command. As usual, we don't have to append the '.rsf' suffix to the end of these names because SCons adds it for us. The second new part to the '''Flow''' command is: '''${SOURCES[1:-1]''' }, referred to as the SCons source list, which tells SCons about the presence of additional input files in the command, and to substitute the names into the command automatically. Without this command, SCons would not include the files in the final command. As an example of what the SCons source list does, compare the two SConstructs below against one another. The top is correct, the bottom is incorrectly configured: �egin{verbatimtab}[4] # Correct from rsf.proj import * Flow('file',None,'spike n1=100') Flow('file1',None,'spike n1=100') Flow('file2',None,'spike n1=100') Flow('file3',None,'spike n1=100') Flow('file4',None,'spike n1=100') Flow('catted',['file','file1','file2','file3','file4'], 'sfcat axis=2$ {SOURCES[1:-1]}') End() \end{verbatimtab} # Wrong from rsf.proj import * Flow('file',None,'spike n1=100') Flow('file1',None,'spike n1=100') Flow('file2',None,'spike n1=100') Flow('file3',None,'spike n1=100') Flow('file4',None,'spike n1=100') Flow('catted',['file','file1','file2','file3','file4'],'sfcat axis=2') End()  If you noticed the command line output from SCons, you would find that for the incorrect SConstruct, SCons ran the following command: < file.rsf sfcat axis=2 > catted.rsf  which is not correct. This is because SCons was not informed that the additional sources actually are used inside the command and did not substitute them in. The SCons source list contains a reference to all of the file names that we passed in our Python list earlier. In order to access those names we have to use a specific notation, but it is essentially a Python list enclosed in curly brackets that begins with a$. Since the source list is a Python list, we can get the file names in a few ways if we follow standard Python list conventions. Standard Python list conventions are:

• List indexing starts with index 0,
• Lists may be negatively indexed, which returns the items from the end (e.g. LIST[-1]),
• Lists may be sliced using the LIST[start:end] notation, where start and end are indices,
• List slicing indices are inclusive for the starting index, and exclusive for the ending index (e.g. LIST[0:4] returns LIST[0],LIST[1],LIST[2],LIST[3] but NOT LIST[4],
• Open slicing indices may be used (e.g. LIST[2:] gets everything from index 2 to the end, and LIST[:4] returns everything from 0 to but not including 4).
• Negative and positive indices may be used together (e.g. LIST[1:-1] returns all elements but the first and last).

These are the most useful conventions to remember, and the ones you will most frequently see. Please see the Python documentation (freely available online) for more information about dealing with Lists. Using the above conventions the following Flow statements are all equivalent for lettings SCons know about the presence of multiple input files: �egin{verbatimtab}[4] Flow('catted',['file','file1','file2','file3','file4'], "' sfcat axis=2 ${\displaystyle {SOURCES[1]}}${SOURCES[2]}

   ${\displaystyle {SOURCES[3]}}${SOURCES[4]}


"') Flow('catted',['file','file1','file2','file3','file4'], "' sfcat axis=2 $\displaystyle {SOURCES[1:5]} "') Flow('catted',['file','file1','file2','file3','file4'], "' sfcat axis=2$ {SOURCES[1:-1]} "') \end{verbatimtab} Note: never use SOURCES[0] because SOURCES[0] corresponds to 'file' which is already used by SCons for standard input. Also, never use open slicing on the SOURCES list, because at the end of the SOURCES list are extra items added by SCons for safe keeping that will break the command if accidentally used.

### Multiple outputs

For multiple outputs, we can use the same conventions as before, except we specify a list of output files instead of input files, and we use the TARGETS SCons list, instead of SOURCES. For example:

### Classes

The least used, but most powerful part of Python that you can bring into your SConstructs are Python classes. For example, if you are writing a script to process multiple models in the exact same way, but that have different parameters you would have to write separate Flow statements to process each of them, OR you could write a Python class that takes the model parameters and uses those parameters to generate Flow statements automatically, similar to functions. However, a class can allow you to group functions together into a single coherent body and allow you to drastically reduce the amount of code that must be reused. We refer the reader to the Python documentation for more information on creating and using classes.

### Final thoughts

Congratulations on completing the Madagascar User tutorial series. Now, you should have all the tools to: use Madagascar programs, write SConstructs to script your processing flows, and combine Python with SCons to make powerful scripts that can process data in ways not previously possible. From here, you can continue learning about how to write your own Madagascar programs, or learn about how to make reproducible documents using the Madagascar framework.