Madagascar - User contributions [en]

Download

2009-07-20T19:52:08Z

Mt: Updated number of figures

[[Image:Fotolia_2043821_XS.jpg|right|]]
You can choose to download either the latest stable release, or the current development version (frequently updated).

==Stable release==

===Precompiled binary packages===

A Mac OS X precompiled binary package of the latest Madagascar stable release is available for download from [https://sourceforge.net/project/showfiles.php?group_id=162909 Sourceforge]. Download a disk image file appropriate for your hardware: madagascar-*.macosx.ppc.dmg for a PowerPC Mac or madagascar-*.macosx.i386.dmg for an Intel Mac. Mount the disk image if it does not mount automatically, launch the installer package inside the disk image, and follow the instructions. Additional instructions can be found in the ReadMe file.

The Mac OS X Madagascar package installs the main programs, documentation files, includes files, and libraries in /usr/local/rsf. You will be asked for an administrator password. The collection of Madagascar examples included with the source code release is not yet included with this package, but that may change so check back here occasionally for updates.

===Source code distribution===

[https://sourceforge.net/project/showfiles.php?group_id=162909 Download the source code distribution securely from Sourceforge], then unpack the directory with
<pre>gunzip < madagascar-*.tar.gz | tar xvf -</pre>
or
<pre>bunzip2 < madagascar-*.tar.bz2 | tar xvf -</pre>
The <tt>bz2</tt> file is a bit smaller, but takes longer to unpack.

Next, follow [[Installation|Installation instructions]] to install.

==Current development version==

You need to have a [http://subversion.tigris.org/ Subversion client] (<tt>svn</tt>) installed. To download the directory with the Madagascar source code, run the following command:
<pre>
svn co <nowiki>https://rsf.svn.sourceforge.net/svnroot/rsf/trunk</nowiki> RSFSRC
</pre>
Replace <tt>RSFSRC</tt> with the path where you want to put the Madagascar source code.

Next, follow [[Installation|Installation instructions]] to install.

You can also [http://rsf.svn.sourceforge.net/viewvc/rsf/trunk/ browse the Subversion repository].

===Updating===

To update the Madagascar source code on your computer with the changes made by developers, <tt>cd</tt> to the directory where you placed the sources and run
<pre>
svn update
</pre>

===Troubleshooting===

*If, after running the <tt>svn co...</tt> command, nothing happens, no message, no return to the command line: you may be behind a proxy. This is especially likely if your computer is on a corporate Intranet. To get past a proxy, you need to tell Subversion: (1) The IP number or URL of the proxy and (2) the port that will allow <tt>svn</tt> through – most likely 80, the standard HTTP port. Open your <tt>~/.subversion/servers</tt> file in a text editor. If this file does not exist, running any <tt>svn</tt> command (even an unsuccessful one, like the one above) will automatically create the file. In the <tt>[global]</tt> section, add the following lines, with your own proxy URL and port names instead of the dummy ones below:
<pre>
http-proxy-host = www-proxy.yourcompany.com
http-proxy-port = 80
</pre> Now your <tt>svn</tt> commands should work. You can find more details at http://subversion.tigris.org/faq.html#proxy.

*If you get a "is already a working copy for a different URL" error, this means you have an existing directory downloaded from another server. Run <tt>svn switch --relocate</tt> to switch servers.

*If you are using an old Linux distribution (e.g. RedHat 9), the version of Subversion included with your distribution may need to be updated to handle the secure URL (<nowiki>https://</nowiki>). If <tt>svn</tt> complains about an "unrecognized URL scheme" (and you've given it the correctly formatted URL), then you need to update it.

==Other packages==
There are two other packages that might be useful in conjunction with RSF:
===Reproducible figures===

*Using Subversion, run <pre> svn co https://rsf.svn.sourceforge.net/svnroot/rsf/figs $RSFROOT/figs </pre>
This installs a wide collection of about 2500 reproducible figures. It may take a long time to download and more than 4 Gb of disk space.
The figures are preserved with the purpose of regression testing whenever the software or the environment change.

You can reproduce the figures by running <tt>scons lock</tt> in individual project directories or by going to <tt>RSF/book</tt> and running
<pre>
sftour sftour sftour scons lock
</pre>

===LATEX package===

[[SEGTeX]] is a LaTeX package for geophysical publications. It can be used with madagascar for writing [[Reproducible Documents|reproducible papers]].

RSF Comprehensive Description

2009-04-06T20:51:44Z

Mt: New page: RSF "files" actually consist of a header file and a data file. Normally one manipulates the header file. This file attempts to document the actual interface implemented in file.c ; it is...

RSF "files" actually consist of a header file and a data file. Normally one manipulates the header file.

This file attempts to document the actual interface implemented in file.c ; it is not a formal specification, and in the event of a disagreement the code should be taken as the official reference.

This is presented to facilitate reading of file.c and development of new RSF utilities.

Corrections and improvements are welcome.

==Data Files==

Data files are rectangular arrays of data. The following data formats are supported

* ASCII (C-compatible input)
* XDR (device independent binary standard)
* native binary (default)

ASCII format is most useful for debugging, while XDR format is useful for portability across formats.

Performance is optimized for native format.

The following data formats are supported:

* unsigned byte
* byte
* int (native int)
* short (2 bytes)
* float (native float)
* complex (real, imaginary float pairs)

==Header Files==

Associated with each such data file is a header file. The header file is 7 bit ASCII (UTF-7).

Lines with no "=" are considered comments and are ignored.
Lines with more than one "=" are illegal.

Lines with a single "=", with no adjacent spaces, assign a value to an alphanumeric named variable

Textstrings must be delimited by pairs of quotes. Numerical values are subject to C's parsing rules.

"in=" parameter contains the fully qualified path to the relevant data and is required

"n#" (n1, n2, n3) etc. is the number of points in a dimension. n1 is the fastest direction (maps directly onto memory).

n1 must be specified. The size of the array is the product of all n# values with the size of the fundamental type

===Conventions===

In addition to the above, many filters enforce the following conventions:

"d#" is the physical spacing in the respective dimension

"o#" is the physical origin in some absolute coordinate system of the respectove dimension

Guide to RSF file format

2009-04-06T20:17:00Z

Mt: link to formal spec page

[[Image:Fotolia_9592362_XS.jpg|right|]]
''This page was created from the LaTeX source in [http://rsf.svn.sourceforge.net/viewvc/rsf/trunk/book/rsf/rsf/format.tex?view=markup book/rsf/rsf/format.tex] using [[latex2wiki]]''

==Principles==
The main design principle behind the RSF file format is KISS ("Keep It
Simple, Stupid!"). The RSF format is borrowed from the SEPlib data format
originally designed at the Stanford Exploration Project
(Claerbout, 1991<ref>Claerbout, J. F., 1991, Introduction to Seplib and SEP utility software, ''in'' SEP-70, 413--436. Stanford Exploration Project.</ref>). The format is made as simple as possible for
maximum convenience, transparency and flexibility.
According to the Unix tradition, common file formats should be in a readable
textual form so that they can be easily examined and processed with universal
tools. Raymond (2004<ref>Raymond, E. S., 2004, The art of UNIX programming: Addison-Wesley.</ref>) writes:
<blockquote>
To design a perfect anti-Unix, make all file formats binary and opaque, and
require heavyweight tools to read and edit them.
</blockquote>
<blockquote>
If you feel an urge to design a complex binary file format, or a complex
binary application protocol, it is generally wise to lie down until the
feeling passes.
</blockquote>
Storing large-scale datasets in a text format may not be economical. RSF
chooses the next best thing: it allows data values to be stored in a binary
format but puts all data attributes in text files that can be read by humans
and processed with universal text-processing utilities.
===Example===
Let us first create some synthetic RSF data.
<pre>
bash$ sfmath n1=1000 output='sin(0.5*x1)' > sin.rsf
</pre>
Open and read the file <tt>sin.rsf</tt>.
<pre>
bash$ cat sin.rsf
sfmath rsf/rsf/rsftour: fomels@egl Sun Jul 31 07:18:48 2005

o1=0
data_format="native_float"
esize=4
in="/tmp/sin.rsf@"
x1=0
d1=1
n1=1000
</pre>
The file contains nine lines with simple readable text. The first line
shows the name of the program, the working directory, the user and
computer that created the file and the time it was created (that
information is recorded for accounting purposes). Other lines contain
parameter-value pairs separated by the "=" sign. The "in"
parameter points to the location of the binary data. Before we discuss
the meaning of parameters in more detail, let us plot the data.
<pre>
bash$ < sin.rsf sfwiggle title='One Trace' | xtpen
</pre>
On your screen, you should see a plot similar to Figure~(fig:sin1).
[[Image:sin1.png|frame|center|An example sinusoid plot.]]
Suppose you want to reformat the data so that instead of one trace of a
thousand samples, it contains twenty traces with fifty samples each. Try
running
<pre>
bash$ < sin.rsf sed 's/n1=1000/n1=100 n2=10/' > sin10.rsf
bash$ < sin10.rsf sfwiggle title=Traces | xtpen
</pre>
or (using pipes)
<pre>
bash$ < sin.rsf sed 's/n1=1000/n1=50 n2=20/' | sfwiggle title=Traces | xtpen
</pre>
On your screen, you should see a plot similar to Figure~(fig:sin2).
[[Image:sin2.png|frame|center|An example sinusoid plot, with data reformatted to twenty traces.]]
What happened? We used <tt>sed</tt>, a standard Unix line editing utility to
change the parameters describing the data dimensions. Because of the
simplicity of this operation, there is no need to create specialized data
formatting tools or to make the <tt>sfwiggle</tt> program accept additional
formatting parameters. Other general-purpose Unix tools that can be applied on
RSF files include <tt>cat</tt>, <tt>echo</tt>, <tt>grep</tt>, etc.
An alternative way to obtain the previous result is to run
<pre>
bash$ ( cat sin.rsf; echo n1=50 n2=20 ) > sin10.rsf
bash$ < sin10.rsf sfwiggle title=Traces | xtpen
</pre>
In this case, the <tt>cat</tt> utility simply copies the contents of the
previous file, and the <tt>echo</tt> utility appends new line "n1=50
n2=20". A new value of the <tt>n1</tt> parameter overwrites the old value
of <tt>n1=1000</tt>, and we achieve the same result as before.
Of course, one could also edit the file by hand with one of the general
purpose text editors. For recording the history of data processing, it is
usually preferable to be able to process files with non-interactive tools.

==Header and Data files==
A simple way to check the layout of an RSF file is with the <tt>sfin</tt>
program.
<pre>
bash$ sfin sin10.rsf
sin10.rsf:
in="/tmp/sin.rsf@"
esize=4 type=float form=native
n1=50 d1=1 o1=0
n2=20 d2=? o2=?
1000 elements 4000 bytes
</pre>
The program reports the following information: the location of the data file
(<tt>/tmp/sin.rsf\@</tt>), the element size (4 bytes), the element
type (floating point), the element form (native), the hypercube dimensions
(<math>50 \times 20</math>), axis scaling (1 and unspecified), and axis origin (0 and
unspecified). It also checks the total number of elements and bytes in the
data file.
Let us examine this information in detail. First, we can verify that the data
file exists and contains the specified number of bytes:
<pre>
bash$ ls -l /tmp/sin.rsf@
-rw-r--r-- 1 sergey users 4000 2004-10-04 00:35 /tmp/sin.rsf@
</pre>
4000 bytes in this file are required to store <math>50 \times 20</math> floating-point
4-byte numbers in a binary form. Thus, the data file contains nothing but the
raw data in a contiguous binary form.
===Datapath===
How did the RSF program (<tt>sfmath</tt>) decide where to put the data file?
In the order of priority, the rules for selecting the data file name and the
data file directory are as follows:

#Check <tt>out=</tt> parameter on the command line. The parameter specifies the output data file location explicitly.
#Specify the path and the file name separately.
#*The rules for the path selection are:
#*#Check <tt>datapath=</tt> parameter on the command line. The parameter specifies a string to prepend to the file name. The string may contain the file directory.
#*#Check <tt>DATAPATH</tt> environmental variable. It has the same meaning as the parameter specified with <tt>datapath=</tt>.
#*#Check for <tt>.datapath</tt> file in the current directory. The file may contain a line <pre> datapath=/path/to_file/ </pre> or <pre> machine_name datapath=/path/to_file/ </pre> if you indent to use different paths on different platforms.
#*#Check for <tt>.datapath</tt> file in the user home directory.
#*#Put the data file in the current directory (similar to <tt>datapath=./</tt>).
#*:
#*The rules for the filename selection are:
#*#If the output RSF file is in the current directory, the name of the data file is made by appending \@.
#*#If the output file is not in the current directory or if it is created temporarily by a program, the name is made by appending random characters to the name of the program and selected to be unique.
#*:
#:
Examples:

*
<pre>
bash$ sfspike n1=10 out=test1 > spike.rsf
bash$ grep in spike.rsf
in="test1"
</pre>
*
<pre>
bash$ sfspike n1=10 datapath=/tmp/ > spike.rsf
bash$ grep in spike.rsf
in="/tmp/spike.rsf@"
</pre>
*
<pre>
bash$ DATAPATH=/tmp/ sfspike n1=10 > spike.rsf
bash$ grep in spike.rsf
in="/tmp/spike.rsf@"
</pre>
*
<pre>
bash$ sfspike n1=10 datapath=/tmp/ > /tmp/spike.rsf
bash$ grep in /tmp/spike.rsf
in="/tmp/sfspikejcARVf"
</pre>

====Packing header and data together====
While the header and data files are separated by default, it is also possible
to pack them together into one file. To do that, specify the program's
"<tt>out</tt>" parameter as <tt>out=stdout</tt>. Example:
<pre>
bash$ sfspike n1=10 out=stdout > spike.rsf
bash$ grep in spike.rsf
Binary file spike.rsf matches
bash$ sfin spike.rsf
spike.rsf:
in="stdin"
esize=4 type=float form=native
n1=10 d1=0.004 o1=0 label1="Time" unit1="s"
10 elements 40 bytes
bash$ ls -l spike.rsf
-rw-r--r-- 1 sergey users 196 2004-11-10 21:39 spike.rsf
</pre>
If you examine the contents of <tt>spike.rsf</tt>, you will find that it
starts with the text header information, followed by special
symbols, followed by binary data.
Packing headers and data together may not be a good idea for data processing
but it works well for storing data: it is easier to move the packed file
around than to move two different files (header and binary) together while
remembering to preserve their connection. Packing header and data together is
also the current mechanism used to push RSF files through Unix pipes.

===Type===
The data stored with RSF can have different types: character, unsigned
character, integer, floating point, or complex. By default, single precision
is used for numbers (<tt>int</tt> and <tt>float</tt> data types in the C
programming language). The number of bytes required for represent these
numbers may depend on the platform.
===Form===
The data stored with RSF can also be in a different form: ASCII, native
binary, and XDR binary. Native binary is often used by default. It is the
binary format employed by the machine that is running the application. On
Linux-running PC, the native binary format will typically correspond to the
so-called little-endian byte ordering. On some other platform, it might be
big-endian ordering. XDR is a binary format designed by Sun for exchanging
files over network. It typically corresponds to big-endian byte ordering. It
is more efficient to process RSF files in the native binary format but, if you
intend to access data from different platforms, it might be a good idea to
store the corresponding file in an XDR format. RSF also allows for an ASCII
(plain text) form of data files.
Conversion between different types and forms is accomplished with
<tt>sfdd</tt> program. Here are some examples. First, let us create synthetic
data.
<pre>
bash$ sfmath n1=10 output='10*sin(0.5*x1)' > sin.rsf
bash$ sfin sin.rsf
sin.rsf:
in="/tmp/sin.rsf@"
esize=4 type=float form=native
n1=10 d1=1 o1=0
10 elements 40 bytes
bash$ < sin.rsf sfdisfil
0: 0 4.794 8.415 9.975 9.093
5: 5.985 1.411 -3.508 -7.568 -9.775
</pre>
Converting the data to the integer type:
<pre>
bash$ < sin.rsf sfdd type=int > isin.rsf
bash$ sfin isin.rsf
isin.rsf:
in="/tmp/isin.rsf@"
esize=4 type=int form=native
n1=10 d1=1 o1=0
10 elements 40 bytes
bash$ < isin.rsf sfdisfil
0: 0 4 8 9 9 5 1 -3 -7 -9
</pre>
Converting the data to the ASCII form:
<pre>
bash$ < sin.rsf sfdd form=ascii > asin.rsf
bash$ < asin.rsf sfdisfil
0: 0 4.794 8.415 9.975 9.093
5: 5.985 1.411 -3.508 -7.568 -9.775
bash$ sfin asin.rsf
asin.rsf:
in="/tmp/asin.rsf@"
esize=0 type=float form=ascii
n1=10 d1=1 o1=0
10 elements
bash$ cat /tmp/asin.rsf@
0 4.79426 8.41471 9.97495 9.09297 5.98472 1.4112 -3.50783
-7.56803 -9.7753
</pre>

===Hypercube===
While RSF stores binary data in a contiguous 1-D array, the conceptual
data model is a multidimensional hypercube. By convention, the
dimensions of the cube are defined with parameters <tt>n1</tt>,
<tt>n2</tt>, <tt>n3</tt>, etc. The fastest axis is <tt>n1</tt>.
Additionally, the grid sampling can be given by parameters
<tt>d1</tt>, <tt>d2</tt>, <tt>d3</tt>, etc. The axes origins are given
by parameters <tt>o1</tt>, <tt>o2</tt>, <tt>o3</tt>, etc. Optionally,
you can also supply the axis label strings <tt>label1</tt>,
<tt>label2</tt>, <tt>label3</tt>, etc., and axis units strings
<tt>unit1</tt>, <tt>unit2</tt>, <tt>unit3</tt>, etc.
==Compatibility with other file formats==
It is possible to exchange RSF-formatted data with other popular data formats.
===Compatibility with SEPlib===
RSF is mostly compatible with its predecessor, the SEPlib file format.
However, there are several important differences:

#SEPlib programs typically use the element size (<tt>esize=</tt> parameter) to distinguish between different data types: <tt>esize=4</tt> corresponds to floating point data, while <tt>esize=8</tt> corresponds to complex data. The RSF type handling mechanism is different: data types are determined from the value of the <tt>data_format</tt> parameter. Madagascar computational programs typically output files with <tt>data_format="native_float"</tt> or <tt>native_complex</tt>.
#The default data form in SEPlib programs is typically XDR and not native as it is in RSF. Thus, to make a dataset created with SEPlib readable by Madagascar programs, you would typically need to add to the history file <tt>data_format="xdr_float"</tt> or <tt>data_format="xdr_complex"</tt> .
#It is possible to pipe the output of Madagascar programs to SEPlib: <pre> bash$ sfspike n1=1 | Attr want=min minimum value = 1 at 1 </pre> However, piping the output of SEPlib programs to RSF (or, for that matter, any other non-SEPlib programs) will result in an unterminated process. Do not try <pre> bash$ Spike n1=1 | sfattr want=min </pre> That happens because SEPlib uses sockets for piping and expects a socket connection from the receiving program. RSF passes data through regular Unix pipes.
#SEP3D is an extension of SEPlib for operating with irregularly sampled data (Biondi et al., 1996<ref>Biondi, B., R. Clapp, and S. Crawley, 1996, SEPlib90: SEPlib for 3-D prestack data, ''in'' SEP-92, 343--364. Stanford Exploration Project.</ref>). There is no equivalent of it in RSF for the reasons explained in the beginning of this guide. Operations with irregular datasets are supported through the use of auxiliary input files that represent the geometry information.

===Reading and writing SEG-Y and SU files===
The SEG-Y format is based on the proposal of Barry et al. (1975<ref>[http://www.seg.org/SEGportalWEBproject/prod/SEG-Publications/Pub-Technical-Standards/Documents/seg_y_rev0.pdf Barry, K. M., D. A. Cavers, and C. W. Kneale, 1975, Report on recommended standards for digital tape formats: Geophysics, '''40''', 344--352]</ref>).
It was revised in 2002<ref>See http://www.seg.org/SEGportalWEBproject/prod/SEG-Publications/Pub-Technical-Standards/Documents/seg_y_rev1.pdf</ref>. The
SU format is a modification of SEG-Y used in Seismic Unix
(Stockwell, 1997<ref>Stockwell, J. W., 1997, Free software in education: A case study of CWP/SU: Seismic Unix: The Leading Edge, '''16''', 1045--1049.</ref>).
To convert files from SEG-Y or SU format to RSF, use the <tt>sfsegyread</tt>
program. Let us first manufacture an example file using SU utilities
(Stockwell, 1999<ref>-------- 1999, The CWP/SU: Seismic Un*x package: Computers and Geosciences, '''25''', 415--419.</ref>):
<pre>
bash$ suplane > plane.su
bash$ segyhdrs < plane.su | segywrite tape=plane.segy
</pre>
To convert it to RSF, use either
<pre>
bash$ sfsuread < plane.su tfile=tfile.rsf endian=0 > plane.rsf
</pre>
or
<pre>
bash$ sfsegyread < plane.segy tfile=tfile.rsf \
hfile=hfile bfile=bfile endian=0 > plane.rsf
</pre>
The endian flag is needed if the SU file originated from a little-endian
machine such as Linux PC.
Several files are generated. The standard output contains an RSF file with the
data (32 traces with 64 samples each):
<pre>
bash$ sfin plane.rsf
plane.rsf:
in="/tmp/plane.rsf@"
esize=4 type=float form=native
n1=64 d1=0.004 o1=0
n2=32 d2=? o2=?
2048 elements 8192 bytes
</pre>
The contents of this file are displayed in the figure.
[[Image:plane.png|frame|center|The output of suplane, converted to RSF and
displayed with <tt>sfwiggle</tt>.]]
The <tt>tfile</tt> is an RSF integer-type file with the trace headers (32
headers with 71 traces each):
<pre>
bash$ sfin tfile.rsf
tfile.rsf:
in="/tmp/tfile.rsf@"
esize=4 type=int form=native
n1=71 d1=? o1=?
n2=32 d2=? o2=?
2272 elements 9088 bytes
</pre>
The contents of trace headers can be quickly examined with the
<tt>sfheaderattr</tt> program.
The <tt>hfile</tt> is the ASCII header file for the whole record.
<pre>
bash$ head -c 242 hfile
C This tape was made at the
C
C Center for Wave Phenomena
</pre>
The <tt>bfile</tt> is the binary header file.

To convert files back from RSF to SEG-Y or SU, use the <tt>sfsegywrite</tt>
program and reverse the input and output:
<pre>
bash$ sfsuwrite > spike.su su=y tfile=tfile.rsf endian=0 < spike.rsf
</pre>
or
<pre>
bash$ sfsegywrite > spike.segy tfile=tfile.rsf \
hfile=hfile bfile=bfile endian=0 < spike.rsf
</pre>

If <tt>hfile=</tt> and <tt>bfile=</tt> are not supplied to <tt>sfsegywrite</tt>, the corresponding headers will be either picked from the default locations (files named <tt>header</tt> and <tt>binary</tt>) or generated on the fly. The trace header file can be generated with <tt>sfsegyheader</tt>. Here is an example:
<pre>
bash$ rm header binary
bash$ sfheadermath < spike.rsf output=N+1 | sfdd type=int > tracl.rsf
bash$ sfsegyheader < spike.rsf tracl=tracl.rsf > tfile.rsf
bash$ sfsegywrite < spike.rsf tfile=tfile.rsf > spike.segy
</pre>

===Reading and writing ASCII files===
Reading and writing ASCII files can be accomplished with the <tt>sfdd</tt>
program. For example, let us take an ASCII file with numbers
<pre>
bash$ cat file.asc
1.0 1.5 3.0
4.8 9.1 7.3
</pre>
Converting it to RSF is as simple as
<pre>
bash$ echo in=file.asc n1=3 n2=2 data_format=ascii_float > file.rsf
bash$ sfin file.rsf
file.rsf:
in="file.asc"
esize=0 type=float form=ascii
n1=3 d1=? o1=?
n2=2 d2=? o2=?
6 elements
</pre>
For more efficient input/output operations, it might be advantageous to
convert the data type to native binary, as follows:
<pre>
bash$ echo in=file.asc n1=3 n2=2 data_format=ascii_float | \
sfdd form=native > file.rsf
bash$ sfin file.rsf
file.rsf:
in="/tmp/file.rsf@"
esize=4 type=float form=native
n1=3 d1=? o1=?
n2=2 d2=? o2=?
6 elements 24 bytes
</pre>
Convert from RSF to ASCII is equally simple:
<pre>
bash$ sfdd form=ascii out=file.asc < file.rsf > /dev/null
bash$ cat file.asc
1 1.5 3 4.8 9.1 7.3
</pre>
You can use the <tt>line=</tt> and <tt>format=</tt> parameters in
<tt>sfdd</tt> to control the ASCII formatting:
<pre>
bash$ sfdd form=ascii out=file.asc \
line=3 format="%3.1f " < file.rsf > /dev/null
bash$ cat file.asc
1.0 1.5 3.0
4.8 9.1 7.3
</pre>
An alternative is to use <tt>sfdisfil</tt>.
<pre>
bash$ sfdisfil > file.asc col=3 format="%3.1f " number=n < file.rsf
bash$ cat file.asc
1.0 1.5 3.0
4.8 9.1 7.3
</pre>

==Other documentation==
This note should give you a general understanding of the RSF file
format.

For more details see the draft [[RSF Data Format Specification]] .

Other relevant documentation is

*[[Introduction to madagascar]]
*[[Why Madagascar]]
*[[Installation|Installation instructions]]
*[http://reproducibility.org/RSF/ Madagascar self-documentation]
*[[Guide to madagascar programs]]
*[[Guide to madagascar API|Guide to the Madagascar programming interface]]
*[[Guide to programming with madagascar]]
*[[Revisiting SEP tour with Madagascar and SCons]]
*[[Reproducible computational experiments using SCons]]

==References==
<references/>

Main Page

2009-03-13T16:37:10Z

Mt:

<big>'''Madagascar'''</big> is an open-source software package for multidimensional data analysis and [[Reproducibility|reproducible]] computational experiments. Its mission is to provide
* a convenient and powerful environment
* a convenient technology transfer tool
for researchers working with digital image and data processing in geophysics and related fields. Technology developed using the Madagascar project management system is transferred in the form of recorded processing histories, which become "computational recipes" to be verified, exchanged, and modified by users of the system.

== Features ==

*Madagascar is a modern package. Started in 2003, and publicly released in 2006 it was developed entirely from scratch. Being a relatively new package, it follows modern software engineering practices such as module encapsulation and test-driven development. A rapid development of a project of this scope (more than 300 main programs and more than 3,000 tests) would not be possible without standing on the shoulders of giants and learning from the 30 years of previous experience in open packages such as SEPlib and Seismic Unix. We have borrowed and reimplemented functionality and ideas from these other packages.
*Madagascar is a test-driven package. Test-driven development is not only an agile software programming practice but also a way of bringing scientific foundation to geophysical research that involves numerical experiments. Bringing reproducibility and peer review, the backbone of any real science, to the field of computational geophysics is the main motivation for Madagascar development. The package consists of two levels: low-level main programs (typically developed in the C programming language and working as data filters) and high-level processing flows (described with the help of the Python programming language) that combine main programs and completely document data processing histories for testing and reproducibility. Experience shows that high-level programming is easily mastered even by beginning students that have no previous programming experience.
*Madagascar is an open-source package. It is distributed under the standard GPL open-source license, which places no restriction on the usage and modification of the code. Moreover, access to modifying the source repository is not controlled by one organization but shared equally among different developers. This enables an open collaboration among different groups spread all over the world, in the true spirit of the open source movement.

*Madagascar uses a simple, flexible, and universal data format that can handle very large datasets but is not tied specifically to seismic data or data of any other particular kind. This "regularly sampled" format is borrowed from the traditional SEPlib. A universal data format allows us to share general-purpose data processing tools with scientists from other disciplines such as petroleum engineers working on large-scale reservoir simulations.

== Google Summer of Code ==

We have applied to participate in the Google Summer of Code. Here is our [[GSOC2009]] ideas page.

== Latest News ==
{| border="1" cellpadding="5" cellspacing="0"
| style="background-color:#ffdead;font-weight:bold;font-style:italic;color:#903" | Madagascar event of the year: [[Delft 2009|School on Reproducible Computational Geophysics]] in Delft, the Netherlands, on June 12-13, 2009
|}
{{#widget:Feed
|feedurl=http://www.reproducibility.org/rsflog/index.php?/feeds/index.rss2
|chan=title
|num=7
|desc=100
|date=n
|targ=n
}}

Main Page

2009-03-13T16:34:38Z

Mt: /* Seeking Testimonials */

<big>'''Madagascar'''</big> is an open-source software package for multidimensional data analysis and [[Reproducibility|reproducible]] computational experiments. Its mission is to provide
* a convenient and powerful environment
* a convenient technology transfer tool
for researchers working with digital image and data processing in geophysics and related fields. Technology developed using the Madagascar project management system is transferred in the form of recorded processing histories, which become "computational recipes" to be verified, exchanged, and modified by users of the system.

== Features ==

*Madagascar is a modern package. Started in 2003, and publicly released in 2006 it was developed entirely from scratch. Being a relatively new package, it follows modern software engineering practices such as module encapsulation and test-driven development. A rapid development of a project of this scope (more than 300 main programs and more than 3,000 tests) would not be possible without standing on the shoulders of giants and learning from the 30 years of previous experience in open packages such as SEPlib and Seismic Unix. We have borrowed and reimplemented functionality and ideas from these other packages.
*Madagascar is a test-driven package. Test-driven development is not only an agile software programming practice but also a way of bringing scientific foundation to geophysical research that involves numerical experiments. Bringing reproducibility and peer review, the backbone of any real science, to the field of computational geophysics is the main motivation for Madagascar development. The package consists of two levels: low-level main programs (typically developed in the C programming language and working as data filters) and high-level processing flows (described with the help of the Python programming language) that combine main programs and completely document data processing histories for testing and reproducibility. Experience shows that high-level programming is easily mastered even by beginning students that have no previous programming experience.
*Madagascar is an open-source package. It is distributed under the standard GPL open-source license, which places no restriction on the usage and modification of the code. Moreover, access to modifying the source repository is not controlled by one organization but shared equally among different developers. This enables an open collaboration among different groups spread all over the world, in the true spirit of the open source movement.

*Madagascar uses a simple, flexible, and universal data format that can handle very large datasets but is not tied specifically to seismic data or data of any other particular kind. This "regularly sampled" format is borrowed from the traditional SEPlib. A universal data format allows us to share general-purpose data processing tools with scientists from other disciplines such as petroleum engineers working on large-scale reservoir simulations.

== Google Summer of Code ==

We have applied to participate in the Google Summer of Code. Here is our [[Ideas Page]].

== Latest News ==
{| border="1" cellpadding="5" cellspacing="0"
| style="background-color:#ffdead;font-weight:bold;font-style:italic;color:#903" | Madagascar event of the year: [[Delft 2009|School on Reproducible Computational Geophysics]] in Delft, the Netherlands, on June 12-13, 2009
|}
{{#widget:Feed
|feedurl=http://www.reproducibility.org/rsflog/index.php?/feeds/index.rss2
|chan=title
|num=7
|desc=100
|date=n
|targ=n
}}

GSOC2009

2009-03-13T16:26:39Z

Mt: /* Geophysics / Numerical Analysis (Mentor: Paul Sava) */

{|border="1" cellpadding="5" cellspacing="0"
|-
| style="background:#efefef;" |
[http://code.google.com/opensource/gsoc/2009/faqs.html Google Summer of Code] is a program that offers student developers stipends to write code for various open source projects. Google will be working with several open source, free software, and technology-related groups to identify and fund several projects over a three month period.
|}

 

[[Image:Google2.png|frame|right|[http://code.google.com/soc/ Google Summer of Code]]]

=Welcome to Madagascar's Google Summer of Code Page=

Madagascar, an open source project, is a leading participant in the [http://en.wikipedia.org/wiki/Open_research Open Research movement]. As described on Wikipedia, the central theme of open research is to make clear accounts of the methodology, along with data and results extracted therefrom, freely available via the internet. This permits a massively distributed collaboration.

Its design is based on a few simple and powerful principles.

From the coder's point of view, Madagascar is written in C and in Python. The C library is a very loosely coupled set of [http://en.wikipedia.org/wiki/Filter_(Unix) unix-style filters], transforming stdin to stdout. The Python is mostly an implementation of a custom build system on top of the rule based build system [http://www.scons.org/ SCons].

Seismic data processing consists of a sequence of steps. Madagascar's filter-based design allows such sequences to be easily composed and abstracted. A key advantage of the Madagascar system is that the computational pipeline is also construed as a build system. Modifications to intermediate steps automatically reinvoke only necessary computations and skip over up-to-date ones, just as a more conventional build system would recompile modules whose code had been touched while reusing modules which are newer than their source. Madagascar extends this model all the way from raw data to publication.

This strategy is a key to [http://csdl2.computer.org/comp/mags/cs/2009/01/mcs2009010005.pdf reproducibility]. By maintaining scripts which contain all transformations from raw data to final publication quality document, Madagascar supports repeatability and testing of scientific computations, thus advancing the collaborative nature of science in the same way that open source advances the collaborative nature of computing.

Directions in which Madagascar is expanding include visualization, parallelization, and user interfaces.

=Project Ideas=

See also the [http://sourceforge.net/tracker/?group_id=162909&atid=825648 feature request tracker].

==Graphical User Interface (''Mentor: Sergey Fomel'')==
* Add an option to [http://rsf.svn.sourceforge.net/viewvc/rsf/trunk/framework/rsfdoc.py?view=markup sfdoc] to output spec files in the format defined for [http://www.henrythorson.com/interface.htm TKSU]. This should make '''TKSU''' immediately applicable. Spec files can be generated automatically at the compile time.
* Rewrite '''TKSU''' in Python, possibly using [http://wiki.python.org/moin/TkInter TkInter]
* See http://sourceforge.net/forum/forum.php?thread_id=1579059&forum_id=552249 for more discussions.
* Investigate alternative solutions.
* skills and interests: GUI, Tcl/Tk, Python

==Data Visualization (''Mentor: Vladimir Bashkardin'')==
* Migrate 2D rendering OpenGL-based code from GSEGYView to Madagascar and create an interactive viewer with zooming/panning features.
* Migrate 3D rendering GLSL-based code from GSEGYView to Madagascar and create a viewer with the support of pluggable shader programs.
* Finish 3D rays viewer
* Create a set of alternatives to sfgraph, sfgrey, sfcontour programs, that would use PLPLOT library instead of VPlot; also, create "pens", that could read from those programs and generate ps, pdf, png output; analyze flexibility of PLPLOT and the possibility to fully mimic VPlot's output (including animation).
* Skills and interests: scientific visualization

==Interactive UI (''Mentor: Michael Tobis'')==
* test and build Python wrappers around existing function to create a novel inetractive environment which is both interactive and reproducible
* help refactor existing SCons scripts to reduce coupling and increase clarity
* integrate with iPython/sage environment
* skills and interests: strong OOP and test-driven development, Python. SCons a big plus.

==Geophysics / Numerical Analysis (''Mentor: Paul Sava'')==
* Implement an optimal algorithm for parallel transposes of arrays with 4 or 5 dimensions, up to a few tens of terabytes in volume, on a multi-node Linux cluster
* As a bonus, FFT one of the transposed dimensions
* Implement a hardware-adaptive transpose algorithm for a 1-node, SMP machine of 8 nodes or more. Investigate speed of transfers, size of caches, memory arrangement, etc, and make it hardware-adaptive. Bonus for out-of-core capabilities.
* Implement 3-D seismic data header storage using the fastest open-source database, then compare header I/O times with the classic approach of having a simple table. Which is the fastest way of implementing a large database knowing that the values it will hold are all bools, ints and floats?
* skills and interests: numerical methods, scientific computation, parallel computing

2009-03-09T17:15:39Z

Mt:

<big>'''Madagascar'''</big> is an open-source software package for multidimensional data analysis and [[Reproducibility|reproducible]] computational experiments. Its mission is to provide
* a convenient and powerful environment
* a convenient technology transfer tool
for researchers working with digital image and data processing in geophysics and related fields. Technology developed using the Madagascar project management system is transferred in the form of recorded processing histories, which become "computational recipes" to be verified, exchanged, and modified by users of the system.

== Features ==

*Madagascar is a modern package. Started in 2003, and publicly released in 2006 it was developed entirely from scratch. Being a relatively new package, it follows modern software engineering practices such as module encapsulation and test-driven development. A rapid development of a project of this scope (more than 300 main programs and more than 3,000 tests) would not be possible without standing on the shoulders of giants and learning from the 30 years of previous experience in open packages such as SEPlib and Seismic Unix. We have borrowed and reimplemented functionality and ideas from these other packages.
*Madagascar is a test-driven package. Test-driven development is not only an agile software programming practice but also a way of bringing scientific foundation to geophysical research that involves numerical experiments. Bringing reproducibility and peer review, the backbone of any real science, to the field of computational geophysics is the main motivation for Madagascar development. The package consists of two levels: low-level main programs (typically developed in the C programming language and working as data filters) and high-level processing flows (described with the help of the Python programming language) that combine main programs and completely document data processing histories for testing and reproducibility. Experience shows that high-level programming is easily mastered even by beginning students that have no previous programming experience.
*Madagascar is an open-source package. It is distributed under the standard GPL open-source license, which places no restriction on the usage and modification of the code. Moreover, access to modifying the source repository is not controlled by one organization but shared equally among different developers. This enables an open collaboration among different groups spread all over the world, in the true spirit of the open source movement.

== Seeking Testimonials ==

Are you a satisfied user of Madagascar? Are you hoping it gets even better? You can help by contributing to a [[Testimonials]] page. Please describe how Madagascar helps you get what you need done.
*Madagascar uses a simple, flexible, and universal data format that can handle very large datasets but is not tied specifically to seismic data or data of any other particular kind. This "regularly sampled" format is borrowed from the traditional SEPlib. A universal data format allows us to share general-purpose data processing tools with scientists from other disciplines such as petroleum engineers working on large-scale reservoir simulations.

== Latest News ==
{| border="1" cellpadding="5" cellspacing="0"
| style="background-color:#ffdead;font-weight:bold;font-style:italic;color:#903" | Madagascar event of the year: [[Delft 2009|School on Reproducible Computational Geophysics]] in Delft, the Netherlands, on June 12-13, 2009
|}
{{#widget:Feed
|feedurl=http://www.reproducibility.org/rsflog/index.php?/feeds/index.rss2
|chan=title
|num=7
|desc=100
|date=n
|targ=n
}}