Difference between revisions of "Introduction to madagascar"

From Madagascar
Jump to navigation Jump to search
(transfer contents)
 
m
 
(12 intermediate revisions by the same user not shown)
Line 1: Line 1:
<center><font size="-1">''This page was created from the LaTeX source in [http://rsf.svn.sourceforge.net/viewcvs.cgi/rsf/trunk/book/rsf/rsf/paper.tex?view=markup book/rsf/rsf/paper.tex] using [[latex2wiki]]''</font></center>
+
This page was originally created from the LaTeX source in [http://rsf.svn.sourceforge.net/viewvc/rsf/trunk/book/rsf/rsf/paper.tex?view=markup book/rsf/rsf/paper.tex] using [[latex2wiki]]. Contents that were not duplicating the main page were eventually moved to the [[Package overview]] page.
 
 
Madagascar is an open-source software package for geophysical data
 
processing and reproducible numerical experiments. Its mission is
 
to provide
 
#a convenient and powerful environment
 
#a convenient technology transfer tool
 
for researchers working with digital image and data processing. The
 
package is available in an open-source form to allow effective
 
collaboration of a wide community of developers. The technology
 
developed using the Madagascar project management system is
 
transferred in the form of recorded processing histories, which become
 
"computational recipes" to be verified, exchanged, and modified by
 
the users of the system.
 
 
 
==What is Madagascar?==
 
 
 
Madagascar is an open-source software package for geophysical data processing
 
and reproducible numerical experiments. Its most noticeable features are:
 
 
 
#Madagascar is a relatively ''new''  package. It started in 2003, was developed entirely from scratch, and publicly released in 2006. Being a new package, it follows modern software engineering practices such as module encapsulation and test-driven development. A rapid development of a project of this scope (more than 300 main programs and more than 3,000 tests) would not be possible without standing on the shoulders of giants and learning from the 30 years of previous experience in open packages such as SEPlib and Seismic Unix. We have borrowed and reimplemented functionality and ideas from these other packages.
 
#Madagascar is a ''test-driven''  package. Test-driven development is not only an agile software programming practice but also a way of bringing scientific foundation to geophysical research that involves numerical experiments. Bringing reproducibility and peer review, the backbone of any real science, to the field of computational geophysics is the main motivation for Madagascar development. The package consists of two levels: low-level main programs (typically developed in the C programming language and working as data filters) and high-level processing flows (described with the help of the Python programming language) that combine main programs and completely document data processing histories for testing and reproducibility. Experience shows that high-level programming is easily mastered even by beginning students that have no previous programming experience.
 
#Madagascar is an ''open-source''  package. It is distributed under the standard GPL open-source license, which places no restriction on the usage and modification of the code. Moreover, access to modifying the source repository is not controlled by one organization but shared equally among different developers. This enables an open collaboration among different groups spread all over the world, in the true spirit of the open source movement.
 
#Madagascar uses a ''simple, flexible, and universal''  data format that can handle very large datasets but is not tied specifically to seismic data or data of any other particular kind. This "regularly sampled" format is borrowed from the traditional SEPlib. A universal data format allows us to share general-purpose data processing tools with scientists from other disciplines such as petroleum engineers working on large-scale reservoir simulations.
 
==What is in Madagascar?==
 
 
 
The package consists of
 
 
 
#A collection of main programs. Most programs act as filters on input data and can be chained in a Unix pipeline. For example: <pre> < data.rsf sfwindow n1=100 | sfbandpass fhi=60 > data2.rsf </pre> This approach follows the Unix philosophy, as formulated by Doug McIlroy, the inventor of Unix pipes (Salus, 1994<ref>Salus, P. H.,  1994, A quarter-century of Unix: Addison-Wesley.</ref>): 
 
##Write programs that do one thing and do it well. 
 
##Write programs to work together. 
 
##Write programs to handle text streams, because that is a universal interface.
 
#: Running a command (such as <tt>sfwindow</tt>) without parameters or the necessary input and output files shows a brief documentation, explaining the program purpose and parameters.  Alternatively, brief documentation is provided by <tt>sfdoc</tt> program. Main program documentation in HTML format is available on the web at http://www.reproducibility.org/RSF/. Madagascar uses ''Regularly Sampled Format''  (RSF) for data files, which is similar to the format used in the SEPlib library developed at the Stanford Exploration Project (SEP). The file format describes regularly sampled hypercubes. Up to 9 dimensions are supported. In accordance with the Unix philosophy, each RSF file (such as <tt>data.rsf</tt>) is a simple readable text. It contains a pointer (<tt>in=</tt> parameter) to the location of the binary data. Madagascar provides programs for conversion to and from other formats such as SEG-Y and SU. Madagascar currently adopts Vplot file format, also developed at SEP, for generated graphics files.
 
#An API (application programmer's interface) for programmers writing their own software to manipulate RSF files. The main software language of the Madagascar package is C. Interfaces to other languages (C++, Fortran-77, Fortran-90, Python, Matlab) are also provided.
 
#A project management system. The system uses and extends [http://www.scons.org/ SCons], an open-source software construction package, to document and maintain data processing flows. Documented projects become computational recipes that can be easily exchanged among Madagascar users.
 
#A collection of reproducible documents, organized in living books. Each reproducible book contains a collection of Madagascar recipes (<tt>SConstruct</tt> files) used to generate book figures. The recipes cover a variety of data processing and imaging tasks described in the books. Figures and recipes serve dual purpose with respect to Madagascar maintenance. They provide demos for introducing new users to the functionality of the package and, at the same time, regression tests for assuring the system stability under change.
 
Follow the links at the end of this paper for additional documentation.
 
 
 
==Copyright notice==
 
 
 
The Madagascar package is released in an open-source form under the standard GNU GPL license. In simple words, there are no restrictions on the use of the software (including copying, modifying, selling, etc.) However, there are restrictions on the software redistribution intended to prevent the package from losing its open-source status. Users are encourages to submit their modifications back to the original distribution to the benefit of the whole user community.
 
 
 
==Alternatives==
 
 
 
In the present form, the Madagascar package, while being completely written from scratch, borrows ideas from the design of [http://sepwww.stanford.edu/software/seplib/ SEPlib], a publicly available software package, maintained by Bob Clapp at the Stanford Exploration Project. Generations of SEP students and researchers contributed to SEPlib. Most important contributions came from Rob Clayton, Jon Claerbout, Dave Hale, Stew Levin, Rick Ottolini, Joe Dellinger, Steve Cole, Dave Nichols, Martin Karrenbach, Biondo Biondi, and Bob Clapp.
 
 
 
Madagascar also borrows ideas from [http://timna.mines.edu/cwpcodes/ Seismic Unix] (SU), a package maintained by John Stockwell at the Center for Wave Phenomenon at the Colorado School of Mines (Stockwell, 1997<ref>Stockwell, J. W.,  1997, Free software in education: A case study of CWP/SU: Seismic Unix: The Leading Edge, '''16''', 1045--1049.</ref>;Stockwell, 1999<ref>--------, 1999, The CWP/SU: Seismic Un*x package: Computers and  Geosciences, '''25''', 415--419.</ref>). Main contributors to SU included Einar Kjartansson, Shuki Ronen, Jack Cohen, Chris Liner, Dave Hale, and John Stockwell. SU is open-source software (distributed with BSD-style license) starting with release 40 (April 10, 2007).
 
 
 
Another option for a seismic processing system is [http://www.freeusp.org/ Free USP]. USP is a processing package originally developed by Amoco and released by BP. Another package, [http://www.freeusp.org/DDS/ DDS] (Data Dictionary System) was also released by BP. Also, ConocoPhillips released its older processing system, [http://www.opengeophysical.com/downloads CPSeis], under an [http://www.opensource.org/docs/definition.php open-source] license.
 
 
 
There are also other publicly available packages, with a smaller number of utilities than the ones described above, but each having unique capabilities, in its own way.
 
 
 
{| class="wikitable" align="center"
 
! colspan="4" style="background:#ffdead;" | GPL-compatible licenses
 
|-
 
! Name || What it is || Written/maintained by ||  License
 
|-
 
| [http://www-geo.phys.ualberta.ca/saig/SeismicLab/index.html SeismicLab] || Matlab toolbox that does preprocessing, imaging and plotting || Mauricio Sacchi (U. of Alberta, Canada) || GPL
 
|-
 
| [http://segymat.sourceforge.net/ SegyMAT] || toolbox to read and write SEG-Y data to and from Matlab and Octave || Thomas Mejer Hansen (U. of Copenhagen, Denmark) ||  LGPL
 
|-
 
| [http://segymat.sourceforge.net/segypy/ SegyPy] || Python port of SegyMAT || same as SegyMAT || LGPL
 
|-
 
| [http://www.icm.csic.es/geo/gma/graf2segy.html image2segy] || Matlab program to transform a raster image of a seismic paper or film record to SEGY. Uses SegyMat. || Marcelli Farran (Institute of Marince Sciences, Barcelona, Spain) || Creative Commons
 
|-
 
| [http://gsegyview.sourceforge.net/ GSEGYView] || cross-platform SEG-Y data viewer with OpenGL graphics hardware acceleration || Vladimir Bashkardin (U. of Texas at Austin) || GPL
 
|-
 
| [http://www.kogeo.de/ kogeo] || MS-Windows only toolkit that features data processing, project databases, interpretation tools, 3-D header manipulation and good visualization tools || Philipp Konerding (U. of Hamburg, Germany) || GPL
 
|-
 
| [http://qiworkbench.org/ qiWorkbench] || extensible Java-based platform for implementing integrated workflows to process, analyze and view seismic data, originally designed by BHP Billiton to implement its integrated workflows to process, analyze and view seismic data || BHP, G&W, INT, CSM and CSIRO || GPL; BSD for APIs in order to allow closed-source commercial plugins
 
|-
 
|[http://www-old.dpr.csiro.au/StochasticSeismicInversion/index.html Delivery]|| Java-based Bayesian seismic inversion code for use in oil reservoir characterisation || CSIRO Petroleum, Australia || The copyright belongs to BHP Billiton and the package is distributed under a GPL+BSD license.
 
|-
 
| [http://geocomputing.narod.ru/benchmark.html GeoBenchmark] || Benchmark for how fast computers are when working with seismic processing and imaging algorithms. More details in "Computers for seismic processing and imaging: a performance study", by E. Kurin, Proceedings of the 2007 SEG Annual Meeting, 2451-2454 || Evgeny Kurin, Geolab Ltd. || public domain
 
|}
 
 
 
{| class="wikitable" align="center"
 
! colspan="4" style="background:#ffdead;" | Other/missing/ambiguous/GPL-incompatible licenses
 
|-
 
! Name || What it is || Written/maintained by || License
 
|-
 
| [http://www.crewes.com/Samples/EduSoftware/ CREWES Educational Software Release] || Matlab toolbox for seismic processing and imaging || CREWES consortium (U. of Calgary, Canada) || Free for non-commercial use
 
|-
 
| [http://sw3d.mff.cuni.cz/software/ SW3D] || good-quality ray-theory based package || SW3D consortium (Charles U., Czech Republic) ||  Not specified
 
|-
 
| [http://bullard.esc.cam.ac.uk/~hobro/Jive3D/ Jive3D] || forward-modelling and tomographic inversion package that is capable of modelling a wide range of seismic travel-time data types || James Hobro (Cambridge U., UK) || Free for noncommercial use or sponsors of the consortium
 
|-
 
| [http://www.geophysics.rice.edu/department/faculty/zelt/rayinvr.html RayInvr] || package that does 2-D ray tracing, traveltime inversion, amplitude calculation and synthetics. Accompanied by a package called zplot for interactive plotting and picking of 2-D and 3-D wide-angle seismic data. || Colin Zelt (Rice U., USA) || Noncommercial use only
 
|-
 
| [http://www.seismo.unr.edu/jrg/ JRG] || Java-based basic reflection processing package with graphics, 3-d and crooked-line capabilities, SEG-Y and sound file I/O, and a GUI || John Louie (U. of Nevada at Reno, USA) || "The software and methods here are the subject of academic research, not commercial products. I would like to know what use you make of my methods, and have your feedback on their success or failure.". Also mention in the title that the package is open-source software.
 
|-
 
| [http://www.mines.edu/~dhale/jtk/ Mines Java Toolkit] || set of Java packages and native (non-Java) code libraries for digital signal processing and 2-D and 3-D graphics || Dave Hale (Colorado School of Mines, USA) || Common Public License
 
|-
 
| [http://seisweb.usask.ca/igeos/ IGeoS - Integrated GeoScience data analysis] || many seismic processing tasks in a wide range of geophysical, and ultimately geoscience data analysis. || Igor Morozov (U. of Saskatchewan, Canada) ||
 
Free for noncommercial use; commercial license available
 
|}
 
  
 
==Other documents==
 
==Other documents==
Line 100: Line 5:
 
* [[Why Madagascar]]   
 
* [[Why Madagascar]]   
 
* [[Installation|Installation instructions]]
 
* [[Installation|Installation instructions]]
*[http://egl.beg.utexas.edu/RSF/ Self-documentation reference for Madagascar programs]  
+
* [http://reproducibility.org/RSF/ Self-documentation reference for Madagascar programs]  
*A [[Programs|guide to Madagascar programs]]
+
* [[Guide to madagascar programs]]
*A [[Format|guide to RSF file format]]
+
* [[Guide to RSF file format]]
*A [[API|guide to Madagascar programming interface]]  
+
* [[Guide to madagascar API|Guide to the Madagascar programming interface]]
*A [[Demo|guide to programming with Madagascar]]  
+
* [[Guide to programming with madagascar]]
*A [[Tour|tour of Madagascar software]]  
+
* [[Revisiting SEP tour with Madagascar and SCons]]
*A [[SCons|guide to SCons interface for reproducible computations]]
+
* [[Reproducible computational experiments using SCons]]
 
 
==References==
 
<references/>
 

Latest revision as of 03:30, 9 January 2009