As the number of pages on the Wiki grows, the navbar starts becoming insufficient for proper organization of all documentation. A top-down view of all materials about Madagascar is also useful for determining whether gaps in coverage exist. This page will stay in the Sandbox for a long while -- until all gaps have been filled.
Ideally the manual will only consist of either links to wiki pages, or own content. "Forking", i.e. creating a modified copy of a page especially for the manual, invariably ends up with one version getting out of synch.
How nice would it be, and what publicity would bring, calling this manual "Reproducible data analysis with Madagascar" and getting it on SEG Bookmart's shelf :-)
- 1 About Madagascar
- 2 Downloading and installing Madagascar
- 3 Using Madagascar
- 3.1 The lightning-quick tour
- 3.2 The Madagascar file formats
- 3.3 Calling existing Madagascar programs
- 3.4 What is reproducibility
- 3.5 Exploring existing reproducible papers
- 3.6 Writing a LaTeX paper in the Madagascar framework
- 3.7 Creating a reproducible paper
- 3.8 Creating a reproducible book
- 4 Developing in Madagascar
- 4.1 Writing your own programs
- 4.2 Adding programs to the central repository
- 4.3 Framework development and maintenance
- 5 Datasets distributed with Madagascar
- 6 Other open-source data analysis packages
For people who do not read manuals
Why use Madagascar?
An articulate description of the reasons on the Why Madagascar page. Have some spectacular pictures obtained with algorithms that are not present in other packages. Describe algorithms/tools unavailable in other open-source geophysical data analysis packages.
A description of the current Madagascar community, with the map of downloads and an estimate of the number of installs, who are the biggest users, outstanding research results obtained with Madagascar, etc. Links to the blog, user mailing list, developer mailing list. Also mention the bug tracker and feature request tracker, encouraging the community to use them more. Mention forums as an alternative for those who want to ask questions or conduct discussions without subscribing to a mailing list.
A history of Madagascar, with the SEPlib/SU part of the "Alternatives" section of the Introduction, and mentions of landmark events (short descriptions where necessary):
- when and where it was started
- 2004-08 (?): made available to selected alpha users
- 2005-02-16: RSF Blog started
- 2006-03-17: Registered on Sourceforge
- 2006-04-19: Name change from RSF to Madagascar
- 2006-06-11: Public launch at the Open Source E&P Software EAGE Workshop (Vienna)
- 2006-06-18: First stable version (0.9.1). Mailing lists created
- 2006-07-23: Madagascar logo created by Scott Rodgers (BEG)
- 2006-08-30/31: Madagascar School and Workshop (Vancouver) (How many participants, from how many universities and how many companies?)
- 2007-04-20: Madagascar School (Austin)
- 2008-05-23/26: Madagascar Implementation Workshop (Golden)
More details can be found on the Conferences page.
Downloading and installing Madagascar
The lightning-quick tour
The Madagascar file formats
The Regularly Sampled Format (RSF)
The current Guide to RSF file format
Handling irregularly sampled data
Explain the principle of the current method (sfheadermath/sfheaderwindow used on the trace header block output by su/segyread)
Calling existing Madagascar programs
Finding out what program you need
- sfdoc -k
- Task-centric program list and all its subordinate nodes
- Collection of 2-3 page reproducible papers -- "How to do raytracing in Madagascar"; "How to do modeling in Madagascar"; etc
- SU to m8r dictionary
- SEPlib to m8r dictionary
- Other such dictionaries, for free or proprietary seismic processing packages. Such dictionaries are also useful because they will highlight algorithms/utilities present in such packages but missing from m8r.
This chapter is now just a sketch, should get quite big. Users approach tools in a task-centric fashion, i.e. Q1:"how do I do X with Madagascar?", A1:"With feature Y"; Q2: "How do I use feature Y to this end?" M8r is very good at answering Q2, but people ask Q1 first. Many of the reproducible papers included so far contain cutting-edge research. Users learning how to use Madagascar need to start with something much more simple, where they do not have to focus on understanding research on top of understanding software.
Learning how to use a given program
- Command-line self-doc
- Local html self-doc ($RSFROOT/doc/index.html). Contains all programs installed on the user's machine and only those programs.
- Online self-doc
- The wiki Guide to Programs.
- Series of dedicated reproducible papers that present the theory behind specific geophysical programs and demonstrate it with various types of inputs and combination of parameters, like this paper does for SEPlib's AMO program.
- Combining together multiple programs -- the reproducible papers; pointer to relevant section of the manual ("Exploring reproducible papers")
What is reproducibility
The whole Reproducibility page, combined with Section 1 from Reproducible computational experiments using SCons
Exploring existing reproducible papers
Papers and books included in the Madagascar distribution
Reproducible Documents and more.
How to reproduce specific figures in existing papers
A frequently encountered case is when a researcher wants to reproduce only one or several figures from an entire paper, but not the entire paper. This can happen because on that system LaTeX dependencies of Madagascar are missing or not working properly, or simply because the researcher is interested only in that result.
- Finding the paper directory: If the interesting article has been found by browsing/hyperlink to Reproducible Documents, then the reproducibility package corresponding to
http://www.reproducibility.org/RSF/book/<bookname>/<papername>/paper_html/can be found in
- Finding result names: Use the html version of the paper, or grep in all .tex files in the directory for a text string that occurs in the figure legend. Multiple-panel figures may have individual names for each panel. [Note: In pdf versions obtained with scons pdf in paper directory, neither the book name nor paper directory name nor figure names are given. LaTeX options to have figure names as well as a Geophysics-style header/footer with more details on the first page may be in order]
- Finding where to launch the re-build: In some cases, rules for creating a result are specified in SConstruct files in subdirectories of the main paper directory. If step 4 fails in the main paper directory, then you will have to find where the figure is built. Because result names may be generated automatically, a simple grep may not be enough and you may need to read the SConstruct and python modules imported by it to figure out if the result is generated there.
- Re-build and display the figure by typing scons resultname.view in the appropriate directory.
How to reproduce entire papers using stored figures
How to reproduce entire papers and all their figures
- See the previous section for how to find the paper directory
- The relevant SCons commands to reproduce the paper (scons pdf and scons read). Troubleshooting:
- If it fails with this kind of messages (details here), you miss TeX system dependencies. Install a TeX system. Tex Live should have it all. Note: It's a 1 Gb download. Too large for many individual users to bother with it and for most IT departments of companies to review for security. We should implement individual dependency checking, like we do in the installation. Otherwise it is equivalent to telling the user that in order to install Madagascar, he should install Debian with all 30000 packages in the repository... the dependencies will be among them.
- If scons pdf in the paper directory pdf figures already in place in order to work, run sftour scons lock.
- Tell the user to expect conditional reproducibility: If Matlab is not present, rsftex will not try to build the figures but will use the stored PDF files (same goes for Mathematica, xfig, etc.)
How to reproduce whole books
Writing a LaTeX paper in the Madagascar framework
Follows the natural progression of learning of somebody who may even not know LaTeX, let alone SCons.
- A paper with no figures.
- A paper with NR-only figures
Creating a reproducible paper
Sections 2 3 from Reproducible computational experiments using SCons. Also, mention the "SCons macros" in book/packages.
Creating a reproducible book
Developing in Madagascar
Writing your own programs
The Madagascar API
- The existing data clipping API demo
- A more complex finite differences API demo – add Python, F77 and Matlab APIs to it
How to add your program
How to document your program
How to test your program
How to parallelize your programs
Tips and tricks
Madagascar library reference
Adding programs to the central repository
Framework development and maintenance
Description of m8r's inner works for those who want to help improve and maintain Madagascar. Maintenance guide and perhaps other stuff.
Datasets distributed with Madagascar
- Description of datasets – pictures of the velocity model, of sample gathers, zero-offset sections, migrated image.
- Comment on which are the main problems they illustrate (internal multiples? overturning waves? etc). Algorithm used for generating them, references to published literature describing the datasets
- Command line options for correctly reading them from the storage format (SEG-Y, most probably) into RSF
- In general, expand the datasets section of Reproducible Documents page to include other datasets
Other open-source data analysis packages
Other open-source geophysical packages. Briefly discuss each of them. Mention "dictionaries" from them to m8r where available (should attempt to have dictionaries for all of them)
Mention other open-source codes that are commonly used by geophysicists:
- GNU Scientific Library