Difference between revisions of "Automatic Testing"

From Madagascar
Jump to navigation Jump to search
Line 5: Line 5:
 
<bash>admin/quick_test.csh</bash>
 
<bash>admin/quick_test.csh</bash>
  
This script currently (June 19, 2010) runs a set of 147 examples make 4GB of total data.  They run in about 2 minutes total on a 2.4 GHz Intel Core 2 Duo MacBook when nothing needs to be updated, and 15 minutes when everything is rebuilt from scratch.  Your mileage will vary.
+
This script currently (June 19, 2010) runs a set of 147 examples in <tt>$RSFSRC/book</tt> that make 4GB of total data.  They run in about 2 minutes total on a 2.4 GHz Intel Core 2 Duo MacBook when nothing needs to be updated, and 15 minutes when everything is rebuilt from scratch.  Your mileage will vary.
  
 
Seven of the 147 examples do not pass all the tests on my system, perhaps because of bugs not yet found :-(  These examples are:
 
Seven of the 147 examples do not pass all the tests on my system, perhaps because of bugs not yet found :-(  These examples are:

Revision as of 10:57, 19 June 2010

Quick Test

To run a "quick" test run this command from $RSFSRC:

<bash>admin/quick_test.csh</bash>

This script currently (June 19, 2010) runs a set of 147 examples in $RSFSRC/book that make 4GB of total data. They run in about 2 minutes total on a 2.4 GHz Intel Core 2 Duo MacBook when nothing needs to be updated, and 15 minutes when everything is rebuilt from scratch. Your mileage will vary.

Seven of the 147 examples do not pass all the tests on my system, perhaps because of bugs not yet found :-( These examples are:

  1. book/gee/mda/levint
  2. book/jsg/lpf/lpf
  3. book/sep/bspl/plane3
  4. book/sep/pwd/alias
  5. book/sep/pwd/hole
  6. book/sep/pwd/signoi
  7. book/sep/stack/miginv

Your mileage will probably vary on this as well. Indeed, I will be curious to see how the tests work on your system (hint, hint :-) Post your testing report on rsf-devel.

Testing Scripts

The following commands use sfbooklist and sffiglist. Run them from $RSFSRC.

  1. List all the examples in book that use <program>
    sfbooklist uses=<program> list=filter book
  2. rebuild all the examples in book that use <program>
    sfbooklist uses=<program> list=filter command=scons book
  3. Compare the newly generated figs with the corresponding reference figs stored in $RSFFIGS (defaulted to $RSFROOT/share/figs)
    sfbooklist uses=<program> list=filter command="sffiglist rsftest=y list=none" book
  4. Display any figs that are different from their $RSFFIGS counterparts
    sfbooklist uses=<program> list=filter command="sffiglist rsftest=y list=none show=diff" book