Commit Graph

117 Commits

Author SHA1 Message Date
Måns Rullgård
bcb0a1546d fate: keep stderr from failing tests
Originally committed as revision 24394 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-21 20:49:34 +00:00
Måns Rullgård
ffe72624e7 fate: run diff even if command fails
The diff may provide useful information even if the command was
unsuccessful.  The test is still treated as failed in this case.

Originally committed as revision 24353 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-20 12:54:43 +00:00
Måns Rullgård
9a3f31efcb fate: allow running regtests through fate frontend
Originally committed as revision 24347 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-20 10:19:28 +00:00
Måns Rullgård
cdaf4b79bb fate: simplify test runner slightly
All tests use the provided helper functions so prepending $target_exec
and using eval is no longer required.

Originally committed as revision 24317 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-18 20:46:44 +00:00
Måns Rullgård
2b18c451bd fate: add some helper functions to simplify test rules
Originally committed as revision 24314 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-18 20:19:30 +00:00
Måns Rullgård
f0388cf716 fate: apply TARGET_EXEC only to commands starting with absolute path
Originally committed as revision 24313 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-18 20:19:25 +00:00
Måns Rullgård
d2f33c8021 fate: use our variable names in test rules imported from Mike's db
Originally committed as revision 24312 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-18 20:19:21 +00:00
Måns Rullgård
122a9af76b fate-run: rename some variables consistently with other files
Originally committed as revision 24311 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-18 20:19:16 +00:00
Måns Rullgård
e153cfa867 fate: add stddev comparator
This allows CMP=stddev in test rules.  The test passes if the reported
stddev is <= the FUZZ value (default 1).

Originally committed as revision 24289 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-17 15:02:34 +00:00
Måns Rullgård
26e87a2261 fate: whitespace cosmetics
Originally committed as revision 24288 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-17 15:02:32 +00:00
Måns Rullgård
83b203dc94 fate: ensure file sizes match in oneoff tests
Originally committed as revision 24164 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-10 16:58:00 +00:00
Måns Rullgård
150ef830f5 fate: fail with a sensible error message if reference file is missing
Originally committed as revision 24152 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-09 23:55:16 +00:00
Måns Rullgård
7d610af6f2 fate: delete output files from successful tests
Originally committed as revision 24150 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-09 23:36:04 +00:00
Måns Rullgård
58d2b15447 fate: exit with error immediately if test command fails
Originally committed as revision 24147 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-09 23:26:33 +00:00
Måns Rullgård
aab3ea5f7d fate: add oneoff comparison method
This adds the 'oneoff' comparison method.  It compares two s16le pcm
files with a max difference <= the fuzz factor treated as success.
Default fuzz is 1.  On failure, the full tiny_psnr output is printed.

Originally committed as revision 24134 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-09 00:40:51 +00:00
Måns Rullgård
3ade7d32c1 fate: pass additional variables to test script
This passes the following additional variables to the test script:

- CMP  method to use comparing output with reference, default 'diff'
- REF  reference file, default 'tests/ref/fate/${test}'
- FUZZ maximum fuzz factor in comparison, meaning depends on CMP

Originally committed as revision 24133 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-07-09 00:40:45 +00:00
Måns Rullgård
2fad097788 Add FATE tests
This adds a "fate" make target which runs the full FATE test suite.
Individual tests can be run with "make fate-$testname".

The location of the FATE test samples must be specified with the
--samples=PATH option to configure.

The tests/fate-update.sh script regenerates the references files and
test list from the online FATE database.  These are checked in since
generating them requires non-standard tools.

Originally committed as revision 22552 to svn://svn.ffmpeg.org/ffmpeg/trunk
2010-03-15 19:23:24 +00:00