doc: rewrite FATE documentation to be self-contained
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This commit is contained in:
parent
e77b4b4ac6
commit
7ffa9ea05a
118
doc/fate.txt
118
doc/fate.txt
@ -1,23 +1,115 @@
|
||||
FATE Automated Testing Environment
|
||||
==================================
|
||||
|
||||
FATE provides a regression testsuite that can be run locally or configured
|
||||
to send reports to fate.ffmpeg.org.
|
||||
In order to run, it needs a large amount of data (samples and references)
|
||||
that is provided separately from the actual source distribution.
|
||||
FATE is an extended regression suite on the client-side and a means
|
||||
for results aggregation and presentation on the server-side.
|
||||
|
||||
Use the following command to get the fate test samples
|
||||
The first part of this document explains how you can use FATE from
|
||||
your FFmpeg source directory to test your ffmpeg binary. The second
|
||||
part describes how you can run FATE to submit the results to FFmpeg's
|
||||
FATE server.
|
||||
|
||||
In any way you can have a look at the publicly viewable FATE results
|
||||
by visiting this website:
|
||||
|
||||
http://fate.ffmpeg.org/
|
||||
|
||||
This is especially recommended for all people contributing source
|
||||
code to FFmpeg, as it can be seen if some test on some platform broke
|
||||
with there recent contribution. This usually happens on the platforms
|
||||
the developers could not test on.
|
||||
|
||||
The second part of this document describes how you can run FATE to
|
||||
submit your results to FFmpeg's FATE server. If you want to submit your
|
||||
results be sure to check that your combination of CPU, OS and compiler
|
||||
is not already listed on the above mentioned website.
|
||||
|
||||
In the third part you can find a comprehensive listing of FATE makefile
|
||||
targets and variables.
|
||||
|
||||
|
||||
1. Using FATE from your FFmpeg source directory
|
||||
-----------------------------------------------
|
||||
|
||||
If you want to run FATE on your machine you need to have the samples
|
||||
in place. You can get the samples via the build target fate-rsync.
|
||||
Use this command from the top-level source directory:
|
||||
|
||||
# make fate-rsync SAMPLES=fate-suite/
|
||||
# make fate SAMPLES=fate-suite/
|
||||
|
||||
To inform the build system about the testsuite location, pass
|
||||
`--samples=<path to the samples>` to configure or set the SAMPLES Make
|
||||
variable or the FATE_SAMPLES environment variable to a suitable value.
|
||||
NOTE: Do not put a '~' character in the samples path to indicate a home
|
||||
The above commands set the samples location by passing a makefile
|
||||
variable via command line. It is also possible to set the samples
|
||||
location at source configuration time by invoking configure with
|
||||
`--samples=<path to the samples directory>'. Afterwards you can
|
||||
invoke the makefile targets without setting the SAMPLES makefile
|
||||
variable. This is illustrated by the following commands:
|
||||
|
||||
# ./configure --samples=fate-suite/
|
||||
# make fate-rsync
|
||||
# make fate
|
||||
|
||||
Yet another way to tell FATE about the location of the sample
|
||||
directory is by making sure the environment variable FATE_SAMPLES
|
||||
contains the path to your samples directory. This can be achieved
|
||||
by e.g. putting that variable in your shell profile or by setting
|
||||
it in your interactive session.
|
||||
|
||||
# FATE_SAMPLES=fate-suite/ make fate
|
||||
|
||||
NOTE:
|
||||
Do not put a '~' character in the samples path to indicate a home
|
||||
directory. Because of shell nuances, this will cause FATE to fail.
|
||||
|
||||
For information on how to set up FATE to send results to the official FFmpeg
|
||||
testing framework, please refer to the following wiki page:
|
||||
http://wiki.multimedia.cx/index.php?title=FATE
|
||||
|
||||
2. Submitting the results to the FFmpeg result aggregation server
|
||||
-----------------------------------------------------------------
|
||||
|
||||
To submit your results to the server you should run fate through the
|
||||
shell script tests/fate.sh from the FFmpeg sources. This script needs
|
||||
to be invoked with a configuration file as its first argument.
|
||||
|
||||
# tests/fate.sh /path/to/fate_config
|
||||
|
||||
A configuration file template with comments describing the individual
|
||||
configuration variables can be found at tests/fate_config.sh.template .
|
||||
|
||||
Create a configuration that suits your needs, based on the configuration
|
||||
template. The `slot' configuration variable can be any string that is not
|
||||
yet used, but it is suggested that you name it adhering to the following
|
||||
pattern <arch>-<os>-<compiler>-<compiler version>. The configuration file
|
||||
itself will be sourced in a shell script, therefore all shell features may
|
||||
be used. This enables you to setup the environment as you need it for your
|
||||
build.
|
||||
|
||||
For your first test runs the `fate_recv' variable should be empty or
|
||||
commented out. This will run everything as normal except that it will omit
|
||||
the submission of the results to the server. The following files should be
|
||||
present in $workdir as specified in the configuration file:
|
||||
|
||||
- configure.log
|
||||
- compile.log
|
||||
- test.log
|
||||
- report
|
||||
- version
|
||||
|
||||
When you have everything working properly you can create an SSH key and
|
||||
send its public part to the FATE server administrator.
|
||||
|
||||
Configure your SSH client to use public key authentication with that key
|
||||
when connecting to the FATE server. Also do not forget to check the identity
|
||||
of the server and to accept its host key. This can usually be achieved by
|
||||
running your SSH client manually and killing it after you accepted the key.
|
||||
The FATE server's fingerprint is:
|
||||
|
||||
b1:31:c8:79:3f:04:1d:f8:f2:23:26:5a:fd:55:fa:92
|
||||
|
||||
The only thing left is to automate the execution of the fate.sh script and
|
||||
the synchronisation of the samples directory.
|
||||
|
||||
|
||||
3. FATE makefile targets and variables
|
||||
--------------------------------------
|
||||
|
||||
FATE Makefile targets:
|
||||
|
||||
@ -27,7 +119,7 @@ fate-list
|
||||
fate
|
||||
Run the FATE test suite (requires the fate-suite dataset).
|
||||
|
||||
Fate Makefile variables:
|
||||
FATE Makefile variables:
|
||||
|
||||
V
|
||||
Verbosity level, can be set to 0, 1 or 2.
|
||||
|
Loading…
Reference in New Issue
Block a user