A new test target named 'modules_integrationtests' is created
and the following test targets were merged into it:
* audio_coding_module_test
* test_fec
* video_coding_integrationtests
* vp8_integrationtests
A couple of other targets were merged into modules_unittests:
* audio_coding_unittests
* audioproc_unittest
* common_unittests
* video_coding_unittests
* video_processing_unittests
* vp8_unittests
I wasn't able to merge audio_decoder_unittests and neteq_unittests due to
conflicts with different defines in these tests.
Some tests that have special requirements aren't merged into
modules_integrationtests yet. I took the opportunity to rename them
since the bot configs will need to be update anyway:
* audio_device_test_api -> audio_device_integrationtests
* video_capture_module_test -> video_capture_integrationtests
* video_render_module_test -> video_render_integrationtests
Exclude files were added for modules_integrationtests to make sure
the memcheck and tsan bots doesn't tests that are too slow
(audio_coding_module_test and vp8_integrationtests were previously
disabled on those bots).
Suppressions for AudioCodingModuleTest needed to be added to get
modules_integrationtests to pass memcheck (even if the test is
excluded from execution).
BUG=1843
TEST=local execution on Linux and trybots (passing except the merged tests of course)
R=andrew@webrtc.org, tina.legrand@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/1656004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4228 4adac7df-926f-26a2-2b94-8c16560cd09d
The change in r3354 caused the --tool argument to not be parsed if it's passed after the test executable. Then it's considered an argument to the test rather than a script flag.
This CL cleans the code a bit and makes it possible to pass all the supported argument in the different ways possible.
NOTICE: To pass arguments to the test executable, you must use the -- argument must be specified before the test arguments start, to signal that everything that comes after it are positional arguments only (which are passed on to the test during execution).
BUG=none
TEST=The following combinations have been tested:
tools/valgrind-webrtc/webrtc_tests.sh -b out/Debug -t test_support_unittests --tool asan
tools/valgrind-webrtc/webrtc_tests.sh -b out/Debug -t test_support_unittests --tool asan -- --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh --tool asan -b out/Debug -t test_support_unittests
tools/valgrind-webrtc/webrtc_tests.sh --tool asan -b out/Debug -t test_support_unittests -- --foo --bar
Review URL: https://webrtc-codereview.appspot.com/1026005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3355 4adac7df-926f-26a2-2b94-8c16560cd09d
* Clarified that the test only supports specifying a single test (multiple didn't work before, so better not claim to support it).
* No longer needs test executable arguments to use ++ instead of --
* Only appends the build_dir path to the test executable if not already
present.
* Simplified suppression path handling.
* Fixed crash when -v was used (import logging was missing)
* Style fixes.
* Thorougly tested with all the supported flags.
I noted that the --gtest_filter flag does not work as expected (it's
only for 'additional gtest_filter arguments', which seems to mean
additional arguments to the gtest filter text files that are used by
Chrome. I left it in here anyway. If --gtest_filter is given after the
test executable it will work, since those arguments are added straight
to the test executable
the test
BUG=none
TEST=I ran the following commands and verified that the suppressions and flags were handled correct:
tools/valgrind-webrtc/webrtc_tests.sh -v --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -t out/Debug/test_support_unittests --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh -v --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -b out/Debug -t test_support_unittests --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh -v --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -b out/Debug -t out/Debug/test_support_unittests --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh -v --tool=tsan --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -t out/Debug/test_support_unittests --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh -v --tool=tsan --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -b out/Debug -t test_support_unittests --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh -v --tool=tsan --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -b out/Debug -t out/Debug/test_support_unittests --foo --bar
Review URL: https://webrtc-codereview.appspot.com/1029005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3354 4adac7df-926f-26a2-2b94-8c16560cd09d
Turns out the bot is running a different version of libpthread, probably because I'm on gPrecise / gcc 4.6 whereas the bot is on 4.4? Anyway, I've generalized that stuff now.
BUG=
TEST=Ran voe & vie_auto_test under valgrind locally.
Review URL: https://webrtc-codereview.appspot.com/908004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2973 4adac7df-926f-26a2-2b94-8c16560cd09d
Also Suppressed FakeMediaProcess errors (bug 898) and took out a test and suppressed general errors (bug 332). Lastly, fixed memory leak in misc test.
BUG=898, 332
TEST=Ran voe_auto_test with repeat=10 through the whole standard suite, under valgrind. Ran without valgrind. Tested that the extended and standard tests still start and are reachable from the menu.
Review URL: https://webrtc-codereview.appspot.com/855009
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2898 4adac7df-926f-26a2-2b94-8c16560cd09d
Ensured there are bugs for all open valgrind issues in the standard tests and suppressed the known issues. This way, we can get it running in continuous integration and keep new issues from entering.
Removed bad check in codec test, added suppressions.
Fixed simple memory leaks in tests.
BUG=Related to bug 329
TEST=Ran the vie_auto_test standard suite many times under valgrind to root out flakiness. Ran the standard suite without valgrind to ensure I didn't break anything.
Review URL: https://webrtc-codereview.appspot.com/843005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2854 4adac7df-926f-26a2-2b94-8c16560cd09d
Added new wrapper script webrtc_tests.bat for executing memory/threading tests on Windows.
Updated webrtc_tests.sh to include modifications in chrome_tests.sh that has happened since we copied it.
To setup TSAN for Windows, see http://www.chromium.org/developers/how-tos/using-valgrind/threadsanitizer
I did like this:
1. Added "third_party/tsan": "http://src.chromium.org/chrome/trunk/deps/third_party/tsan"
to custom_deps in my .gclient file
2. gclient sync
3. SET GYP_DEFINES=build_with_tool=tsan && gclient runhooks
4. Compiled.
5. Ran the test using the wrapper script (see below).
To setup Dr Memory for Windows, see http://www.chromium.org/developers/how-tos/using-drmemory
I did like this:
1. Added "third_party/drmemory": "http://src.chromium.org/svn/trunk/deps/third_party/drmemory",
to custom_deps in my .gclient file (using the drmemory.DEPS as described on Chromium's wiki ends up in the wrong location)
2. gclient sync
3. SET GYP_DEFINES=build_with_tool=drmemory && gclient runhooks
4. Compiled.
5. Ran the test using the wrapper script (see below).
TEST=
On Windows:
tools\valgrind-webrtc\webrtc_tests.bat --tool=tsan --test build\Debug\voice_engine_unittests.exe
tools\valgrind-webrtc\webrtc_tests.bat --tool=drmemory_light --test build\Debug\voice_engine_unittests.exe
tools\valgrind-webrtc\webrtc_tests.bat --tool=drmemory_full --test build\Debug\voice_engine_unittests.exe
On Linux:
tools/valgrind-webrtc/webrtc_tests.sh --tool=memcheck --test out/Release/test_support_unittests
tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan --test out/Release/test_support_unittests
tools/valgrind-webrtc/webrtc_tests.sh --tool=asan --test out/Release/test_support_unittests
Review URL: https://webrtc-codereview.appspot.com/845004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2846 4adac7df-926f-26a2-2b94-8c16560cd09d
This is also needed to make it possible to run unit tests easily using Chromium's buildbot source code.
BUG=None
TEST=tools/valgrind-webrtc/webrtc_tests.sh --test test_support_unittests --build_dir=out/Debug
Review URL: https://webrtc-codereview.appspot.com/784007
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2738 4adac7df-926f-26a2-2b94-8c16560cd09d
Skeleton suppression files for future WebRTC suppressions are added and are included in addition to the ones Chrome are using and maintaining when our wrapper script executes.
Also added tweaked PRESUBMIT checks based on the Chrome code, that verifies
that suppressions are added correctly. I tested that they work by adding an invalid
suppression.
BUG=544
TEST=Tested running tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t out/Debug/system_wrappers_unittests and it reports far less errors. Tested adding bad suppression and it was caught by the PRESUBMIT check.
Review URL: https://webrtc-codereview.appspot.com/601004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2304 4adac7df-926f-26a2-2b94-8c16560cd09d
To be able to re-use as much as possible from Chromium's scripts, I've created two customized scripts for running tests:
- webrtc_tests.py: a customized version of chrome_tests.py with WebRTC tests instead.
- webrtc_tests.sh: a customized version of the chrome_tests.sh wrapper script, to launch the above script.
The webrtc_tests.sh script is setting up PYTHONPATH so that tools/valgrind is available for the webrtc_tests.py script.
The webrtc_tests.py script inherits the chrome_tests.py script as much as possible, to minimize maintenance and maximize readability.
Having this mirrored setup of directories, allows us to use the same directory hierarchy for suppression files too. This CL only adds suppression files for memcheck, but we can add files for tsan later easily.
The suppression file is currently empty for Linux. For Mac I copied all the Chromium third-party suppressions. We will need a lot more added for Mac before Valgrind runs becomes usable for that platform.
The platform specific naming of the suppression files are handled automatically when webrtc_tests.sh is used to launch the tests.
Example:
Plain memcheck (default tool): tools/valgrind-webrtc/webrtc_tests.sh -t system_wrappers
Run ThreadSanitizer: tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t system_wrappers
Previously mentioned AddressSanitizer requires additional scripts to be added and it not covered in this CL.
BUG=
TEST=
Review URL: http://webrtc-codereview.appspot.com/322010
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1240 4adac7df-926f-26a2-2b94-8c16560cd09d
