A new test target named 'modules_integrationtests' is created
and the following test targets were merged into it:
* audio_coding_module_test
* test_fec
* video_coding_integrationtests
* vp8_integrationtests
A couple of other targets were merged into modules_unittests:
* audio_coding_unittests
* audioproc_unittest
* common_unittests
* video_coding_unittests
* video_processing_unittests
* vp8_unittests
I wasn't able to merge audio_decoder_unittests and neteq_unittests due to
conflicts with different defines in these tests.
Some tests that have special requirements aren't merged into
modules_integrationtests yet. I took the opportunity to rename them
since the bot configs will need to be update anyway:
* audio_device_test_api -> audio_device_integrationtests
* video_capture_module_test -> video_capture_integrationtests
* video_render_module_test -> video_render_integrationtests
Exclude files were added for modules_integrationtests to make sure
the memcheck and tsan bots doesn't tests that are too slow
(audio_coding_module_test and vp8_integrationtests were previously
disabled on those bots).
Suppressions for AudioCodingModuleTest needed to be added to get
modules_integrationtests to pass memcheck (even if the test is
excluded from execution).
BUG=1843
TEST=local execution on Linux and trybots (passing except the merged tests of course)
R=andrew@webrtc.org, tina.legrand@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/1656004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4228 4adac7df-926f-26a2-2b94-8c16560cd09d
This CL will lower the number of test targets in WebRTC by:
Add common_audio_unittests and merge the following targets into it (copied from http://review.webrtc.org/1584006):
* resampler_unittests
* signal_processing_unittests
* vad_unittests
Merge into modules_unittests:
* bitrate_controller_unittests
* desktop_capture_unittests
* media_file_unittests
* remote_bitrate_estimator_unittests
* rtp_rtcp_unittests
* paced_sender_unittests
Merge into test_support_unittests:
* channel_transport_unittests
channel_transport.gyp was also removed in favor for test.gyp.
I had to remove a main method from rtcp_format_remb_unittest.cc
since it caused the fileutils.h code to not be able to find the
right project root path in ordrer to provide correct paths
to test files.
Buildbot configuration update will be synced with the commit of this CL.
TEST=trybots
BUG=1843
R=andrew@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/1639004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4213 4adac7df-926f-26a2-2b94-8c16560cd09d
By using the C++ version of Zxing, we can avoid having Java and Ant
as a dependency when running Video quality analysis on the bots.
This makes it far more easy to setup automation on new machines.
I also moved the scripts into the webrtc/ folder so it will be synced by default when building in Chrome (eliminating the need of a separate solution).
This CL also removes the need of the FFMPEG_HOME variable and replaces
its use with a command line flag to make the tool run smoothly on
Windows.
BUG=none
TEST=locally running the script on Windows, Mac and Linux.
Review URL: https://webrtc-codereview.appspot.com/1099007
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3640 4adac7df-926f-26a2-2b94-8c16560cd09d
- Added pipe rule flush handling
- Also fixed a bug preventing any rule settings other than default from being
used no matter what preset was chosen
- Fixed some comments.
BUGS=none
TEST= Windows and linux
Review URL: https://webrtc-codereview.appspot.com/1158006
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3639 4adac7df-926f-26a2-2b94-8c16560cd09d
Due to a bug in the RTP module, which appeared during packet loss, we have had too short delay in the Win Large Test. When the bug was fixed we had a regression error that should be fixed with this update.
Review URL: https://webrtc-codereview.appspot.com/1091005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3466 4adac7df-926f-26a2-2b94-8c16560cd09d
The change in r3354 caused the --tool argument to not be parsed if it's passed after the test executable. Then it's considered an argument to the test rather than a script flag.
This CL cleans the code a bit and makes it possible to pass all the supported argument in the different ways possible.
NOTICE: To pass arguments to the test executable, you must use the -- argument must be specified before the test arguments start, to signal that everything that comes after it are positional arguments only (which are passed on to the test during execution).
BUG=none
TEST=The following combinations have been tested:
tools/valgrind-webrtc/webrtc_tests.sh -b out/Debug -t test_support_unittests --tool asan
tools/valgrind-webrtc/webrtc_tests.sh -b out/Debug -t test_support_unittests --tool asan -- --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh --tool asan -b out/Debug -t test_support_unittests
tools/valgrind-webrtc/webrtc_tests.sh --tool asan -b out/Debug -t test_support_unittests -- --foo --bar
Review URL: https://webrtc-codereview.appspot.com/1026005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3355 4adac7df-926f-26a2-2b94-8c16560cd09d
* Clarified that the test only supports specifying a single test (multiple didn't work before, so better not claim to support it).
* No longer needs test executable arguments to use ++ instead of --
* Only appends the build_dir path to the test executable if not already
present.
* Simplified suppression path handling.
* Fixed crash when -v was used (import logging was missing)
* Style fixes.
* Thorougly tested with all the supported flags.
I noted that the --gtest_filter flag does not work as expected (it's
only for 'additional gtest_filter arguments', which seems to mean
additional arguments to the gtest filter text files that are used by
Chrome. I left it in here anyway. If --gtest_filter is given after the
test executable it will work, since those arguments are added straight
to the test executable
the test
BUG=none
TEST=I ran the following commands and verified that the suppressions and flags were handled correct:
tools/valgrind-webrtc/webrtc_tests.sh -v --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -t out/Debug/test_support_unittests --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh -v --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -b out/Debug -t test_support_unittests --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh -v --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -b out/Debug -t out/Debug/test_support_unittests --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh -v --tool=tsan --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -t out/Debug/test_support_unittests --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh -v --tool=tsan --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -b out/Debug -t test_support_unittests --foo --bar
tools/valgrind-webrtc/webrtc_tests.sh -v --tool=tsan --gtest_repeat=2 --keep_logs --tool_flags=--trace-children=yes -b out/Debug -t out/Debug/test_support_unittests --foo --bar
Review URL: https://webrtc-codereview.appspot.com/1029005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3354 4adac7df-926f-26a2-2b94-8c16560cd09d
* Improved sort checker so we sort includes more.
* Fixed vars in brackets and varsLikeTHIS.
* Added automatic x++ to ++x conversion in for loops.
TEST=Ran on various source files and verified manually.
Review URL: https://webrtc-codereview.appspot.com/1017004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3325 4adac7df-926f-26a2-2b94-8c16560cd09d
This makes it possible to compile on the bots without hardcoding paths
to Ant, Java and ffmpeg deep into the Python scripts (hardcoded paths exists only in the buildbot configuration).
For bots, the ANT_HOME, JAVA_HOME and FFMPEG_HOME environment variables must be set to the install locations for each of these dependencies, for Windows.
This CL also improves the return code handling to make failures easier to detect when things break.
TEST=running build_zxing.py without Ant or Java in the PATH, but with
ANT_HOME, JAVA_HOME and FFMPEG_HOME set. Running Chromium's src/chrome/test/functional/webrtc_video_quality.py.
BUG=none
Review URL: https://webrtc-codereview.appspot.com/1002005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3311 4adac7df-926f-26a2-2b94-8c16560cd09d
Turns out the bot is running a different version of libpthread, probably because I'm on gPrecise / gcc 4.6 whereas the bot is on 4.4? Anyway, I've generalized that stuff now.
BUG=
TEST=Ran voe & vie_auto_test under valgrind locally.
Review URL: https://webrtc-codereview.appspot.com/908004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2973 4adac7df-926f-26a2-2b94-8c16560cd09d
These changes make the execution abort earlier on an error (like a tool is not found) and makes it easier to figure out what's wrong.
Made build_zxing.py executable.
BUG=None
TEST=Local runs of the PyAuto test src/chrome/test/functional/webrtc_video_quality.py in Chromium.
Review URL: https://webrtc-codereview.appspot.com/840005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2899 4adac7df-926f-26a2-2b94-8c16560cd09d
Also Suppressed FakeMediaProcess errors (bug 898) and took out a test and suppressed general errors (bug 332). Lastly, fixed memory leak in misc test.
BUG=898, 332
TEST=Ran voe_auto_test with repeat=10 through the whole standard suite, under valgrind. Ran without valgrind. Tested that the extended and standard tests still start and are reachable from the menu.
Review URL: https://webrtc-codereview.appspot.com/855009
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2898 4adac7df-926f-26a2-2b94-8c16560cd09d
Ensured there are bugs for all open valgrind issues in the standard tests and suppressed the known issues. This way, we can get it running in continuous integration and keep new issues from entering.
Removed bad check in codec test, added suppressions.
Fixed simple memory leaks in tests.
BUG=Related to bug 329
TEST=Ran the vie_auto_test standard suite many times under valgrind to root out flakiness. Ran the standard suite without valgrind to ensure I didn't break anything.
Review URL: https://webrtc-codereview.appspot.com/843005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2854 4adac7df-926f-26a2-2b94-8c16560cd09d
Added new wrapper script webrtc_tests.bat for executing memory/threading tests on Windows.
Updated webrtc_tests.sh to include modifications in chrome_tests.sh that has happened since we copied it.
To setup TSAN for Windows, see http://www.chromium.org/developers/how-tos/using-valgrind/threadsanitizer
I did like this:
1. Added "third_party/tsan": "http://src.chromium.org/chrome/trunk/deps/third_party/tsan"
to custom_deps in my .gclient file
2. gclient sync
3. SET GYP_DEFINES=build_with_tool=tsan && gclient runhooks
4. Compiled.
5. Ran the test using the wrapper script (see below).
To setup Dr Memory for Windows, see http://www.chromium.org/developers/how-tos/using-drmemory
I did like this:
1. Added "third_party/drmemory": "http://src.chromium.org/svn/trunk/deps/third_party/drmemory",
to custom_deps in my .gclient file (using the drmemory.DEPS as described on Chromium's wiki ends up in the wrong location)
2. gclient sync
3. SET GYP_DEFINES=build_with_tool=drmemory && gclient runhooks
4. Compiled.
5. Ran the test using the wrapper script (see below).
TEST=
On Windows:
tools\valgrind-webrtc\webrtc_tests.bat --tool=tsan --test build\Debug\voice_engine_unittests.exe
tools\valgrind-webrtc\webrtc_tests.bat --tool=drmemory_light --test build\Debug\voice_engine_unittests.exe
tools\valgrind-webrtc\webrtc_tests.bat --tool=drmemory_full --test build\Debug\voice_engine_unittests.exe
On Linux:
tools/valgrind-webrtc/webrtc_tests.sh --tool=memcheck --test out/Release/test_support_unittests
tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan --test out/Release/test_support_unittests
tools/valgrind-webrtc/webrtc_tests.sh --tool=asan --test out/Release/test_support_unittests
Review URL: https://webrtc-codereview.appspot.com/845004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2846 4adac7df-926f-26a2-2b94-8c16560cd09d
This is also needed to make it possible to run unit tests easily using Chromium's buildbot source code.
BUG=None
TEST=tools/valgrind-webrtc/webrtc_tests.sh --test test_support_unittests --build_dir=out/Debug
Review URL: https://webrtc-codereview.appspot.com/784007
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2738 4adac7df-926f-26a2-2b94-8c16560cd09d
Since Chromium has moved to this policy, we should too.
Code is copied from /depot_tools/presubmit_canned_checks.py but modified for our purpose.
BUG=
TEST=Tested git cl presubmit with a modified .cc file with the 2011 header and one with the 2012.
Review URL: https://webrtc-codereview.appspot.com/770005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2691 4adac7df-926f-26a2-2b94-8c16560cd09d
The barcode encoder generates barcodes as PNG files, than converts them to YUV files,
then stitches these into one YUV video file and finaly overlays this video over the
base video to produce a YUV video of the base with encoded barcodes in the upper part
of the video.
The decoder gets a YUV video with overlaid barcodes and first splits it into PNG files,
than decodes every file, and finally generates a stats file.
The encoder and decoder use Zxing and its Java command-line tools for the encoding and
decoding. They also use ffmpeg for the conversion between PNG and YUV and vice versa.
BUG=
TEST=
The tools could be used from trunk as:
./tools/barcode_tools/barcode_encoder.py --barcode_height=32 --base_frame_width=352
--base_frame_height=288 --base_yuv=<path_nad_name_to_base_file>
--output_yuv=<path_nad_name_to_output_file>
./tools/barcode_tools/barcode_decoder.py --yuv_file=<path_and_name_to_yuv_file>
--yuv_frame_width=352 --yuv=frame_height=288 --barcode_height=32
--stats_file=<path_and_name_of_stats_file>
By default the width of the barcodes is the same as the width of the base YUV frames.
This could be changes with the command-line option --barcode_width
Review URL: https://webrtc-codereview.appspot.com/679010
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2612 4adac7df-926f-26a2-2b94-8c16560cd09d
This fix is needed for our own build slaves to work properly on Windows and is caused by the hacky way we created the Libvpx waterfall to avoid duplicating unnecessary Python code.
TBR=phoglund
BUG=None
TEST=Tested on Windows build slave.
Review URL: https://webrtc-codereview.appspot.com/639009
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2387 4adac7df-926f-26a2-2b94-8c16560cd09d
The test is now named 'bitrate_controller_unittests'.
This CL also enables it on the bots. The test is excluded on ASAN since
it fails when compiled with projects generated with GYP_DEFINES='asan=1' (see issue 555).
BUG=None
TEST=bitrate_controller_unittests was tested in Debug+Release on Linux, Mac and Windows + TSAN and memcheck.
Review URL: https://webrtc-codereview.appspot.com/612004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2333 4adac7df-926f-26a2-2b94-8c16560cd09d
Chrome bots are now using the correct scheduler.
Created separate watchlist e-mail for libvpx builds to avoid spamming WebRTC when these builds fail.
BUG=None
TEST=Tested on local master and slaves.
Review URL: https://webrtc-codereview.appspot.com/620004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2331 4adac7df-926f-26a2-2b94-8c16560cd09d
I think I found a decent way to avoid duplicating the whole configuration for the libvpx waterfall. It's not perfect but it works. I still haven't figured out what the best way to manage the slaves would be however, since they will need to know which master to connect to, or they'll pick the default they're configured for in slaves.cfg, which is the WebRTC waterfall (can be overridden with the TESTING_MASTER property, but that's only to be used for development and testing.
BUG=None
TEST=Tested on local master and slaves.
Review URL: https://webrtc-codereview.appspot.com/595005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2310 4adac7df-926f-26a2-2b94-8c16560cd09d
Skeleton suppression files for future WebRTC suppressions are added and are included in addition to the ones Chrome are using and maintaining when our wrapper script executes.
Also added tweaked PRESUBMIT checks based on the Chrome code, that verifies
that suppressions are added correctly. I tested that they work by adding an invalid
suppression.
BUG=544
TEST=Tested running tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t out/Debug/system_wrappers_unittests and it reports far less errors. Tested adding bad suppression and it was caught by the PRESUBMIT check.
Review URL: https://webrtc-codereview.appspot.com/601004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2304 4adac7df-926f-26a2-2b94-8c16560cd09d
The old scheduler was missing the branch configuration, that was started being used as a consequence of adapting the Chromium Buildbot scripts.
TBR=phoglund
BUG=None
TEST=Tested on live master+slaves with manual edits.
Review URL: https://webrtc-codereview.appspot.com/576012
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2250 4adac7df-926f-26a2-2b94-8c16560cd09d
Also removed the old Linux Chrome and Chrome Bloat bots from utils.py.
Some of these changes may seem a bit confusing and messy but they must be considered a step on the way to moving away from our inheritance based Factories into the style Chrome users, where most configuration is in the .cfg files, factory setup is in the factories and step details are put into the commands-files.
BUG=None
TEST=Tested with local master and production slaves.
Review URL: https://webrtc-codereview.appspot.com/581006
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2238 4adac7df-926f-26a2-2b94-8c16560cd09d
The change in r2143 changed so we're reusing Chromium scripts for setting up the web status. However Chrome doesn't use the tgrid page it seems (they put the Console page on that URL instead) but we need it for our LKGR parsing.
BUG=None
TEST=Tested on local master.
Review URL: https://webrtc-codereview.appspot.com/541004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2146 4adac7df-926f-26a2-2b94-8c16560cd09d
The fastbuild flag means debug symbols are not generated. This has no
impact on our tests as long as we don't need to debug with the built
binaries, i.e. is perfect for the buildbots. Compilation is 30% faster
or more.
BUG=None
TEST=Tested on local master and linux, mac and win slaves.
Review URL: https://webrtc-codereview.appspot.com/499001
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2062 4adac7df-926f-26a2-2b94-8c16560cd09d
Added back the gviz_api to the tools deps - will remove it from the main project DEPS in a different patch. Rewired those symlinks too.
Made the build status loader algorithm more scalable. It read in the whole database on each load which is probably unsustainable in the long run. Also, it will now forget bots that have offline for more than ~5 revisions.
Fixed a bug in the coverage tracker.
BUG=
TEST=
Review URL: https://webrtc-codereview.appspot.com/486002
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2039 4adac7df-926f-26a2-2b94-8c16560cd09d
Hijacking the gyp inside_chromium_build finally came back to bite us.
Chrome started using it to control the path to the gold linker, which
broke our build. This change removes use of inside_chromium_build from
WebRTC.
If peerconnection rolls past this point, libjingle.gyp will need to be updated.
BUG=
TEST=build on Linux, Mac, Win
Review URL: https://webrtc-codereview.appspot.com/493007
git-svn-id: http://webrtc.googlecode.com/svn/trunk@2038 4adac7df-926f-26a2-2b94-8c16560cd09d
Enabling webrtc_utility_unittests on Windows (was never
added).
Enabling audio_device_test_api on Mac and Windows, since updated in r1861.
BUG=None
TEST=webrtc_utility_unittests in Debug+Release on Windows. audio_device_test_api in Debug+Release on Linux, Mac and Windows.
Review URL: https://webrtc-codereview.appspot.com/448001
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1862 4adac7df-926f-26a2-2b94-8c16560cd09d
I also took the chance to refactor the code from a long main method into a couple of functions instead.
Download progress also improved a bit.
Added a new flag (--base_url) to make it possible to use another Base URL instead of the default one.
BUG=None
TEST=Tested on local machine with various
Review URL: https://webrtc-codereview.appspot.com/431003
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1826 4adac7df-926f-26a2-2b94-8c16560cd09d
Will now recognize warnings as a status. Returns proper HTTP status codes for the most common errors now. Will be more strict when checking build status data (no newlines in bot names).
Prepared cron scripts.
Prepared dashboard for production use.
BUG=
TEST=
Review URL: https://webrtc-codereview.appspot.com/404003
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1772 4adac7df-926f-26a2-2b94-8c16560cd09d
Requires Coverity Static Analysis to be installed on the machine running the script.
The password is kept in a local file that is read by the script.
Not tested on Windows but Chromium seem to run their script on Windows. We'll use Linux for the periodic execution.
It has been running without issues for a month now.
BUG=
TEST=Tested successfully on my local Linux workstation with tools/coverity/coverity.py --source-dir=$HOME/code/webrtc/trunk --coverity-bin-dir=$HOME/coverity-integrity-center/static-analysis/bin --coverity-intermediate-dir=$HOME/cov-temp --coverity-db-host=webrtc-coverity.eem.corp.google.com --coverity-target=Linux --coverity-password-file=$HOME/coverity-password-file
Review URL: https://webrtc-codereview.appspot.com/345001
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1760 4adac7df-926f-26a2-2b94-8c16560cd09d
So now it will do like this, depending on build type:
* xcode: build all with xcode projects and run the xcode built tests.
* make: build all with makefile projects and run the makefile built tests
* both: build all with xcode projects, then build all with makefile projects, then run the xcode built tests only (not the makefile built ones)
BUG=None
TEST=Run on local master.
Review URL: https://webrtc-codereview.appspot.com/409001
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1758 4adac7df-926f-26a2-2b94-8c16560cd09d
- Sync Chrome's DEPS to chromium_deps/, and use the From() keyword
to pull the correct revisions from it, rather than having to manually
enter them.
- This idea is borrowed from the WebKit DEPS:
http://trac.webkit.org/browser/trunk/Source/WebKit/chromium/DEPS
- Fix update.py's DEPS parsing to handle From().
- Roll Chrome 120526:122775.
- Organize the deps alphabetically.
- Sync the in-tree gold linker, which is now required due to a change
in the linker flags.
- Add the new deps to .gitignore.
BUG=
TEST=build on all platforms
Review URL: https://webrtc-codereview.appspot.com/401004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1755 4adac7df-926f-26a2-2b94-8c16560cd09d
This is done by adding a whitespace before the last underscore on
strings above length 25, which should only affect the _unittest and
_integrationtest suffixes for our tests, that are the widest unwrappable
strings in our waterfall display.
I also removed all reduntant use of ShellCommand in favour for the
AddCommonStep function (that wraps the description text).
Also fixed so Mac tests are not halting the build.
Cleaned up some code style formatting.
BUG=
TEST=
Review URL: https://webrtc-codereview.appspot.com/400005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1735 4adac7df-926f-26a2-2b94-8c16560cd09d
