This is a re-land attempt of http://review.webrtc.org/1673004/
It now includes a build/isolate.gypi in WebRTC that includes the same
file as the one that would be included when WebRTC is used in a Chromium
checkout. It is needed since it is not possible to use variables in GYP's
includes sections.
Implemented according to the instructions at
http://www.chromium.org/developers/testing/isolated-testing
Workflow has been like this:
1. create _run GYP target
2. create a stripped down .isolate file
3. export GYP_DEFINES="$GYP_DEFINES test_isolation_mode=check"
4. runhooks
5. compile
6. test if the test would run (i.e. find it's dependencies) without
actually executing it:
tools/swarm_client/isolate.py run --isolated out/Release/testname.isolated
7. If failing, run the fix_test_cases.py script like this:
tools/swarm_client/googletest/fix_test_cases.py --isolated out/Release/testname.isolated
All tests that run on the bots for WebRTC has got _run target
and .isolate file created.
"Normal tests" that run fine on any machine:
* audio_decoder_unittests
* common_audio_unittests
* common_video_unittests
* metrics_unittests
* modules_tests
* modules_unittests
* neteq_unittests
* system_wrappers_unittests
* test_support_unittests
* tools_unittests
* video_engine_core_unittests
* voice_engine_unittests
Tests that requires bare-metal and audio/video devices:
* audio_device_tests
* video_capture_tests
I also added the isolate boilerplate code for the following
tests that are not yet pure gtest binaries (which means they
cannot run isolated yet):
* video_render_tests
* vie_auto_test
* voe_auto_test
TEST=running isolate.py as described above. WebRTC trybots passing. Created a Chromium checkout with third_party/webrtc ToT and this patch applied, passing the runhooks step.
BUG=1916
R=henrike@webrtc.org, tommi@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/2056004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4590 4adac7df-926f-26a2-2b94-8c16560cd09d
As this breaks the FYI bots in
http://build.chromium.org/p/chromium.webrtc.fyi/waterfall
due to different path to isolate.gypi (which cannot easily
be resolved due to limitations in GYP)
> Isolate GYP target and .isolate files for tests
>
> Implemented according to the instructions at
> http://www.chromium.org/developers/testing/isolated-testing
>
> Workflow has been like this:
> 1. create _run GYP target
> 2. create a stripped down .isolate file
> 3. export GYP_DEFINES="$GYP_DEFINES test_isolation_mode=check"
> 4. runhooks
> 5. compile
> 6. test if the test would run (i.e. find it's dependencies) without
> actually executing it:
> tools/swarm_client/isolate.py run --isolated out/Release/testname.isolated
> 7. If failing, run the fix_test_cases.py script like this:
> tools/swarm_client/fix_test_cases.py --isolated out/Release/testname.isolated
>
> All tests that run on the bots for WebRTC has got _run target
> and .isolate file created.
>
> "Normal tests" that run fine on any machine:
> * audio_decoder_unittests
> * common_audio_unittests
> * common_video_unittests
> * metrics_unittests
> * modules_integrationtests
> * modules_unittests
> * neteq_unittests
> * system_wrappers_unittests
> * test_support_unittests
> * tools_unittests
> * video_engine_core_unittests
> * voice_engine_unittests
>
> Tests that requires bare-metal and audio/video devices:
> * audio_device_integrationtests
> * video_capture_integrationtests
>
> I also added the isolate boilerplate code for the following
> tests that are not yet pure gtest binaries (which means they
> cannot run isolated yet):
> * video_render_integrationtests
> * vie_auto_test
> * voe_auto_test
>
> TEST=running isolate.py as described above.
> BUG=1916
> R=tommi@webrtc.org
>
> Review URL: https://webrtc-codereview.appspot.com/1673004TBR=kjellander@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/2040004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4548 4adac7df-926f-26a2-2b94-8c16560cd09d
Implemented according to the instructions at
http://www.chromium.org/developers/testing/isolated-testing
Workflow has been like this:
1. create _run GYP target
2. create a stripped down .isolate file
3. export GYP_DEFINES="$GYP_DEFINES test_isolation_mode=check"
4. runhooks
5. compile
6. test if the test would run (i.e. find it's dependencies) without
actually executing it:
tools/swarm_client/isolate.py run --isolated out/Release/testname.isolated
7. If failing, run the fix_test_cases.py script like this:
tools/swarm_client/fix_test_cases.py --isolated out/Release/testname.isolated
All tests that run on the bots for WebRTC has got _run target
and .isolate file created.
"Normal tests" that run fine on any machine:
* audio_decoder_unittests
* common_audio_unittests
* common_video_unittests
* metrics_unittests
* modules_integrationtests
* modules_unittests
* neteq_unittests
* system_wrappers_unittests
* test_support_unittests
* tools_unittests
* video_engine_core_unittests
* voice_engine_unittests
Tests that requires bare-metal and audio/video devices:
* audio_device_integrationtests
* video_capture_integrationtests
I also added the isolate boilerplate code for the following
tests that are not yet pure gtest binaries (which means they
cannot run isolated yet):
* video_render_integrationtests
* vie_auto_test
* voe_auto_test
TEST=running isolate.py as described above.
BUG=1916
R=tommi@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/1673004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4547 4adac7df-926f-26a2-2b94-8c16560cd09d
Things included in this CL:
- updated READMEs to provide an exact/reproable set of steps for getting the app
running.
- gyp changes to build the iOS AppRTCDemo sample app using gyp+ninja instead of
the hand-crafted Xcode project (which has never worked in its checked-in
form), including a gyp action to sign the sample app for deployment to an iOS
device (the app can also be used in the simulator)
- deleted the busted hand-crafted Xcode project for the sample app
- updated the sample app to match the PeerConnection API that ended up landing
(in a surprising twist of fate, the API landed quite a bit later than the
sample app and this is the first time the CR-time changes in the API are
reflected in the sample app)
- updated the sample app to reflect apprtc.appspot.com HTML/JS changes (equiv to
the AppRTCClient.java changes in http://s10/47299162)
- picked up the iossim DEPS to enable launching the sample app in the simulator
from the command-line.
- renamed some files to match capitalization of the classes they contain (Ice ->
ICE) per ObjC naming guidelines.
- ran the files involved in this CL through clang-format to deal with xcode
formatting craxy.
BUG=2106
RISK=P2
TESTED=unittest builds with ninja and passes on OS=mac; sample app builds with ninja and runs on simulator and device, though no audio flows from simulator/device (will fix in a follow-up CL)
R=andrew@webrtc.org, justincohen@google.com, wu@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/1874005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4466 4adac7df-926f-26a2-2b94-8c16560cd09d
Until we have WebRTC test targets created for Android, this test
makes it possible to move forward for buildbot automation.
TEST=Android NDK buildbot and local execution of:
source build/android/envsetup.sh
gclient runhooks
ninjar -C out/Debug
verified the out/Debug/simple_apk dir exists and has the files.
BUG=1882
R=henrika@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/1688005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4245 4adac7df-926f-26a2-2b94-8c16560cd09d
The md5sum and forwarder2 binaries from Chromium's
src/tools/android are needed to be able to run tests using the
test framework launched by build/android/run_tests.py.
Since they depend on Chromium's base, we're using a precompiled
copy for WebRTC's purposes.
Linux works out of the box if Chromium's Android build instructions
at https://code.google.com/p/chromium/wiki/AndroidBuildInstructions
are used. Mac runs into problems earlier in the build toolchain,
but as Mac is not a supported Android development platform in Chrome,
the files will have to be copied manually on that platform for now.
TEST=Synced, built and ran a test APK using run_tests.py.
BUG=1882
R=andrew@webrtc.org, henrike@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/1679005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4241 4adac7df-926f-26a2-2b94-8c16560cd09d
Add a build target for the Video demo app for Android that only
exists when OS=='android' during build.
Note that this doesn't solve webrtc:1029, it's more like a workaround
waiting for the complete solution, which is to great a proper GYP target
that doesn't involve an action and an external script.
BUG=1029
TEST=Built successfully with:
source build/android/envsetup.sh
gclient runhooks
ninja -C out/Debug
Also verified the target is not present when OS is not 'android'.
Review URL: https://webrtc-codereview.appspot.com/1286004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3769 4adac7df-926f-26a2-2b94-8c16560cd09d
This CL enables building with Android NDK in the way that Chromium buildbots do it.
== Overview ==
* Add Android dependencies to DEPS (SDK, NDK, Android test runner). This also makes it possible to use Android's build/android/run_tests.py script to execute tests on Android devices.
* Add a Python script to build the WebRTC Video demo for Android using ndk-build and Ant. This is designed as an annotation script for Buildbots but is also fine to run locally.
* Update Android.mk so it works with the compiler output from a build performed by build/android/buildbot/bb_run_bot.py (which is how Chrome buildbots build).
== Syncing Android dependencies ==
To get the dependencies added in DEPS synced out, you must change the last line
of your .gclient file to look like this:
];target_os = ["android"]
That will append another variable to the .gclient file that causes these
dependencies to be synced during gclient sync.
If you want to get additional platform-specific dependencies in the same
checkout, add them to the list too, e.g. target_os = ["android", "unix"].
== Android.mk ==
The fix in Android.mk is needed since Chrome is building using build/android/buildbot/bb_run_bot.py, which only output the libraries into out/Debug. With the change it works for both that and a normal build (which copies the library files from out/Debug/obj.target/subpath to out/Debug anyway as a part of the build).
== svn:ignore ==
NOTICE: Before submitting, the following directories should be added to svn:ignore in third_party to avoid them from being removed and re-synced for every build:
* android_testrunner
* android_tools
* WebKit
This has to be done in a manual SVN commit since it's not possible to include in a git-svn CL (and I don't want to migrate this to a SVN CL).
BUG=none
TEST=local builds
Review URL: https://webrtc-codereview.appspot.com/1024009
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3497 4adac7df-926f-26a2-2b94-8c16560cd09d
- Sync Chrome's DEPS to chromium_deps/, and use the From() keyword
to pull the correct revisions from it, rather than having to manually
enter them.
- This idea is borrowed from the WebKit DEPS:
http://trac.webkit.org/browser/trunk/Source/WebKit/chromium/DEPS
- Fix update.py's DEPS parsing to handle From().
- Roll Chrome 120526:122775.
- Organize the deps alphabetically.
- Sync the in-tree gold linker, which is now required due to a change
in the linker flags.
- Add the new deps to .gitignore.
BUG=
TEST=build on all platforms
Review URL: https://webrtc-codereview.appspot.com/401004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1755 4adac7df-926f-26a2-2b94-8c16560cd09d
The access token is now transferred between the request and track scripts.
Almost completed the implementation, using oauth2.
Initial, incomplete implementation of the permission-request script.
The coverage tracking is broken temporarily, but it can now make OAuth requests.
BUG=
TEST=
Review URL: https://webrtc-codereview.appspot.com/366002
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1530 4adac7df-926f-26a2-2b94-8c16560cd09d
To be able to re-use as much as possible from Chromium's scripts, I've created two customized scripts for running tests:
- webrtc_tests.py: a customized version of chrome_tests.py with WebRTC tests instead.
- webrtc_tests.sh: a customized version of the chrome_tests.sh wrapper script, to launch the above script.
The webrtc_tests.sh script is setting up PYTHONPATH so that tools/valgrind is available for the webrtc_tests.py script.
The webrtc_tests.py script inherits the chrome_tests.py script as much as possible, to minimize maintenance and maximize readability.
Having this mirrored setup of directories, allows us to use the same directory hierarchy for suppression files too. This CL only adds suppression files for memcheck, but we can add files for tsan later easily.
The suppression file is currently empty for Linux. For Mac I copied all the Chromium third-party suppressions. We will need a lot more added for Mac before Valgrind runs becomes usable for that platform.
The platform specific naming of the suppression files are handled automatically when webrtc_tests.sh is used to launch the tests.
Example:
Plain memcheck (default tool): tools/valgrind-webrtc/webrtc_tests.sh -t system_wrappers
Run ThreadSanitizer: tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t system_wrappers
Previously mentioned AddressSanitizer requires additional scripts to be added and it not covered in this CL.
BUG=
TEST=
Review URL: http://webrtc-codereview.appspot.com/322010
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1240 4adac7df-926f-26a2-2b94-8c16560cd09d
Separated new-style tests from old-style tests. Abstracted code for reuse.
Fully separated the new automated tests from the old-style tests. We now have old-style tests running in manual mode, old-style tests running in automated mode and new-style tests that uses input files and make actual video comparisons.
Introduced a small "library" of helper functions in order to move a lot
of stuff out of the original base and codec tests, which have been made
dependent on the new "library" (which is a header file and a source
file). The new-style tests also depends on this "library".
The comparison test flags are now required only when the comparison tests actually runs.
Separated comparison tests into its own test since it seems we will be running classic vie_auto_test using a fake video driver on Linux.
Made tbInterfaces follow Google conventions.
Merge branch 'render_to_file' into vivi_driver
Resolution alignment testing is now optional behind a flag.
BUG=
TEST=
Review URL: http://webrtc-codereview.appspot.com/269011
git-svn-id: http://webrtc.googlecode.com/svn/trunk@962 4adac7df-926f-26a2-2b94-8c16560cd09d
