We encounter a sample-underrun if NetEq is initialized with a sampling rate fs =16000 and receive Opus packets with frame-size less than 5 ms. The reason is as follows.
Let say NetEq buffer has 4 packets of Opus each of size 2.5ms this means that internally timestamp of packets incremented by 80 (internally Opus treated as 32 kHz codec). Given the initial sampling rate of NetEq, at the first time that it wants to fetch packets, it targets to fetch 160 samples. Therefore, it will only extracts 2 packets. Decoding these packets give us exactly 160 samples (5 ms at 32 kHz), however, upon decoding the first packet the internal sampling rate will be updated to 32 kHz. So it is expected that sync buffer to deliver 320 samples while it does only have 160 samples (or maybe few more as it starts with some zeros). And we encounter and under-run.
Even if we ignore the under-run "assert(sync_buffer_->FutureLength() >= expand_->overlap_length())" (neteq_impl.cc::811) is trigered. I'm not sure what happens if we remove this assert perhaps NetEq will work fine in subsequent calls. However the first under-run is blocking ACM2 test to pass.
Here I have a solution to update sample rate as soon as a packet is inserted, if required. It not a very efficient approach as we do the same reset in NetEqImpl::Decode().
It is a bit tricky to reproduce this because the TOT ACM tests do not run ACM2. In https://webrtc-codereview.appspot.com/2192005/ I have a patch to run both ACMs. To reproduce the problem, one can patch that CL and run
$ out/Debug/modules_tests --gtest_filter=AudioCodingModuleTest.TestOpus
Note that we would not encounter any problem if NetEq4 is initiated with 32000 Hz sampling rate. You can test this by setting |kNeteqInitSampleRateHz| to 32000 in webrtc/modules/audio_coding/main/acm2/acm_receiver.cc
BUG=
R=andrew@webrtc.org, henrik.lundin@webrtc.org, kjellander@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/2306004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4896 4adac7df-926f-26a2-2b94-8c16560cd09d
If there is audio in a codec's audio buffer and sample-rate or number of channels change the audio buffer has to reset. Otherwise, the amount of audio in the buffer is misinterpreted any syncronization between 10ms audio blocks and their associated timestamps is lost.
For instance, assume changing from stereo to mono when there is 10ms stereo in the buffer. The "new" codec will interpret this as 20 ms audio, therefore, 2 blocks of 10 ms, but there is only one timestamp. This will results in ACMGenericCodec::in_timestamp_ix_write_ updated to a negative number after an encode is performed.
The drawback with this solution is that if packet-size of the codec is changed then audio buffer is reset wich is not necessary. We accept this as it is a rare case in practice that clients of ACM re-register send codecs to change packet-size.
R=andrew@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/2151006
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4887 4adac7df-926f-26a2-2b94-8c16560cd09d
The untrusted delay mode provides an option to reduce the "known delay"
parameter passed down to the core AEC. This is necessary to handle the
very low latencies observed with the Chromium audio backend on Mac.
Prior to this change, it was possible to pass a negative value. The AEC
produced good output in practice, but it turned out this tripped a
heretofore unnoticed assert in ProcessBlock().
This change avoids the assert, and maintains the good output across a
set of Mac recordings. Bit-exact in some cases, and in the remaining,
quickly converging to identical output.
The assert was hit on the last webrtc roll in Chromium in
content_browsertests on Mac.
Corresponds to:
https://chromereviews.googleplex.com/9960013
TBR=bjornv
TESTED=Verified locally that "content_browsertests
--gtest_filter=WebrtcBrowserTest.*"" passes.
Review URL: https://webrtc-codereview.appspot.com/2328005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4886 4adac7df-926f-26a2-2b94-8c16560cd09d
- Reduce the window size from 20 to 10 seconds. If there is
any spurious high decode time, it will be faster to pass it.
- Ignore more samples at first because HW decoder has higher
initialization latency.
BUG=crbug.com/298176
TEST=Run apprtc loopback on Chromebook Daisy.
R=stefan@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/2315005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4874 4adac7df-926f-26a2-2b94-8c16560cd09d
a different thread.
One example is the _playWarning can be changed in AudioDeviceLinuxALSA::Init, which is called on the application's thread. At the same time it can be read via PlayoutWarning() on the VoE's process_thread.
RISK=P2
TESTED=try bots and tsan test:
tools/valgrind-webrtc/webrtc_tests.sh --tool=tsan -t out/Debug/libjingle_peerconnection_unittest --gtest_filter=PeerConnectionFactoryTestInternal.CreatePCUsingInternalModules
BUG=1205
R=xians@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/2315004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4866 4adac7df-926f-26a2-2b94-8c16560cd09d
As far as I observe, there are several reasons for this:
1. webrtc/modules/audio_coding/neteq4/neteq_impl.cc : 870
assert(new_codec_);
This is related to webrtc/modules/audio_coding/neteq4/decision_logic_normal.cc : 81
kUndefined can happen without new_codec_ being set
2. webrtc/modules/audio_coding/neteq4/neteq_impl.cc : 745
assert(sync_buffer_->FutureLength() >= expand_->overlap_length());
3. some other assert triggered.
The above happens not very often and goes away with no assertion.
3. (most common, this CL addresses this)
webrtc/modules/rtp_rtcp/source/rtp_receiver_audio.cc : 201
payload_data_length = payload_length - rtp_header->header.paddingLength;
There are situations that
payload_length < rtp_header->header.paddingLength;
OLD ACM + NetEq3 can handle this:
a) webrtc/modules/audio_coding/main/source/acm_neteq.cc : 477
int16_t payload_length = static_cast<int16_t>(length_payload);
payload_length becomes negative in this situation
b) webrtc/modules/audio_coding/neteq/recin.c
WebRtcNetEQ_RecInInternal() handles negative payload length
I do not want to touch VoE, so I tried to let ACM2 and NetEq4 handle negative payload length.
This CL does not follow the exact way of OLD ACM + NetEq3. I stopped negative payload length at ACM and did not allow it go to NetEq4.
To try this, apply my uploaded patch : https://webrtc-codereview.appspot.com/2281004/
Let me know if you see better solutions.
R=henrik.lundin@webrtc.org, stefan@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/2292005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4860 4adac7df-926f-26a2-2b94-8c16560cd09d
Re-land: http://review.webrtc.org/2151007/TBR=bjornv@webrtc.org
Original change description:
This mode extends the filter length from the current 48 ms to 128 ms.
It is runtime selectable which allows it to be enabled through
experiment. We reuse the DelayCorrection infrastructure to avoid having
to replumb everything up to libjingle.
Increases AEC complexity by ~50% on modern x86 CPUs.
Measurements (in percent of usage on one core):
Machine/CPU Normal Extended
MacBook Retina (Early 2013),
Core i7 Ivy Bridge (2.7 GHz, hyperthreaded) 0.6% 0.9%
MacBook Air (Late 2010), Core 2 Duo (2.13 GHz) 1.4% 2.7%
Chromebook Pixel, Core i5 Ivy Bridge (1.8 GHz) 0.6% 1.0%
Samsung ARM Chromebook,
Samsung Exynos 5 Dual (1.7 GHz) 3.2% 5.6%
The relative value is large of course but the absolute should be
acceptable in order to have a working AEC on some platforms.
Detailed changes to the algorithm:
- The filter length is changed from 48 to 128 ms. This comes with tuning
of several parameters: i) filter adaptation stepsize and error
threshold; ii) non-linear processing smoothing and overdrive.
- Option to ignore the reported delays on platforms which we deem
sufficiently unreliable. Currently this will be enabled in Chromium for
Mac.
- Faster startup times by removing the excessive "startup phase"
processing of reported delays.
- Much more conservative adjustments to the far-end read pointer. We
smooth the delay difference more heavily, and back off from the
difference more. Adjustments force a readaptation of the filter, so they
should be avoided except when really necessary.
Corresponds to these changes:
https://chromereviews.googleplex.com/9412014https://chromereviews.googleplex.com/9514013https://chromereviews.googleplex.com/9960013
BUG=454,827,1261
Review URL: https://webrtc-codereview.appspot.com/2295006
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4848 4adac7df-926f-26a2-2b94-8c16560cd09d
DWM doesn't update window decorations in bitmap captured
from GetWindowDC(). Work it around by calling PrintWindow()
after each resize (which somehow affects what's later
captured from GetWindowDC()). That solution avoids the
downsides of PrintWindow() (namely flickering) while still
allowing to capture window decorations correctly.
BUG=crbug.com/289759
R=alexeypa@chromium.org
Review URL: https://webrtc-codereview.appspot.com/2295004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4847 4adac7df-926f-26a2-2b94-8c16560cd09d
The map was just mapping an index to a pointer of a POD, so the code is easily
simplified by using a vector (with implicit index key) and the POD as a value.
(also fixes a leak in the windows code, which lacked a virtual dtor for
VideoCaptureCapabilityWindows but was deleting through a base pointer).
R=perkj@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/2298004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4840 4adac7df-926f-26a2-2b94-8c16560cd09d
