generic-library/vpx
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
215b010f4b
vpx/test/error_resilience_test.cc

231 lines
7.1 KiB
C++
Raw Normal View History

Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
/*
Copyright (c) 2012 The WebM project authors. All Rights Reserved.
Use of this source code is governed by a BSD-style license
that can be found in the LICENSE file in the root of the source
tree. An additional intellectual property rights grant can be found
in the file PATENTS. All contributing project authors may
be found in the AUTHORS file in the root of the source tree.
*/
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
#include "third_party/googletest/src/include/gtest/gtest.h"
Support multiple codecs in test infrastructure This commit starts to convert the tests to a system where the codec to be used is provided by a factory object. Currently no tests are instantiated for VP9 since they all fail for various reasons, but it was verified that they're called and the correct codec is instantiated. Change-Id: Ia7506df2ca3a7651218ba3ca560634f08c9fbdeb
2013-01-18 20:51:12 +01:00
#include "test/codec_factory.h"
Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
#include "test/encode_test_driver.h"
#include "test/i420_video_source.h"
Support multiple codecs in test infrastructure This commit starts to convert the tests to a system where the codec to be used is provided by a factory object. Currently no tests are instantiated for VP9 since they all fail for various reasons, but it was verified that they're called and the correct codec is instantiated. Change-Id: Ia7506df2ca3a7651218ba3ca560634f08c9fbdeb
2013-01-18 20:51:12 +01:00
#include "test/util.h"
Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
namespace {
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
const int kMaxErrorFrames = 8;
const int kMaxDroppableFrames = 8;
Support multiple codecs in test infrastructure This commit starts to convert the tests to a system where the codec to be used is provided by a factory object. Currently no tests are instantiated for VP9 since they all fail for various reasons, but it was verified that they're called and the correct codec is instantiated. Change-Id: Ia7506df2ca3a7651218ba3ca560634f08c9fbdeb
2013-01-18 20:51:12 +01:00
class ErrorResilienceTest : public ::libvpx_test::EncoderTest,
public ::libvpx_test::CodecTestWithParam<libvpx_test::TestMode> {
Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
protected:
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
ErrorResilienceTest() : EncoderTest(GET_PARAM(0)),
psnr_(0.0),
nframes_(0),
mismatch_psnr_(0.0),
mismatch_nframes_(0),
encoding_mode_(GET_PARAM(1)) {
Reset();
}
Support multiple codecs in test infrastructure This commit starts to convert the tests to a system where the codec to be used is provided by a factory object. Currently no tests are instantiated for VP9 since they all fail for various reasons, but it was verified that they're called and the correct codec is instantiated. Change-Id: Ia7506df2ca3a7651218ba3ca560634f08c9fbdeb
2013-01-18 20:51:12 +01:00
Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
virtual ~ErrorResilienceTest() {}
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
void Reset() {
error_nframes_ = 0;
droppable_nframes_ = 0;
}
Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
virtual void SetUp() {
InitializeConfig();
SetMode(encoding_mode_);
}
virtual void BeginPassHook(unsigned int /*pass*/) {
psnr_ = 0.0;
nframes_ = 0;
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
mismatch_psnr_ = 0.0;
mismatch_nframes_ = 0;
Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
}
virtual void PSNRPktHook(const vpx_codec_cx_pkt_t *pkt) {
psnr_ += pkt->data.psnr.psnr[0];
nframes_++;
}
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
virtual void PreEncodeFrameHook(libvpx_test::VideoSource *video) {
frame_flags_ &= ~(VP8_EFLAG_NO_UPD_LAST |
VP8_EFLAG_NO_UPD_GF |
VP8_EFLAG_NO_UPD_ARF);
if (droppable_nframes_ > 0 &&
(cfg_.g_pass == VPX_RC_LAST_PASS || cfg_.g_pass == VPX_RC_ONE_PASS)) {
for (unsigned int i = 0; i < droppable_nframes_; ++i) {
if (droppable_frames_[i] == nframes_) {
std::cout << " Encoding droppable frame: "
<< droppable_frames_[i] << "\n";
frame_flags_ |= (VP8_EFLAG_NO_UPD_LAST |
VP8_EFLAG_NO_UPD_GF |
VP8_EFLAG_NO_UPD_ARF);
return;
}
}
}
}
Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
double GetAveragePsnr() const {
if (nframes_)
return psnr_ / nframes_;
return 0.0;
}
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
double GetAverageMismatchPsnr() const {
if (mismatch_nframes_)
return mismatch_psnr_ / mismatch_nframes_;
return 0.0;
}
virtual bool DoDecode() const {
if (error_nframes_ > 0 &&
(cfg_.g_pass == VPX_RC_LAST_PASS || cfg_.g_pass == VPX_RC_ONE_PASS)) {
for (unsigned int i = 0; i < error_nframes_; ++i) {
if (error_frames_[i] == nframes_ - 1) {
std::cout << " Skipping decoding frame: "
<< error_frames_[i] << "\n";
return 0;
}
}
}
return 1;
}
virtual void MismatchHook(const vpx_image_t *img1,
const vpx_image_t *img2) {
double mismatch_psnr = compute_psnr(img1, img2);
mismatch_psnr_ += mismatch_psnr;
++mismatch_nframes_;
// std::cout << "Mismatch frame psnr: " << mismatch_psnr << "\n";
}
void SetErrorFrames(int num, unsigned int *list) {
if (num > kMaxErrorFrames)
num = kMaxErrorFrames;
else if (num < 0)
num = 0;
error_nframes_ = num;
for (unsigned int i = 0; i < error_nframes_; ++i)
error_frames_[i] = list[i];
}
void SetDroppableFrames(int num, unsigned int *list) {
if (num > kMaxDroppableFrames)
num = kMaxDroppableFrames;
else if (num < 0)
num = 0;
droppable_nframes_ = num;
for (unsigned int i = 0; i < droppable_nframes_; ++i)
droppable_frames_[i] = list[i];
}
unsigned int GetMismatchFrames() {
return mismatch_nframes_;
}
Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
private:
double psnr_;
unsigned int nframes_;
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
unsigned int error_nframes_;
unsigned int droppable_nframes_;
double mismatch_psnr_;
unsigned int mismatch_nframes_;
unsigned int error_frames_[kMaxErrorFrames];
unsigned int droppable_frames_[kMaxDroppableFrames];
Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
libvpx_test::TestMode encoding_mode_;
};
TEST_P(ErrorResilienceTest, OnVersusOff) {
const vpx_rational timebase = { 33333333, 1000000000 };
cfg_.g_timebase = timebase;
cfg_.rc_target_bitrate = 2000;
cfg_.g_lag_in_frames = 25;
init_flags_ = VPX_CODEC_USE_PSNR;
libvpx_test::I420VideoSource video("hantro_collage_w352h288.yuv", 352, 288,
timebase.den, timebase.num, 0, 30);
// Error resilient mode OFF.
cfg_.g_error_resilient = 0;
ASSERT_NO_FATAL_FAILURE(RunLoop(&video));
const double psnr_resilience_off = GetAveragePsnr();
EXPECT_GT(psnr_resilience_off, 25.0);
// Error resilient mode ON.
cfg_.g_error_resilient = 1;
ASSERT_NO_FATAL_FAILURE(RunLoop(&video));
const double psnr_resilience_on = GetAveragePsnr();
EXPECT_GT(psnr_resilience_on, 25.0);
// Test that turning on error resilient mode hurts by 10% at most.
if (psnr_resilience_off > 0.0) {
const double psnr_ratio = psnr_resilience_on / psnr_resilience_off;
EXPECT_GE(psnr_ratio, 0.9);
EXPECT_LE(psnr_ratio, 1.1);
}
}
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
TEST_P(ErrorResilienceTest, DropFramesWithoutRecovery) {
const vpx_rational timebase = { 33333333, 1000000000 };
cfg_.g_timebase = timebase;
Revert "Adjust error resilience test data rate." This reverts commit b24735c622cefe3a03633edd69dff65d2a6ee0dd since the adjusted threshold doesn't allow the existing tests to pass. Will disable the failing test in a separate commit. Change-Id: I26d41cf6175f300bbad493cecdc96e6b0dd6f2fe
2013-05-07 21:58:32 +02:00
cfg_.rc_target_bitrate = 500;
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
init_flags_ = VPX_CODEC_USE_PSNR;
libvpx_test::I420VideoSource video("hantro_collage_w352h288.yuv", 352, 288,
timebase.den, timebase.num, 0, 30);
// Error resilient mode ON.
cfg_.g_error_resilient = 1;
// Set an arbitrary set of error frames same as droppable frames
unsigned int num_droppable_frames = 2;
unsigned int droppable_frame_list[] = {5, 16};
SetDroppableFrames(num_droppable_frames, droppable_frame_list);
SetErrorFrames(num_droppable_frames, droppable_frame_list);
ASSERT_NO_FATAL_FAILURE(RunLoop(&video));
// Test that no mismatches have been found
std::cout << " Mismatch frames: "
<< GetMismatchFrames() << "\n";
EXPECT_EQ(GetMismatchFrames(), (unsigned int) 0);
// reset previously set error/droppable frames
Reset();
Partially disable error resilience test Disables the part of the error-resilient test that tests the quality after dropping undroppable frames. It's not clear how to set the threshold for this correctly at the moment. Change-Id: I3ee4a0d475498f44711fdef05749f305e8d08591
2013-05-07 22:02:45 +02:00
#if 0
// TODO(jkoleszar): This test is disabled for the time being as too
// sensitive. It's not clear how to set a reasonable threshold for
// this behavior.
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
// Now set an arbitrary set of error frames that are non-droppable
unsigned int num_error_frames = 3;
unsigned int error_frame_list[] = {3, 10, 20};
SetErrorFrames(num_error_frames, error_frame_list);
ASSERT_NO_FATAL_FAILURE(RunLoop(&video));
Partially disable error resilience test Disables the part of the error-resilient test that tests the quality after dropping undroppable frames. It's not clear how to set the threshold for this correctly at the moment. Change-Id: I3ee4a0d475498f44711fdef05749f305e8d08591
2013-05-07 22:02:45 +02:00
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
// Test that dropping an arbitrary set of inter frames does not hurt too much
// Note the Average Mismatch PSNR is the average of the PSNR between
// decoded frame and encoder's version of the same frame for all frames
// with mismatch.
const double psnr_resilience_mismatch = GetAverageMismatchPsnr();
std::cout << " Mismatch PSNR: "
<< psnr_resilience_mismatch << "\n";
EXPECT_GT(psnr_resilience_mismatch, 20.0);
Partially disable error resilience test Disables the part of the error-resilient test that tests the quality after dropping undroppable frames. It's not clear how to set the threshold for this correctly at the moment. Change-Id: I3ee4a0d475498f44711fdef05749f305e8d08591
2013-05-07 22:02:45 +02:00
#endif
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
}
Support multiple codecs in test infrastructure This commit starts to convert the tests to a system where the codec to be used is provided by a factory object. Currently no tests are instantiated for VP9 since they all fail for various reasons, but it was verified that they're called and the correct codec is instantiated. Change-Id: Ia7506df2ca3a7651218ba3ca560634f08c9fbdeb
2013-01-18 20:51:12 +01:00
VP8_INSTANTIATE_TEST_CASE(ErrorResilienceTest, ONE_PASS_TEST_MODES);
Adds an error-resilient mode with test Adds an error-resilient mode where frames can be continued to be decoded even when there are errors (due to network losses) on a prior frame. Specifically, backward updates are turned off and probabilities of various symbols are reset to defaults at the beginning of each frame. Further, the last frame's mvs are not used for the mv reference list, and the sorting of the initial list based on search on previous frames is turned off as well. Also adds a test where an arbitrary set of frames are skipped from decoding to simulate errors. The test verifies (1) that if the error frames are droppable - i.e. frame buffer updates have been turned off - there are no mismatch errors for the remaining frames after the error frames; and (2) if the error-frames are non droppable, there are not only no decoding errors but the mismatch PSNR between the decoder's version of the post-error frames and the encoder's version is at least 20 dB. Change-Id: Ie6e2bcd436b1e8643270356d3a930e8989ff52a5
2013-01-15 15:43:35 +01:00
VP9_INSTANTIATE_TEST_CASE(ErrorResilienceTest, ONE_PASS_TEST_MODES);
Unit Test for Error Resilience Mode This unit test compares the difference in quality with error resilience enabled and disabled. The test runs for all of the one-pass encoding modes. The test ensures that the effect of turning on error resilience makes less than a 10% difference in PSNR. Further cases should be added to do a more comprehensive test. Change-Id: I1fc747fc78c9459bc6c74494f4b38308dbed0c32
2012-10-02 21:16:27 +02:00
} // namespace
Reference in New Issue Copy Permalink