generic-library/vpx
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
c98bfcfbbb
vpx/vp9/encoder/vp9_rdopt.c

3234 lines
116 KiB
C
Raw Normal View History

Initial WebM release
2010-05-18 17:58:33 +02:00
/*
Use WebM in copyright notice for consistency Changes 'The VP8 project' to 'The WebM project', for consistency with other webmproject.org repositories. Fixes issue #97. Change-Id: I37c13ed5fbdb9d334ceef71c6350e9febed9bbba
2010-09-09 14:16:39 +02:00
* Copyright (c) 2010 The WebM project authors. All Rights Reserved.
Initial WebM release
2010-05-18 17:58:33 +02:00
*
cosmetics: trim trailing whitespace When the license headers were updated, they accidentally contained trailing whitespace, so unfortunately we have to touch all the files again. Change-Id: I236c05fade06589e417179c0444cb39b09e4200d
2010-06-18 18:39:21 +02:00
* Use of this source code is governed by a BSD-style license
LICENSE: update with latest text Change-Id: Ieebea089095d9073b3a94932791099f614ce120c
2010-06-04 22:19:40 +02:00
* that can be found in the LICENSE file in the root of the source
* tree. An additional intellectual property rights grant can be found
cosmetics: trim trailing whitespace When the license headers were updated, they accidentally contained trailing whitespace, so unfortunately we have to touch all the files again. Change-Id: I236c05fade06589e417179c0444cb39b09e4200d
2010-06-18 18:39:21 +02:00
* in the file PATENTS. All contributing project authors may
LICENSE: update with latest text Change-Id: Ieebea089095d9073b3a94932791099f614ce120c
2010-06-04 22:19:40 +02:00
* be found in the AUTHORS file in the root of the source tree.
Initial WebM release
2010-05-18 17:58:33 +02:00
*/
#include <stdio.h>
#include <math.h>
#include <limits.h>
#include <assert.h>
Add vp9_ prefix to all vp9 files Support for gyp which doesn't support multiple objects in the same static library having the same basename. Change-Id: Ib947eefbaf68f8b177a796d23f875ccdfa6bc9dc
2012-11-27 22:59:17 +01:00
Removing MAXF macro and using MAX instead. Change-Id: I51c53692b1150005645bf362c5e5a8275178a8fd
2013-05-02 20:57:16 +02:00
#include "vp9/common/vp9_pragmas.h"
fixed includes to be fully specified Change-Id: Ia1cce221f8511561b9cbd8edb7726fbc286ff243
2012-11-28 19:41:40 +01:00
#include "vp9/encoder/vp9_tokenize.h"
#include "vp9/encoder/vp9_treewriter.h"
#include "vp9/encoder/vp9_onyx_int.h"
#include "vp9/encoder/vp9_modecosts.h"
#include "vp9/encoder/vp9_encodeintra.h"
Add vp9_ prefix to all vp9 files Support for gyp which doesn't support multiple objects in the same static library having the same basename. Change-Id: Ib947eefbaf68f8b177a796d23f875ccdfa6bc9dc
2012-11-27 22:59:17 +01:00
#include "vp9/common/vp9_entropymode.h"
#include "vp9/common/vp9_reconinter.h"
#include "vp9/common/vp9_reconintra.h"
#include "vp9/common/vp9_findnearmv.h"
#include "vp9/common/vp9_quant_common.h"
fixed includes to be fully specified Change-Id: Ia1cce221f8511561b9cbd8edb7726fbc286ff243
2012-11-28 19:41:40 +01:00
#include "vp9/encoder/vp9_encodemb.h"
#include "vp9/encoder/vp9_quantize.h"
#include "vp9/encoder/vp9_variance.h"
#include "vp9/encoder/vp9_mcomp.h"
#include "vp9/encoder/vp9_rdopt.h"
#include "vp9/encoder/vp9_ratectrl.h"
Initial WebM release
2010-05-18 17:58:33 +02:00
#include "vpx_mem/vpx_mem.h"
Add vp9_ prefix to all vp9 files Support for gyp which doesn't support multiple objects in the same static library having the same basename. Change-Id: Ib947eefbaf68f8b177a796d23f875ccdfa6bc9dc
2012-11-27 22:59:17 +01:00
#include "vp9/common/vp9_systemdependent.h"
#include "vp9/encoder/vp9_encodemv.h"
#include "vp9/common/vp9_seg_common.h"
#include "vp9/common/vp9_pred_common.h"
#include "vp9/common/vp9_entropy.h"
support building vp8 and vp9 into a single lib Change-Id: Ib8f8a66c9fd31e508cdc9caa662192f38433aa3d
2012-11-09 02:09:30 +01:00
#include "vp9_rtcd.h"
Add vp9_ prefix to all vp9 files Support for gyp which doesn't support multiple objects in the same static library having the same basename. Change-Id: Ib947eefbaf68f8b177a796d23f875ccdfa6bc9dc
2012-11-27 22:59:17 +01:00
#include "vp9/common/vp9_mvref_common.h"
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
#include "vp9/common/vp9_common.h"
New Motion Reference Search Alternative strategy for finding a list of candidate motion vectors to use as reference values in mv coding and as nearest and near. Sort by sad in vp8_find_best_ref_mvs() rather than just pick the best. Allow 0,0 as a best ref option but not a nearest or near unless there are no alternatives. Encode/Decode verified on at least some clips. Some commented out experimental and stats code still in place. Gain over existing code averages about 1% on derf (alll metrics) with improvement on all clips. Other test results pending. The entropy coding of the mode (nearest/near etc) still depends upon and requires the old "findnear" code so this needs looking at and may provide room for further gains. Change-Id: I871d7cba1d1c379c4bad9bcccce1fb19c46b8247
2012-08-24 16:44:01 +02:00
Compound prediction for splitmv macroblocks. Change-Id: I0af3395500b1cb0ed629249eb6636a0c9322cb18
2012-04-18 22:51:58 +02:00
#define INVALID_MV 0x80008000
Adds support for switchable interpolation filters. Allows for swtiching/setting interpolation filters at the MB level. A frame level flag indicates whether to use a specifc filter for the entire frame or to signal the interpolation filter for each MB. When switchable filters are used, the encoder chooses between 8-tap and 8-tap sharp filters. The code currently has options to explore other variations as well, which will be cleaned up subsequently. One issue with the framework is that encoding is slow. I tried to do some tricks to speed things up but it is still slow. Decoding speed should not be affected since the number of filter taps remain unchanged. With the current version, we are up 0.5% on derf on average but some videos city/mobile improve by close to 4 and 2% respectively. If we did a full-search by turning the SEARCH_BEST_FILTER flag on, the results are somewhat better. The framework can be combined with filtered prediction, and I seek feedback regarding that. Rebased. Change-Id: I8f632cb2c111e76284140a2bd480945d6d42b77a
2012-07-18 22:43:01 +02:00
/* Factor to weigh the rate for switchable interp filters */
#define SWITCHABLE_INTERP_RATE_FACTOR 1
Residual coding to cache energy class of tokens. Proposal for tuning the residual coding by changing how the context from previous tokens is calculated. Storing the energy class of previous tokens instead of the token itself eases the critical path of HW implementations. Change-Id: I6d71d856b84518f6c88de771ddd818436f794bab
2013-05-23 12:08:00 +02:00
DECLARE_ALIGNED(16, extern const uint8_t,
vp9_pt_energy_class[MAX_ENTROPY_TOKENS]);
Change non-function symbol vp8_ prefixes to vp9_. For local symbols, make them static instead. Change-Id: I13d60947a46f711bc8991e16100cea2a13e3a22e
2012-10-31 01:12:12 +01:00
const MODE_DEFINITION vp9_mode_order[MAX_MODES] = {
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
{ZEROMV, LAST_FRAME, NONE},
{DC_PRED, INTRA_FRAME, NONE},
Initial WebM release
2010-05-18 17:58:33 +02:00
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
{NEARESTMV, LAST_FRAME, NONE},
{NEARMV, LAST_FRAME, NONE},
Initial WebM release
2010-05-18 17:58:33 +02:00
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
{ZEROMV, GOLDEN_FRAME, NONE},
{NEARESTMV, GOLDEN_FRAME, NONE},
Initial WebM release
2010-05-18 17:58:33 +02:00
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
{ZEROMV, ALTREF_FRAME, NONE},
{NEARESTMV, ALTREF_FRAME, NONE},
Initial WebM release
2010-05-18 17:58:33 +02:00
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
{NEARMV, GOLDEN_FRAME, NONE},
{NEARMV, ALTREF_FRAME, NONE},
Initial WebM release
2010-05-18 17:58:33 +02:00
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
{V_PRED, INTRA_FRAME, NONE},
{H_PRED, INTRA_FRAME, NONE},
{D45_PRED, INTRA_FRAME, NONE},
{D135_PRED, INTRA_FRAME, NONE},
{D117_PRED, INTRA_FRAME, NONE},
{D153_PRED, INTRA_FRAME, NONE},
{D27_PRED, INTRA_FRAME, NONE},
{D63_PRED, INTRA_FRAME, NONE},
Initial WebM release
2010-05-18 17:58:33 +02:00
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
{TM_PRED, INTRA_FRAME, NONE},
Initial WebM release
2010-05-18 17:58:33 +02:00
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
{NEWMV, LAST_FRAME, NONE},
{NEWMV, GOLDEN_FRAME, NONE},
{NEWMV, ALTREF_FRAME, NONE},
Initial WebM release
2010-05-18 17:58:33 +02:00
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
{SPLITMV, LAST_FRAME, NONE},
{SPLITMV, GOLDEN_FRAME, NONE},
{SPLITMV, ALTREF_FRAME, NONE},
Dual 16x16 inter prediction. This patch introduces the concept of dual inter16x16 prediction. A 16x16 inter-predicted macroblock can use 2 references instead of 1, where both references use the same mvmode (new, near/est, zero). In the case of newmv, this means that two MVs are coded instead of one. The frame can be encoded in 3 ways: all MBs single-prediction, all MBs dual prediction, or per-MB single/dual prediction selection ("hybrid"), in which case a single bit is coded per-MB to indicate whether the MB uses single or dual inter prediction. In the future, we can (maybe?) get further gains by mixing this with Adrian's 32x32 work, per-segment dual prediction settings, or adding support for dual splitmv/8x8mv inter prediction. Gain (on derf-set, CQ mode) is ~2.8% (SSIM) or ~3.6% (glb PSNR). Most gain is at medium/high bitrates, but there's minor gains at low bitrates also. Output was confirmed to match between encoder and decoder. Note for optimization people: this patch introduces a 2nd version of 16x16/8x8 sixtap/bilin functions, which does an avg instead of a store. They may want to look and make sure this is implemented to their satisfaction so we can optimize it best in the future. Change-ID: I59dc84b07cbb3ccf073ac0f756d03d294cb19281
2011-12-06 20:53:02 +01:00
Rename B_PRED to I4X4_PRED So it is consistent with I8x8_PRED. Change-Id: Iefa65124b2419690d83e526c611129c0ede29d11
2013-04-11 21:16:35 +02:00
{I4X4_PRED, INTRA_FRAME, NONE},
Dual 16x16 inter prediction. This patch introduces the concept of dual inter16x16 prediction. A 16x16 inter-predicted macroblock can use 2 references instead of 1, where both references use the same mvmode (new, near/est, zero). In the case of newmv, this means that two MVs are coded instead of one. The frame can be encoded in 3 ways: all MBs single-prediction, all MBs dual prediction, or per-MB single/dual prediction selection ("hybrid"), in which case a single bit is coded per-MB to indicate whether the MB uses single or dual inter prediction. In the future, we can (maybe?) get further gains by mixing this with Adrian's 32x32 work, per-segment dual prediction settings, or adding support for dual splitmv/8x8mv inter prediction. Gain (on derf-set, CQ mode) is ~2.8% (SSIM) or ~3.6% (glb PSNR). Most gain is at medium/high bitrates, but there's minor gains at low bitrates also. Output was confirmed to match between encoder and decoder. Note for optimization people: this patch introduces a 2nd version of 16x16/8x8 sixtap/bilin functions, which does an avg instead of a store. They may want to look and make sure this is implemented to their satisfaction so we can optimize it best in the future. Change-ID: I59dc84b07cbb3ccf073ac0f756d03d294cb19281
2011-12-06 20:53:02 +01:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
/* compound prediction modes */
{ZEROMV, LAST_FRAME, GOLDEN_FRAME},
{NEARESTMV, LAST_FRAME, GOLDEN_FRAME},
{NEARMV, LAST_FRAME, GOLDEN_FRAME},
Dual 16x16 inter prediction. This patch introduces the concept of dual inter16x16 prediction. A 16x16 inter-predicted macroblock can use 2 references instead of 1, where both references use the same mvmode (new, near/est, zero). In the case of newmv, this means that two MVs are coded instead of one. The frame can be encoded in 3 ways: all MBs single-prediction, all MBs dual prediction, or per-MB single/dual prediction selection ("hybrid"), in which case a single bit is coded per-MB to indicate whether the MB uses single or dual inter prediction. In the future, we can (maybe?) get further gains by mixing this with Adrian's 32x32 work, per-segment dual prediction settings, or adding support for dual splitmv/8x8mv inter prediction. Gain (on derf-set, CQ mode) is ~2.8% (SSIM) or ~3.6% (glb PSNR). Most gain is at medium/high bitrates, but there's minor gains at low bitrates also. Output was confirmed to match between encoder and decoder. Note for optimization people: this patch introduces a 2nd version of 16x16/8x8 sixtap/bilin functions, which does an avg instead of a store. They may want to look and make sure this is implemented to their satisfaction so we can optimize it best in the future. Change-ID: I59dc84b07cbb3ccf073ac0f756d03d294cb19281
2011-12-06 20:53:02 +01:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
{ZEROMV, ALTREF_FRAME, LAST_FRAME},
{NEARESTMV, ALTREF_FRAME, LAST_FRAME},
{NEARMV, ALTREF_FRAME, LAST_FRAME},
Dual 16x16 inter prediction. This patch introduces the concept of dual inter16x16 prediction. A 16x16 inter-predicted macroblock can use 2 references instead of 1, where both references use the same mvmode (new, near/est, zero). In the case of newmv, this means that two MVs are coded instead of one. The frame can be encoded in 3 ways: all MBs single-prediction, all MBs dual prediction, or per-MB single/dual prediction selection ("hybrid"), in which case a single bit is coded per-MB to indicate whether the MB uses single or dual inter prediction. In the future, we can (maybe?) get further gains by mixing this with Adrian's 32x32 work, per-segment dual prediction settings, or adding support for dual splitmv/8x8mv inter prediction. Gain (on derf-set, CQ mode) is ~2.8% (SSIM) or ~3.6% (glb PSNR). Most gain is at medium/high bitrates, but there's minor gains at low bitrates also. Output was confirmed to match between encoder and decoder. Note for optimization people: this patch introduces a 2nd version of 16x16/8x8 sixtap/bilin functions, which does an avg instead of a store. They may want to look and make sure this is implemented to their satisfaction so we can optimize it best in the future. Change-ID: I59dc84b07cbb3ccf073ac0f756d03d294cb19281
2011-12-06 20:53:02 +01:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
{ZEROMV, GOLDEN_FRAME, ALTREF_FRAME},
{NEARESTMV, GOLDEN_FRAME, ALTREF_FRAME},
{NEARMV, GOLDEN_FRAME, ALTREF_FRAME},
Dual 16x16 inter prediction. This patch introduces the concept of dual inter16x16 prediction. A 16x16 inter-predicted macroblock can use 2 references instead of 1, where both references use the same mvmode (new, near/est, zero). In the case of newmv, this means that two MVs are coded instead of one. The frame can be encoded in 3 ways: all MBs single-prediction, all MBs dual prediction, or per-MB single/dual prediction selection ("hybrid"), in which case a single bit is coded per-MB to indicate whether the MB uses single or dual inter prediction. In the future, we can (maybe?) get further gains by mixing this with Adrian's 32x32 work, per-segment dual prediction settings, or adding support for dual splitmv/8x8mv inter prediction. Gain (on derf-set, CQ mode) is ~2.8% (SSIM) or ~3.6% (glb PSNR). Most gain is at medium/high bitrates, but there's minor gains at low bitrates also. Output was confirmed to match between encoder and decoder. Note for optimization people: this patch introduces a 2nd version of 16x16/8x8 sixtap/bilin functions, which does an avg instead of a store. They may want to look and make sure this is implemented to their satisfaction so we can optimize it best in the future. Change-ID: I59dc84b07cbb3ccf073ac0f756d03d294cb19281
2011-12-06 20:53:02 +01:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
{NEWMV, LAST_FRAME, GOLDEN_FRAME},
{NEWMV, ALTREF_FRAME, LAST_FRAME },
{NEWMV, GOLDEN_FRAME, ALTREF_FRAME},
Compound prediction for splitmv macroblocks. Change-Id: I0af3395500b1cb0ed629249eb6636a0c9322cb18
2012-04-18 22:51:58 +02:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
{SPLITMV, LAST_FRAME, GOLDEN_FRAME},
{SPLITMV, ALTREF_FRAME, LAST_FRAME },
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
{SPLITMV, GOLDEN_FRAME, ALTREF_FRAME},
Initial WebM release
2010-05-18 17:58:33 +02:00
};
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#if CONFIG_BALANCED_COEFTREE
static void fill_token_costs(vp9_coeff_count *c,
vp9_coeff_count *cnoskip,
vp9_coeff_probs_model *p,
TX_SIZE tx_size) {
int i, j, k, l;
for (i = 0; i < BLOCK_TYPES; i++)
for (j = 0; j < REF_TYPES; j++)
for (k = 0; k < COEF_BANDS; k++)
for (l = 0; l < PREV_COEF_CONTEXTS; l++) {
vp9_prob probs[ENTROPY_NODES];
vp9_model_to_full_probs(p[i][j][k][l], probs);
vp9_cost_tokens((int *)cnoskip[i][j][k][l], probs,
vp9_coef_tree);
// Replace the eob node prob with a very small value so that the
// cost approximately equals the cost without the eob node
probs[1] = 1;
vp9_cost_tokens((int *)c[i][j][k][l], probs, vp9_coef_tree);
}
}
#else
Introduce vp9_coeff_probs/counts/stats/accum types. Use these, instead of the 4/5-dimensional arrays, to hold statistics, counts, accumulations and probabilities for coefficient tokens. This commit also re-allows ENTROPY_STATS to compile. Change-Id: If441ffac936f52a3af91d8f2922ea8a0ceabdaa5
2012-12-08 01:09:59 +01:00
static void fill_token_costs(vp9_coeff_count *c,
Updating the model coef experiment Cleans up the experiment. Actually uses reduced counts for backward updates, and reduced number of probabilities in the context. No change in bitstream when the experiment is on. Between expt on and off: derfraw300 is down only -0.062% (which is better than when expts were run previously). Change-Id: I55285a049a0c22810bdb42914212ab5a4f8521b5
2013-05-17 15:40:25 +02:00
vp9_coeff_probs_model *p,
Framework changes in nzc to allow more flexibility The patch adds the flexibility to use standard EOB based coding on smaller block sizes and nzc based coding on larger blocksizes. The tx-sizes that use nzc based coding and those that use EOB based coding are controlled by a function get_nzc_used(). By default, this function uses nzc based coding for 16x16 and 32x32 transform blocks, which seem to bridge the performance gap substantially. All sets are now lower by 0.5% to 0.7%, as opposed to ~1.8% before. Change-Id: I06abed3df57b52d241ea1f51b0d571c71e38fd0b
2013-03-26 23:23:30 +01:00
TX_SIZE tx_size) {
Split coefficient token tables intra vs. inter. Change-Id: I5416455f8f129ca0f450d00e48358d2012605072
2013-02-19 22:36:38 +01:00
int i, j, k, l;
Framework changes in nzc to allow more flexibility The patch adds the flexibility to use standard EOB based coding on smaller block sizes and nzc based coding on larger blocksizes. The tx-sizes that use nzc based coding and those that use EOB based coding are controlled by a function get_nzc_used(). By default, this function uses nzc based coding for 16x16 and 32x32 transform blocks, which seem to bridge the performance gap substantially. All sets are now lower by 0.5% to 0.7%, as opposed to ~1.8% before. Change-Id: I06abed3df57b52d241ea1f51b0d571c71e38fd0b
2013-03-26 23:23:30 +01:00
for (i = 0; i < BLOCK_TYPES; i++)
Split coefficient token tables intra vs. inter. Change-Id: I5416455f8f129ca0f450d00e48358d2012605072
2013-02-19 22:36:38 +01:00
for (j = 0; j < REF_TYPES; j++)
for (k = 0; k < COEF_BANDS; k++)
Updating the model coef experiment Cleans up the experiment. Actually uses reduced counts for backward updates, and reduced number of probabilities in the context. No change in bitstream when the experiment is on. Between expt on and off: derfraw300 is down only -0.062% (which is better than when expts were run previously). Change-Id: I55285a049a0c22810bdb42914212ab5a4f8521b5
2013-05-17 15:40:25 +02:00
for (l = 0; l < PREV_COEF_CONTEXTS; l++) {
vp9_prob probs[ENTROPY_NODES];
Using 128 entry look up table for coef models Reverts to using 128 bit LUT for the coef models rather than 48 to ease hardware implementation. Also incorporates some cleanups including removing various hooks to support different lookup tables based on block_type and ref_type. Change-Id: I54100c120cca07a2ebd3a7776bc4630fa6a153f6
2013-05-22 16:28:29 +02:00
vp9_model_to_full_probs(p[i][j][k][l], probs);
Updating the model coef experiment Cleans up the experiment. Actually uses reduced counts for backward updates, and reduced number of probabilities in the context. No change in bitstream when the experiment is on. Between expt on and off: derfraw300 is down only -0.062% (which is better than when expts were run previously). Change-Id: I55285a049a0c22810bdb42914212ab5a4f8521b5
2013-05-17 15:40:25 +02:00
vp9_cost_tokens_skip((int *)c[i][j][k][l], probs,
vp9_coef_tree);
}
Coding con-zero count rather than EOB for coeffs This patch revamps the entropy coding of coefficients to code first a non-zero count per coded block and correspondingly remove the EOB token from the token set. STATUS: Main encode/decode code achieving encode/decode sync - done. Forward and backward probability updates to the nzcs - done. Rd costing updates for nzcs - done. Note: The dynamic progrmaming apporach used in trellis quantization is not exactly compatible with nzcs. A suboptimal approach has been used instead where branch costs are updated to account for changes in the nzcs. TODO: Training the default probs/counts for nzcs Change-Id: I951bc1e22f47885077a7453a09b0493daa77883d
2013-02-20 19:16:24 +01:00
}
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#endif
Coding con-zero count rather than EOB for coeffs This patch revamps the entropy coding of coefficients to code first a non-zero count per coded block and correspondingly remove the EOB token from the token set. STATUS: Main encode/decode code achieving encode/decode sync - done. Forward and backward probability updates to the nzcs - done. Rd costing updates for nzcs - done. Note: The dynamic progrmaming apporach used in trellis quantization is not exactly compatible with nzcs. A suboptimal approach has been used instead where branch costs are updated to account for changes in the nzcs. TODO: Training the default probs/counts for nzcs Change-Id: I951bc1e22f47885077a7453a09b0493daa77883d
2013-02-20 19:16:24 +01:00
Adjust style to match Google Coding Style a little more closely. Most of these were picked up in the previous commit (prefix change from vp8_ to vp9_), but I'm pushing this separately so that it's easier to review. Change-Id: I91e959895778b8632d7d33375523df8a7568a490
2012-10-30 22:25:33 +01:00
static int rd_iifactor[32] = { 4, 4, 3, 2, 1, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, };
Initial WebM release
2010-05-18 17:58:33 +02:00
experiment extending the quantizer range Prior to this change, VP8 min quantizer is 4, which caps the highest quality around 51DB. This experimental change extends the min quantizer to 1, removes the cap and allows the highest quality to be around ~73DB, consistent with the fdct/idct round trip error. To test this change, at configure time use options: --enable-experimental --enable-extend_qrange The following is a brief log of changes in each of the patch sets patch set 1: In this commit, the quantization/dequantization constants are kept unchanged, instead scaling factor 4 is rolled into fdct/idct. Fixed Q0 encoding tests on mobile: Before: 9560.567kbps Overall PSNR:50.255DB VPXSSIM:98.288 Now: 18035.774kbps Overall PSNR:73.022DB VPXSSIM:99.991 patch set 2: regenerated dc/ac quantizer lookup tables based on the scaling factor rolled in the fdct/idct. Also slightly extended the range towards the high quantizer end. patch set 3: slightly tweaked the quantizer tables and generated bits_per_mb table based on Paul's suggestions. patch set 4: fix a typo in idct, re-calculated tables relating active max Q to active min Q patch set 5: added rdmult lookup table based on Q patch set 6: fix rdmult scale: dct coefficient has scaled up by 4 patch set 7: make transform coefficients to be within 16bits patch set 8: normalize 2nd order quantizers patch set 9: fix mis-spellings patch set 10: change the configure script and macros to allow experimental code to be enabled at configure time with --enable-extend_qrange patch set 11: rebase for merge Change-Id: Ib50641ddd44aba2a52ed890222c309faa31cc59c
2010-12-02 00:50:14 +01:00
// 3* dc_qlookup[Q]*dc_qlookup[Q];
Further resolution of QIndex LUTS; This commit resolves further QIndex look up tables to facilitate experimentation with the quantizer range. In some cases rather than remove the look up tables completely I have created functions that are called once to populate them using a formulaic approach base on the actual quantizer. The use of these functions based on best fit of data from the original tables does affect the results on some clips but across the derf test set the effect was broadly neutral. Change-Id: I8baa61c97ce87dc09a6340d56fdeb681b9345793
2011-11-23 12:32:20 +01:00
adjusted sad_per_bit to correlate with quantizer Re-calibrated sad_per_bit16 and sad_per_bit4 tables to linearly correlated to quantizer values, these two variables are used in motion search for costing motion vectors. This change has an small positive effect on compression. Change-Id: Ic9b5ea6fb8d5078ef663ba4899db019cc51f4166
2010-12-24 04:59:12 +01:00
/* values are now correlated to quantizer */
Removal of Qindex LUTS. One of the problems arising when tweaking or adjusting the quantizer tables is that there are a lot of look up tables that depend on the QINDEX. Any adjustment to the link between QINDEX and real quantizer therefore tends to break aspects of for example the rate control. In this check in I have replaced several of the look up tables with functions that approximate the same results as the old Q luts but use a formulaic approach based on real Q values rather than QIndex. This should hopefully make it easier to experiment with changes to the Q tables without always having to go through and hand optimize a set of look up tables. Once things stabilize we may choose to re-instate luts for the sake of performance. Patch 2: Addressed Ronald's comments. vp8_init_me_luts() Added so luts only initialized once. Change-Id: Ic80db2212d2fd01e08e8cb5c7dca1fda1102be57
2011-11-21 16:45:10 +01:00
static int sad_per_bit16lut[QINDEX_RANGE];
static int sad_per_bit4lut[QINDEX_RANGE];
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
void vp9_init_me_luts() {
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
int i;
// Initialize the sad lut tables using a formulaic calculation for now
// This is to make it easier to resolve the impact of experimental changes
// to the quantizer tables.
for (i = 0; i < QINDEX_RANGE; i++) {
sad_per_bit16lut[i] =
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
(int)((0.0418 * vp9_convert_qindex_to_q(i)) + 2.4107);
Removing MAXF macro and using MAX instead. Change-Id: I51c53692b1150005645bf362c5e5a8275178a8fd
2013-05-02 20:57:16 +02:00
sad_per_bit4lut[i] = (int)(0.063 * vp9_convert_qindex_to_q(i) + 2.742);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
}
Removal of Qindex LUTS. One of the problems arising when tweaking or adjusting the quantizer tables is that there are a lot of look up tables that depend on the QINDEX. Any adjustment to the link between QINDEX and real quantizer therefore tends to break aspects of for example the rate control. In this check in I have replaced several of the look up tables with functions that approximate the same results as the old Q luts but use a formulaic approach based on real Q values rather than QIndex. This should hopefully make it easier to experiment with changes to the Q tables without always having to go through and hand optimize a set of look up tables. Once things stabilize we may choose to re-instate luts for the sake of performance. Patch 2: Addressed Ronald's comments. vp8_init_me_luts() Added so luts only initialized once. Change-Id: Ic80db2212d2fd01e08e8cb5c7dca1fda1102be57
2011-11-21 16:45:10 +01:00
}
Initial WebM release
2010-05-18 17:58:33 +02:00
rdopt: make locally used symbols static. Change-Id: Id33c448084ca5c1c0525a05c3c4f35d6b0805fbe
2012-10-30 02:04:33 +01:00
static int compute_rd_mult(int qindex) {
Consistent names for quant-related functions and variables. Change-Id: I3a6d601e90e8740b9c26dd0afbfe9d467b75d367
2013-04-26 21:30:20 +02:00
const int q = vp9_dc_quant(qindex, 0);
Changes to rd error_per_bit calculation. Specifically changes to retain more precision especially at low Q through to the point of use. Change-Id: Ief5f010f2ca4daaabef49520e7edb46c35daf397
2013-03-07 16:02:57 +01:00
return (11 * q * q) >> 2;
Further resolution of QIndex LUTS; This commit resolves further QIndex look up tables to facilitate experimentation with the quantizer range. In some cases rather than remove the look up tables completely I have created functions that are called once to populate them using a formulaic approach base on the actual quantizer. The use of these functions based on best fit of data from the original tables does affect the results on some clips but across the derf test set the effect was broadly neutral. Change-Id: I8baa61c97ce87dc09a6340d56fdeb681b9345793
2011-11-23 12:32:20 +01:00
}
Code cleanup. Removing redundant 'extern' keyword, lowercase variable names. Change-Id: I608e8d8579aba8981f5fac3493f77b4481b13808
2013-03-02 02:39:31 +01:00
void vp9_initialize_me_consts(VP9_COMP *cpi, int qindex) {
cpi->mb.sadperbit16 = sad_per_bit16lut[qindex];
cpi->mb.sadperbit4 = sad_per_bit4lut[qindex];
Initial WebM release
2010-05-18 17:58:33 +02:00
}
experiment extending the quantizer range Prior to this change, VP8 min quantizer is 4, which caps the highest quality around 51DB. This experimental change extends the min quantizer to 1, removes the cap and allows the highest quality to be around ~73DB, consistent with the fdct/idct round trip error. To test this change, at configure time use options: --enable-experimental --enable-extend_qrange The following is a brief log of changes in each of the patch sets patch set 1: In this commit, the quantization/dequantization constants are kept unchanged, instead scaling factor 4 is rolled into fdct/idct. Fixed Q0 encoding tests on mobile: Before: 9560.567kbps Overall PSNR:50.255DB VPXSSIM:98.288 Now: 18035.774kbps Overall PSNR:73.022DB VPXSSIM:99.991 patch set 2: regenerated dc/ac quantizer lookup tables based on the scaling factor rolled in the fdct/idct. Also slightly extended the range towards the high quantizer end. patch set 3: slightly tweaked the quantizer tables and generated bits_per_mb table based on Paul's suggestions. patch set 4: fix a typo in idct, re-calculated tables relating active max Q to active min Q patch set 5: added rdmult lookup table based on Q patch set 6: fix rdmult scale: dct coefficient has scaled up by 4 patch set 7: make transform coefficients to be within 16bits patch set 8: normalize 2nd order quantizers patch set 9: fix mis-spellings patch set 10: change the configure script and macros to allow experimental code to be enabled at configure time with --enable-extend_qrange patch set 11: rebase for merge Change-Id: Ib50641ddd44aba2a52ed890222c309faa31cc59c
2010-12-02 00:50:14 +01:00
Code cleanup. Removing redundant 'extern' keyword, lowercase variable names. Change-Id: I608e8d8579aba8981f5fac3493f77b4481b13808
2013-03-02 02:39:31 +01:00
void vp9_initialize_rd_consts(VP9_COMP *cpi, int qindex) {
Combine vp8_block_error_c and vp8_submb_error_c. Change-Id: I58792bac6c7e5a524a6dcdb5241ef18bcc0c3a45
2012-08-07 00:03:04 +02:00
int q, i;
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove vp8 in local symbols. For non-static functions, change the prefix to vp9_. For static functions, remove the prefix. Also fix some comments, remove unused code or unused function prototypes. Change-Id: I1f8be05362f66060fe421c3d4c9a906fdf835de5
2012-10-31 22:40:53 +01:00
vp9_clear_system_state(); // __asm emms;
Initial WebM release
2010-05-18 17:58:33 +02:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
// Further tests required to see if optimum is different
// for key frames, golden frames and arf frames.
// if (cpi->common.refresh_golden_frame ||
// cpi->common.refresh_alt_ref_frame)
Removing MAXF macro and using MAX instead. Change-Id: I51c53692b1150005645bf362c5e5a8275178a8fd
2013-05-02 20:57:16 +02:00
qindex = clamp(qindex, 0, MAXQ);
Further resolution of QIndex LUTS; This commit resolves further QIndex look up tables to facilitate experimentation with the quantizer range. In some cases rather than remove the look up tables completely I have created functions that are called once to populate them using a formulaic approach base on the actual quantizer. The use of these functions based on best fit of data from the original tables does affect the results on some clips but across the derf test set the effect was broadly neutral. Change-Id: I8baa61c97ce87dc09a6340d56fdeb681b9345793
2011-11-23 12:32:20 +01:00
Code cleanup. Removing redundant 'extern' keyword, lowercase variable names. Change-Id: I608e8d8579aba8981f5fac3493f77b4481b13808
2013-03-02 02:39:31 +01:00
cpi->RDMULT = compute_rd_mult(qindex);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
if (cpi->pass == 2 && (cpi->common.frame_type != KEY_FRAME)) {
if (cpi->twopass.next_iiratio > 31)
cpi->RDMULT += (cpi->RDMULT * rd_iifactor[31]) >> 4;
else
cpi->RDMULT +=
Code cleanup. Removing redundant 'extern' keyword, lowercase variable names. Change-Id: I608e8d8579aba8981f5fac3493f77b4481b13808
2013-03-02 02:39:31 +01:00
(cpi->RDMULT * rd_iifactor[cpi->twopass.next_iiratio]) >> 4;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
}
Changes to rd error_per_bit calculation. Specifically changes to retain more precision especially at low Q through to the point of use. Change-Id: Ief5f010f2ca4daaabef49520e7edb46c35daf397
2013-03-07 16:02:57 +01:00
cpi->mb.errorperbit = cpi->RDMULT >> 6;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
cpi->mb.errorperbit += (cpi->mb.errorperbit == 0);
cap the best quantizer for 2nd order DC This commit also removes artificial RDMULT cap for low quantizers. The intention is to address some abnormal behavior of mode selections at the low quantizer end, where many macroblocks were coded with SPLITMV with all partitions using same motion vector including (0,0). This change improves the compression quality substantially for high quality encodings in both PSNR and SSIM terms. Overall effect on mid/low rate range is also positive for all metrics, but smaller in magnitude. Change-Id: I864b29c4bd9ff610d2545fa94a19cc7e80c02667
2011-01-26 07:24:22 +01:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
vp9_set_speed_features(cpi);
Initial WebM release
2010-05-18 17:58:33 +02:00
Code cleanup. Removing redundant 'extern' keyword, lowercase variable names. Change-Id: I608e8d8579aba8981f5fac3493f77b4481b13808
2013-03-02 02:39:31 +01:00
q = (int)pow(vp9_dc_quant(qindex, 0) >> 2, 1.25);
q <<= 2;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
if (q < 8)
q = 8;
experiment extending the quantizer range Prior to this change, VP8 min quantizer is 4, which caps the highest quality around 51DB. This experimental change extends the min quantizer to 1, removes the cap and allows the highest quality to be around ~73DB, consistent with the fdct/idct round trip error. To test this change, at configure time use options: --enable-experimental --enable-extend_qrange The following is a brief log of changes in each of the patch sets patch set 1: In this commit, the quantization/dequantization constants are kept unchanged, instead scaling factor 4 is rolled into fdct/idct. Fixed Q0 encoding tests on mobile: Before: 9560.567kbps Overall PSNR:50.255DB VPXSSIM:98.288 Now: 18035.774kbps Overall PSNR:73.022DB VPXSSIM:99.991 patch set 2: regenerated dc/ac quantizer lookup tables based on the scaling factor rolled in the fdct/idct. Also slightly extended the range towards the high quantizer end. patch set 3: slightly tweaked the quantizer tables and generated bits_per_mb table based on Paul's suggestions. patch set 4: fix a typo in idct, re-calculated tables relating active max Q to active min Q patch set 5: added rdmult lookup table based on Q patch set 6: fix rdmult scale: dct coefficient has scaled up by 4 patch set 7: make transform coefficients to be within 16bits patch set 8: normalize 2nd order quantizers patch set 9: fix mis-spellings patch set 10: change the configure script and macros to allow experimental code to be enabled at configure time with --enable-extend_qrange patch set 11: rebase for merge Change-Id: Ib50641ddd44aba2a52ed890222c309faa31cc59c
2010-12-02 00:50:14 +01:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
if (cpi->RDMULT > 1000) {
cpi->RDDIV = 1;
cpi->RDMULT /= 100;
Initial WebM release
2010-05-18 17:58:33 +02:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
for (i = 0; i < MAX_MODES; i++) {
if (cpi->sf.thresh_mult[i] < INT_MAX) {
cpi->rd_threshes[i] = cpi->sf.thresh_mult[i] * q / 100;
} else {
cpi->rd_threshes[i] = INT_MAX;
}
cpi->rd_baseline_thresh[i] = cpi->rd_threshes[i];
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
} else {
cpi->RDDIV = 100;
Initial WebM release
2010-05-18 17:58:33 +02:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
for (i = 0; i < MAX_MODES; i++) {
if (cpi->sf.thresh_mult[i] < (INT_MAX / q)) {
cpi->rd_threshes[i] = cpi->sf.thresh_mult[i] * q;
} else {
cpi->rd_threshes[i] = INT_MAX;
}
cpi->rd_baseline_thresh[i] = cpi->rd_threshes[i];
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
}
Initial WebM release
2010-05-18 17:58:33 +02:00
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#if CONFIG_BALANCED_COEFTREE
Introduce vp9_coeff_probs/counts/stats/accum types. Use these, instead of the 4/5-dimensional arrays, to hold statistics, counts, accumulations and probabilities for coefficient tokens. This commit also re-allows ENTROPY_STATS to compile. Change-Id: If441ffac936f52a3af91d8f2922ea8a0ceabdaa5
2012-12-08 01:09:59 +01:00
fill_token_costs(cpi->mb.token_costs[TX_4X4],
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
cpi->mb.token_costs_noskip[TX_4X4],
Framework changes in nzc to allow more flexibility The patch adds the flexibility to use standard EOB based coding on smaller block sizes and nzc based coding on larger blocksizes. The tx-sizes that use nzc based coding and those that use EOB based coding are controlled by a function get_nzc_used(). By default, this function uses nzc based coding for 16x16 and 32x32 transform blocks, which seem to bridge the performance gap substantially. All sets are now lower by 0.5% to 0.7%, as opposed to ~1.8% before. Change-Id: I06abed3df57b52d241ea1f51b0d571c71e38fd0b
2013-03-26 23:23:30 +01:00
cpi->common.fc.coef_probs_4x4, TX_4X4);
Introduce vp9_coeff_probs/counts/stats/accum types. Use these, instead of the 4/5-dimensional arrays, to hold statistics, counts, accumulations and probabilities for coefficient tokens. This commit also re-allows ENTROPY_STATS to compile. Change-Id: If441ffac936f52a3af91d8f2922ea8a0ceabdaa5
2012-12-08 01:09:59 +01:00
fill_token_costs(cpi->mb.token_costs[TX_8X8],
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
cpi->mb.token_costs_noskip[TX_8X8],
Framework changes in nzc to allow more flexibility The patch adds the flexibility to use standard EOB based coding on smaller block sizes and nzc based coding on larger blocksizes. The tx-sizes that use nzc based coding and those that use EOB based coding are controlled by a function get_nzc_used(). By default, this function uses nzc based coding for 16x16 and 32x32 transform blocks, which seem to bridge the performance gap substantially. All sets are now lower by 0.5% to 0.7%, as opposed to ~1.8% before. Change-Id: I06abed3df57b52d241ea1f51b0d571c71e38fd0b
2013-03-26 23:23:30 +01:00
cpi->common.fc.coef_probs_8x8, TX_8X8);
Introduce vp9_coeff_probs/counts/stats/accum types. Use these, instead of the 4/5-dimensional arrays, to hold statistics, counts, accumulations and probabilities for coefficient tokens. This commit also re-allows ENTROPY_STATS to compile. Change-Id: If441ffac936f52a3af91d8f2922ea8a0ceabdaa5
2012-12-08 01:09:59 +01:00
fill_token_costs(cpi->mb.token_costs[TX_16X16],
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
cpi->mb.token_costs_noskip[TX_16X16],
Framework changes in nzc to allow more flexibility The patch adds the flexibility to use standard EOB based coding on smaller block sizes and nzc based coding on larger blocksizes. The tx-sizes that use nzc based coding and those that use EOB based coding are controlled by a function get_nzc_used(). By default, this function uses nzc based coding for 16x16 and 32x32 transform blocks, which seem to bridge the performance gap substantially. All sets are now lower by 0.5% to 0.7%, as opposed to ~1.8% before. Change-Id: I06abed3df57b52d241ea1f51b0d571c71e38fd0b
2013-03-26 23:23:30 +01:00
cpi->common.fc.coef_probs_16x16, TX_16X16);
Introduce vp9_coeff_probs/counts/stats/accum types. Use these, instead of the 4/5-dimensional arrays, to hold statistics, counts, accumulations and probabilities for coefficient tokens. This commit also re-allows ENTROPY_STATS to compile. Change-Id: If441ffac936f52a3af91d8f2922ea8a0ceabdaa5
2012-12-08 01:09:59 +01:00
fill_token_costs(cpi->mb.token_costs[TX_32X32],
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
cpi->mb.token_costs_noskip[TX_32X32],
Framework changes in nzc to allow more flexibility The patch adds the flexibility to use standard EOB based coding on smaller block sizes and nzc based coding on larger blocksizes. The tx-sizes that use nzc based coding and those that use EOB based coding are controlled by a function get_nzc_used(). By default, this function uses nzc based coding for 16x16 and 32x32 transform blocks, which seem to bridge the performance gap substantially. All sets are now lower by 0.5% to 0.7%, as opposed to ~1.8% before. Change-Id: I06abed3df57b52d241ea1f51b0d571c71e38fd0b
2013-03-26 23:23:30 +01:00
cpi->common.fc.coef_probs_32x32, TX_32X32);
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#else
fill_token_costs(cpi->mb.token_costs[TX_4X4],
cpi->common.fc.coef_probs_4x4, TX_4X4);
fill_token_costs(cpi->mb.token_costs[TX_8X8],
cpi->common.fc.coef_probs_8x8, TX_8X8);
fill_token_costs(cpi->mb.token_costs[TX_16X16],
cpi->common.fc.coef_probs_16x16, TX_16X16);
fill_token_costs(cpi->mb.token_costs[TX_32X32],
cpi->common.fc.coef_probs_32x32, TX_32X32);
#endif
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
Contextual entropy coding of partition syntax This commit enables selecting probability models for recursive block partition information syntax, depending on its above/left partition information, as well as the current block size. These conditional probability models are reasonably stationary and consistent across frames, hence the backward adaptive approach is used to maintain and update the contextual models. It achieves coding performance gains (on top of enabling rectangular block sizes): derf: 0.242% yt: 0.391% hd: 0.376% stdhd: 0.645% Change-Id: Ie513d9673337f0d27abd65fb566b711d0844ec2e
2013-04-23 19:12:18 +02:00
for (i = 0; i < NUM_PARTITION_CONTEXTS; i++)
Fairly basic integration of rectangular blocks in encoding RD loop. Adds RD integration for 32x16, 16x32, 64x32 and 32x64 rectangular blocks. Derf almost +0.6%, HD a little over +1.0%, STDHD +1.3%. Change-Id: Id651fdb6a655fdbb5c47009757e63317acfb88a5
2013-04-17 18:25:06 +02:00
vp9_cost_tokens(cpi->mb.partition_cost[i],
cpi->common.fc.partition_prob[i],
vp9_partition_tree);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
/*rough estimate for costing*/
cpi->common.kf_ymode_probs_index = cpi->common.base_qindex >> 4;
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
vp9_init_mode_costs(cpi);
Initial WebM release
2010-05-18 17:58:33 +02:00
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
if (cpi->common.frame_type != KEY_FRAME) {
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
vp9_build_nmv_cost_table(
New motion vector entropy coding Adds a new experiment with redesigned/refactored motion vector entropy coding. The patch also takes a first step towards separating the integer and fractional pel components of a MV. However the fractional pel encoding still depends on the integer pel part and so they are not fully independent. Further experiments are in progress to see how much they can be decoupled without affecting performance. All components including entropy coding/decoding, costing for MV search, forward updates and backward updates to probability tables, have been implemented. Results so far: derf: +0.19% std-hd: +0.28% yt: +0.80% hd: +1.15% Patch: Simplifies the fractional pel models: derf: +0.284% std-hd: +0.289% yt: +0.849% hd: +1.254% Patch: Some changes in the models, rebased. derf: +0.330% std-hd: +0.306% yt: +0.816% hd: +1.225% Change-Id: I646b3c48f3587f4cc909639b78c3798da6402678
2012-07-26 22:42:07 +02:00
cpi->mb.nmvjointcost,
cpi->mb.e_mbd.allow_high_precision_mv ?
cpi->mb.nmvcost_hp : cpi->mb.nmvcost,
&cpi->common.fc.nmvc,
cpi->mb.e_mbd.allow_high_precision_mv, 1, 1);
}
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Use standard integer types for pixel values and coefficients. For coefficients, use int16_t (instead of short); for pixel values in 16-bit intermediates, use uint16_t (instead of unsigned short); for all others, use uint8_t (instead of unsigned char). Change-Id: I3619cd9abf106c3742eccc2e2f5e89a62774f7da
2012-12-19 00:31:19 +01:00
int vp9_block_error_c(int16_t *coeff, int16_t *dqcoeff, int block_size) {
Combine vp8_block_error_c and vp8_submb_error_c. Change-Id: I58792bac6c7e5a524a6dcdb5241ef18bcc0c3a45
2012-08-07 00:03:04 +02:00
int i, error = 0;
Initial WebM release
2010-05-18 17:58:33 +02:00
Combine vp8_block_error_c and vp8_submb_error_c. Change-Id: I58792bac6c7e5a524a6dcdb5241ef18bcc0c3a45
2012-08-07 00:03:04 +02:00
for (i = 0; i < block_size; i++) {
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
int this_diff = coeff[i] - dqcoeff[i];
error += this_diff * this_diff;
}
Initial WebM release
2010-05-18 17:58:33 +02:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
return error;
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Coding con-zero count rather than EOB for coeffs This patch revamps the entropy coding of coefficients to code first a non-zero count per coded block and correspondingly remove the EOB token from the token set. STATUS: Main encode/decode code achieving encode/decode sync - done. Forward and backward probability updates to the nzcs - done. Rd costing updates for nzcs - done. Note: The dynamic progrmaming apporach used in trellis quantization is not exactly compatible with nzcs. A suboptimal approach has been used instead where branch costs are updated to account for changes in the nzcs. TODO: Training the default probs/counts for nzcs Change-Id: I951bc1e22f47885077a7453a09b0493daa77883d
2013-02-20 19:16:24 +01:00
static INLINE int cost_coeffs(VP9_COMMON *const cm, MACROBLOCK *mb,
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
int plane, int block, PLANE_TYPE type,
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
ENTROPY_CONTEXT *A,
ENTROPY_CONTEXT *L,
Move EOB to per-plane data Continue migrating data from BLOCKD/MACROBLOCKD to the per-plane structures. Change-Id: Ibbfa68d6da438d32dcbe8df68245ee28b0a2fa2c
2013-04-04 21:03:27 +02:00
TX_SIZE tx_size,
int y_blocks) {
Move eob from BLOCKD to MACROBLOCKD. Consistent with VP8. Change-Id: I8c316ee49f072e15abbb033a80e9c36617891f07
2013-02-27 19:00:24 +01:00
MACROBLOCKD *const xd = &mb->e_mbd;
Make superblocks independent of macroblock code and data. Split macroblock and superblock tokenization and detokenization functions and coefficient-related data structs so that the bitstream layout and related code of superblock coefficients looks less like it's a hack to fit macroblocks in superblocks. In addition, unify chroma transform size selection from luma transform size (i.e. always use the same size, as long as it fits the predictor); in practice, this means 32x32 and 64x64 superblocks using the 16x16 luma transform will now use the 16x16 (instead of the 8x8) chroma transform, and 64x64 superblocks using the 32x32 luma transform will now use the 32x32 (instead of the 16x16) chroma transform. Lastly, add a trellis optimize function for 32x32 transform blocks. HD gains about 0.3%, STDHD about 0.15% and derf about 0.1%. There's a few negative points here and there that I might want to analyze a little closer. Change-Id: Ibad7c3ddfe1acfc52771dfc27c03e9783e054430
2013-03-04 23:12:17 +01:00
MB_MODE_INFO *mbmi = &xd->mode_info_context->mbmi;
int pt;
Remove some Y2-related code. Change-Id: I4f46d142c2a8d1e8a880cfac63702dcbfb999b78
2013-02-15 19:15:42 +01:00
int c = 0;
Use above/left (instead of previous in scan-order) as token context. Pearson correlation for above or left is significantly higher than for previous-in-scan-order (absolute values depend on position in scan, but in general, we gain about 0.1-0.2 by using either above or left; using both basically just makes this even better). For eob branch skipping, we continue to use the previous token in scan order. This helps about 0.9% on derf after re-training on a limited data set. Full re-training and results on larger-resolution clips are pending. Note that this commit breaks trellis, so we can probably get further gains out of it by fixing trellis at some later point. Change-Id: Iead68e296fc3a105cca746b5e3da9555d6010cfe
2013-03-27 00:46:09 +01:00
int cost = 0, pad;
const int *scan, *nb;
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
const int eob = xd->plane[plane].eobs[block];
const int16_t *qcoeff_ptr = BLOCK_OFFSET(xd->plane[plane].qcoeff,
block, 16);
Make superblocks independent of macroblock code and data. Split macroblock and superblock tokenization and detokenization functions and coefficient-related data structs so that the bitstream layout and related code of superblock coefficients looks less like it's a hack to fit macroblocks in superblocks. In addition, unify chroma transform size selection from luma transform size (i.e. always use the same size, as long as it fits the predictor); in practice, this means 32x32 and 64x64 superblocks using the 16x16 luma transform will now use the 16x16 (instead of the 8x8) chroma transform, and 64x64 superblocks using the 32x32 luma transform will now use the 32x32 (instead of the 16x16) chroma transform. Lastly, add a trellis optimize function for 32x32 transform blocks. HD gains about 0.3%, STDHD about 0.15% and derf about 0.1%. There's a few negative points here and there that I might want to analyze a little closer. Change-Id: Ibad7c3ddfe1acfc52771dfc27c03e9783e054430
2013-03-04 23:12:17 +01:00
const int ref = mbmi->ref_frame != INTRA_FRAME;
Make cost_coeffs() more efficient. Cache the constant offset in one variable to prevent re-loading that in each loop iteration, and mark the function as inline so we can use the fact that the transform size is always known in the caller. Almost 1% faster encoding overall. Change-Id: Id78325a60b025057d8f4ecd9003a74086ccbf85a
2013-02-07 02:16:36 +01:00
unsigned int (*token_costs)[PREV_COEF_CONTEXTS][MAX_ENTROPY_TOKENS] =
Split coefficient token tables intra vs. inter. Change-Id: I5416455f8f129ca0f450d00e48358d2012605072
2013-02-19 22:36:38 +01:00
mb->token_costs[tx_size][type][ref];
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
ENTROPY_CONTEXT above_ec, left_ec;
End of orientation zero group experiment Adds an experiment that codes an end-of-orientation symbol for every eligible zero encountered in scan order. This cleans out various other sub-experiments that were part of the origiinal patch, which will be later included if found useful. Results are slightly positive on all sets (0.1 - 0.2% range). Change-Id: I57765c605fefc7fb9d1b57f1b356843602abefaf
2013-03-28 18:42:23 +01:00
TX_TYPE tx_type = DCT_DCT;
Change the RD cost functions to take into account the seg_eob. Change-Id: I2f96e5b205f70fdb559f01637c4b5730246c38a8
2012-08-07 03:29:59 +02:00
Coding con-zero count rather than EOB for coeffs This patch revamps the entropy coding of coefficients to code first a non-zero count per coded block and correspondingly remove the EOB token from the token set. STATUS: Main encode/decode code achieving encode/decode sync - done. Forward and backward probability updates to the nzcs - done. Rd costing updates for nzcs - done. Note: The dynamic progrmaming apporach used in trellis quantization is not exactly compatible with nzcs. A suboptimal approach has been used instead where branch costs are updated to account for changes in the nzcs. TODO: Training the default probs/counts for nzcs Change-Id: I951bc1e22f47885077a7453a09b0493daa77883d
2013-02-20 19:16:24 +01:00
const int segment_id = xd->mode_info_context->mbmi.segment_id;
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#if CONFIG_BALANCED_COEFTREE
unsigned int (*token_costs_noskip)[PREV_COEF_CONTEXTS][MAX_ENTROPY_TOKENS] =
mb->token_costs_noskip[tx_size][type][ref];
#else
Updating the model coef experiment Cleans up the experiment. Actually uses reduced counts for backward updates, and reduced number of probabilities in the context. No change in bitstream when the experiment is on. Between expt on and off: derfraw300 is down only -0.062% (which is better than when expts were run previously). Change-Id: I55285a049a0c22810bdb42914212ab5a4f8521b5
2013-05-17 15:40:25 +02:00
vp9_prob coef_probs[COEF_BANDS][PREV_COEF_CONTEXTS][ENTROPY_NODES];
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#endif
Use above/left (instead of previous in scan-order) as token context. Pearson correlation for above or left is significantly higher than for previous-in-scan-order (absolute values depend on position in scan, but in general, we gain about 0.1-0.2 by using either above or left; using both basically just makes this even better). For eob branch skipping, we continue to use the previous token in scan order. This helps about 0.9% on derf after re-training on a limited data set. Full re-training and results on larger-resolution clips are pending. Note that this commit breaks trellis, so we can probably get further gains out of it by fixing trellis at some later point. Change-Id: Iead68e296fc3a105cca746b5e3da9555d6010cfe
2013-03-27 00:46:09 +01:00
int seg_eob, default_eob;
uint8_t token_cache[1024];
Change to band calculation. Change band calculation back to simpler model based on the order in which coefficients are coded in scan order not the absolute coefficient positions. With the scatter scan experiment enabled the results were appear broadly neutral on derf (-0.028) but up a little on std-hd +0.134). Without the scatterscan experiment on the results were up derf as well. Change-Id: Ie9ef03ce42a6b24b849a4bebe950d4a5dffa6791
2013-05-09 19:47:58 +02:00
const uint8_t * band_translate;
Coding con-zero count rather than EOB for coeffs This patch revamps the entropy coding of coefficients to code first a non-zero count per coded block and correspondingly remove the EOB token from the token set. STATUS: Main encode/decode code achieving encode/decode sync - done. Forward and backward probability updates to the nzcs - done. Rd costing updates for nzcs - done. Note: The dynamic progrmaming apporach used in trellis quantization is not exactly compatible with nzcs. A suboptimal approach has been used instead where branch costs are updated to account for changes in the nzcs. TODO: Training the default probs/counts for nzcs Change-Id: I951bc1e22f47885077a7453a09b0493daa77883d
2013-02-20 19:16:24 +01:00
// Check for consistency of tx_size with mode info
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
assert((!type && !plane) || (type && plane));
Coding con-zero count rather than EOB for coeffs This patch revamps the entropy coding of coefficients to code first a non-zero count per coded block and correspondingly remove the EOB token from the token set. STATUS: Main encode/decode code achieving encode/decode sync - done. Forward and backward probability updates to the nzcs - done. Rd costing updates for nzcs - done. Note: The dynamic progrmaming apporach used in trellis quantization is not exactly compatible with nzcs. A suboptimal approach has been used instead where branch costs are updated to account for changes in the nzcs. TODO: Training the default probs/counts for nzcs Change-Id: I951bc1e22f47885077a7453a09b0493daa77883d
2013-02-20 19:16:24 +01:00
if (type == PLANE_TYPE_Y_WITH_DC) {
assert(xd->mode_info_context->mbmi.txfm_size == tx_size);
} else {
TX_SIZE tx_size_uv = get_uv_tx_size(xd);
assert(tx_size == tx_size_uv);
}
Entropy coding for hybrid transform Separates the entropy coding context models for 4x4, 8x8 and 16x16 ADST variants. There is a small improvement for HD (hd/std-hd) by about 0.1-0.2%. Results on derf/yt are about the same, probably because there is not enough statistics. Results may improve somewhat once the initial probability tables are updated for the hybrid transforms which is coming soon. Change-Id: Ic7c0c62dacc68ef551054fdb575be8b8507d32a8
2012-09-10 07:42:35 +02:00
switch (tx_size) {
Re-add support for ADST in superblocks. This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-06 00:18:06 +01:00
case TX_4X4: {
End of orientation zero group experiment Adds an experiment that codes an end-of-orientation symbol for every eligible zero encountered in scan order. This cleans out various other sub-experiments that were part of the origiinal patch, which will be later included if found useful. Results are slightly positive on all sets (0.1 - 0.2% range). Change-Id: I57765c605fefc7fb9d1b57f1b356843602abefaf
2013-03-28 18:42:23 +01:00
tx_type = (type == PLANE_TYPE_Y_WITH_DC) ?
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
get_tx_type_4x4(xd, block) : DCT_DCT;
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
above_ec = A[0] != 0;
left_ec = L[0] != 0;
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#if !CONFIG_BALANCED_COEFTREE
Updating the model coef experiment Cleans up the experiment. Actually uses reduced counts for backward updates, and reduced number of probabilities in the context. No change in bitstream when the experiment is on. Between expt on and off: derfraw300 is down only -0.062% (which is better than when expts were run previously). Change-Id: I55285a049a0c22810bdb42914212ab5a4f8521b5
2013-05-17 15:40:25 +02:00
vp9_model_to_full_probs_sb(cm->fc.coef_probs_4x4[type][ref],
Using 128 entry look up table for coef models Reverts to using 128 bit LUT for the coef models rather than 48 to ease hardware implementation. Also incorporates some cleanups including removing various hooks to support different lookup tables based on block_type and ref_type. Change-Id: I54100c120cca07a2ebd3a7776bc4630fa6a153f6
2013-05-22 16:28:29 +02:00
coef_probs);
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#endif
Use above/left (instead of previous in scan-order) as token context. Pearson correlation for above or left is significantly higher than for previous-in-scan-order (absolute values depend on position in scan, but in general, we gain about 0.1-0.2 by using either above or left; using both basically just makes this even better). For eob branch skipping, we continue to use the previous token in scan order. This helps about 0.9% on derf after re-training on a limited data set. Full re-training and results on larger-resolution clips are pending. Note that this commit breaks trellis, so we can probably get further gains out of it by fixing trellis at some later point. Change-Id: Iead68e296fc3a105cca746b5e3da9555d6010cfe
2013-03-27 00:46:09 +01:00
seg_eob = 16;
Adding get_scan_{4x4, 8x8, 16x16} functions. Change-Id: Id4306ef6d65d4a3984aed50b775bdf48d4f6c438
2013-04-22 23:08:41 +02:00
scan = get_scan_4x4(tx_type);
Change to band calculation. Change band calculation back to simpler model based on the order in which coefficients are coded in scan order not the absolute coefficient positions. With the scatter scan experiment enabled the results were appear broadly neutral on derf (-0.028) but up a little on std-hd +0.134). Without the scatterscan experiment on the results were up derf as well. Change-Id: Ie9ef03ce42a6b24b849a4bebe950d4a5dffa6791
2013-05-09 19:47:58 +02:00
band_translate = vp9_coefband_trans_4x4;
Combine cost_coeffs functions. Change-Id: I3421d07fdcf2148d75c8dbfc8c44bc2d96a3a8f6
2012-08-06 21:15:24 +02:00
break;
Re-add support for ADST in superblocks. This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-06 00:18:06 +01:00
}
Add col/row-based coefficient scanning patterns for 1D 8x8/16x16 ADSTs. These are mostly just for experimental purposes. I saw small gains (in the 0.1% range) when playing with this on derf. Change-Id: Ib21eed477bbb46bddcd73b21c5c708a5b46abedc
2013-03-25 20:30:00 +01:00
case TX_8X8: {
const BLOCK_SIZE_TYPE sb_type = xd->mode_info_context->mbmi.sb_type;
Use b_width/height_log2 instead of mb_ where appropriate. Basic assumption: when talking about transform units, use b_; when talking about macroblock indices, use mb_. Change-Id: Ifd163f595d4924ff892de4eb0401ccd56dc81884
2013-04-24 18:18:16 +02:00
const int sz = 1 + b_width_log2(sb_type);
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
const int x = block & ((1 << sz) - 1), y = block - x;
End of orientation zero group experiment Adds an experiment that codes an end-of-orientation symbol for every eligible zero encountered in scan order. This cleans out various other sub-experiments that were part of the origiinal patch, which will be later included if found useful. Results are slightly positive on all sets (0.1 - 0.2% range). Change-Id: I57765c605fefc7fb9d1b57f1b356843602abefaf
2013-03-28 18:42:23 +01:00
TX_TYPE tx_type = (type == PLANE_TYPE_Y_WITH_DC) ?
get_tx_type_8x8(xd, y + (x >> 1)) : DCT_DCT;
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
above_ec = (A[0] + A[1]) != 0;
left_ec = (L[0] + L[1]) != 0;
Adding get_scan_{4x4, 8x8, 16x16} functions. Change-Id: Id4306ef6d65d4a3984aed50b775bdf48d4f6c438
2013-04-22 23:08:41 +02:00
scan = get_scan_8x8(tx_type);
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#if !CONFIG_BALANCED_COEFTREE
Updating the model coef experiment Cleans up the experiment. Actually uses reduced counts for backward updates, and reduced number of probabilities in the context. No change in bitstream when the experiment is on. Between expt on and off: derfraw300 is down only -0.062% (which is better than when expts were run previously). Change-Id: I55285a049a0c22810bdb42914212ab5a4f8521b5
2013-05-17 15:40:25 +02:00
vp9_model_to_full_probs_sb(cm->fc.coef_probs_8x8[type][ref],
Using 128 entry look up table for coef models Reverts to using 128 bit LUT for the coef models rather than 48 to ease hardware implementation. Also incorporates some cleanups including removing various hooks to support different lookup tables based on block_type and ref_type. Change-Id: I54100c120cca07a2ebd3a7776bc4630fa6a153f6
2013-05-22 16:28:29 +02:00
coef_probs);
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#endif
Use above/left (instead of previous in scan-order) as token context. Pearson correlation for above or left is significantly higher than for previous-in-scan-order (absolute values depend on position in scan, but in general, we gain about 0.1-0.2 by using either above or left; using both basically just makes this even better). For eob branch skipping, we continue to use the previous token in scan order. This helps about 0.9% on derf after re-training on a limited data set. Full re-training and results on larger-resolution clips are pending. Note that this commit breaks trellis, so we can probably get further gains out of it by fixing trellis at some later point. Change-Id: Iead68e296fc3a105cca746b5e3da9555d6010cfe
2013-03-27 00:46:09 +01:00
seg_eob = 64;
Change to band calculation. Change band calculation back to simpler model based on the order in which coefficients are coded in scan order not the absolute coefficient positions. With the scatter scan experiment enabled the results were appear broadly neutral on derf (-0.028) but up a little on std-hd +0.134). Without the scatterscan experiment on the results were up derf as well. Change-Id: Ie9ef03ce42a6b24b849a4bebe950d4a5dffa6791
2013-05-09 19:47:58 +02:00
band_translate = vp9_coefband_trans_8x8plus;
Combine cost_coeffs functions. Change-Id: I3421d07fdcf2148d75c8dbfc8c44bc2d96a3a8f6
2012-08-06 21:15:24 +02:00
break;
Add col/row-based coefficient scanning patterns for 1D 8x8/16x16 ADSTs. These are mostly just for experimental purposes. I saw small gains (in the 0.1% range) when playing with this on derf. Change-Id: Ib21eed477bbb46bddcd73b21c5c708a5b46abedc
2013-03-25 20:30:00 +01:00
}
case TX_16X16: {
const BLOCK_SIZE_TYPE sb_type = xd->mode_info_context->mbmi.sb_type;
Use b_width/height_log2 instead of mb_ where appropriate. Basic assumption: when talking about transform units, use b_; when talking about macroblock indices, use mb_. Change-Id: Ifd163f595d4924ff892de4eb0401ccd56dc81884
2013-04-24 18:18:16 +02:00
const int sz = 2 + b_width_log2(sb_type);
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
const int x = block & ((1 << sz) - 1), y = block - x;
End of orientation zero group experiment Adds an experiment that codes an end-of-orientation symbol for every eligible zero encountered in scan order. This cleans out various other sub-experiments that were part of the origiinal patch, which will be later included if found useful. Results are slightly positive on all sets (0.1 - 0.2% range). Change-Id: I57765c605fefc7fb9d1b57f1b356843602abefaf
2013-03-28 18:42:23 +01:00
TX_TYPE tx_type = (type == PLANE_TYPE_Y_WITH_DC) ?
get_tx_type_16x16(xd, y + (x >> 2)) : DCT_DCT;
Adding get_scan_{4x4, 8x8, 16x16} functions. Change-Id: Id4306ef6d65d4a3984aed50b775bdf48d4f6c438
2013-04-22 23:08:41 +02:00
scan = get_scan_16x16(tx_type);
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#if !CONFIG_BALANCED_COEFTREE
Updating the model coef experiment Cleans up the experiment. Actually uses reduced counts for backward updates, and reduced number of probabilities in the context. No change in bitstream when the experiment is on. Between expt on and off: derfraw300 is down only -0.062% (which is better than when expts were run previously). Change-Id: I55285a049a0c22810bdb42914212ab5a4f8521b5
2013-05-17 15:40:25 +02:00
vp9_model_to_full_probs_sb(cm->fc.coef_probs_16x16[type][ref],
Using 128 entry look up table for coef models Reverts to using 128 bit LUT for the coef models rather than 48 to ease hardware implementation. Also incorporates some cleanups including removing various hooks to support different lookup tables based on block_type and ref_type. Change-Id: I54100c120cca07a2ebd3a7776bc4630fa6a153f6
2013-05-22 16:28:29 +02:00
coef_probs);
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#endif
Use above/left (instead of previous in scan-order) as token context. Pearson correlation for above or left is significantly higher than for previous-in-scan-order (absolute values depend on position in scan, but in general, we gain about 0.1-0.2 by using either above or left; using both basically just makes this even better). For eob branch skipping, we continue to use the previous token in scan order. This helps about 0.9% on derf after re-training on a limited data set. Full re-training and results on larger-resolution clips are pending. Note that this commit breaks trellis, so we can probably get further gains out of it by fixing trellis at some later point. Change-Id: Iead68e296fc3a105cca746b5e3da9555d6010cfe
2013-03-27 00:46:09 +01:00
seg_eob = 256;
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
above_ec = (A[0] + A[1] + A[2] + A[3]) != 0;
left_ec = (L[0] + L[1] + L[2] + L[3]) != 0;
Change to band calculation. Change band calculation back to simpler model based on the order in which coefficients are coded in scan order not the absolute coefficient positions. With the scatter scan experiment enabled the results were appear broadly neutral on derf (-0.028) but up a little on std-hd +0.134). Without the scatterscan experiment on the results were up derf as well. Change-Id: Ie9ef03ce42a6b24b849a4bebe950d4a5dffa6791
2013-05-09 19:47:58 +02:00
band_translate = vp9_coefband_trans_8x8plus;
Combine cost_coeffs functions. Change-Id: I3421d07fdcf2148d75c8dbfc8c44bc2d96a3a8f6
2012-08-06 21:15:24 +02:00
break;
Add col/row-based coefficient scanning patterns for 1D 8x8/16x16 ADSTs. These are mostly just for experimental purposes. I saw small gains (in the 0.1% range) when playing with this on derf. Change-Id: Ib21eed477bbb46bddcd73b21c5c708a5b46abedc
2013-03-25 20:30:00 +01:00
}
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
case TX_32X32:
Merge Scatter Scan experiment. Removal from under configure flag. A bit renaming Change-Id: I2213229dfe852001dfec16b149f47c52ce88f3aa
2013-05-22 15:09:55 +02:00
scan = vp9_default_scan_32x32;
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#if !CONFIG_BALANCED_COEFTREE
Updating the model coef experiment Cleans up the experiment. Actually uses reduced counts for backward updates, and reduced number of probabilities in the context. No change in bitstream when the experiment is on. Between expt on and off: derfraw300 is down only -0.062% (which is better than when expts were run previously). Change-Id: I55285a049a0c22810bdb42914212ab5a4f8521b5
2013-05-17 15:40:25 +02:00
vp9_model_to_full_probs_sb(cm->fc.coef_probs_32x32[type][ref],
Using 128 entry look up table for coef models Reverts to using 128 bit LUT for the coef models rather than 48 to ease hardware implementation. Also incorporates some cleanups including removing various hooks to support different lookup tables based on block_type and ref_type. Change-Id: I54100c120cca07a2ebd3a7776bc4630fa6a153f6
2013-05-22 16:28:29 +02:00
coef_probs);
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#endif
Use above/left (instead of previous in scan-order) as token context. Pearson correlation for above or left is significantly higher than for previous-in-scan-order (absolute values depend on position in scan, but in general, we gain about 0.1-0.2 by using either above or left; using both basically just makes this even better). For eob branch skipping, we continue to use the previous token in scan order. This helps about 0.9% on derf after re-training on a limited data set. Full re-training and results on larger-resolution clips are pending. Note that this commit breaks trellis, so we can probably get further gains out of it by fixing trellis at some later point. Change-Id: Iead68e296fc3a105cca746b5e3da9555d6010cfe
2013-03-27 00:46:09 +01:00
seg_eob = 1024;
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
above_ec = (A[0] + A[1] + A[2] + A[3] + A[4] + A[5] + A[6] + A[7]) != 0;
left_ec = (L[0] + L[1] + L[2] + L[3] + L[4] + L[5] + L[6] + L[7]) != 0;
Change to band calculation. Change band calculation back to simpler model based on the order in which coefficients are coded in scan order not the absolute coefficient positions. With the scatter scan experiment enabled the results were appear broadly neutral on derf (-0.028) but up a little on std-hd +0.134). Without the scatterscan experiment on the results were up derf as well. Change-Id: Ie9ef03ce42a6b24b849a4bebe950d4a5dffa6791
2013-05-09 19:47:58 +02:00
band_translate = vp9_coefband_trans_8x8plus;
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
break;
Combine cost_coeffs functions. Change-Id: I3421d07fdcf2148d75c8dbfc8c44bc2d96a3a8f6
2012-08-06 21:15:24 +02:00
default:
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
abort();
Combine cost_coeffs functions. Change-Id: I3421d07fdcf2148d75c8dbfc8c44bc2d96a3a8f6
2012-08-06 21:15:24 +02:00
break;
}
Move EOB to per-plane data Continue migrating data from BLOCKD/MACROBLOCKD to the per-plane structures. Change-Id: Ibbfa68d6da438d32dcbe8df68245ee28b0a2fa2c
2013-04-04 21:03:27 +02:00
assert(eob <= seg_eob);
More refactoring of RD cost to make later changes easier. Change-Id: Ia6a290a6a53fdb88c971bea2de47ac73dcb7a364
2012-08-06 20:22:28 +02:00
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
pt = combine_entropy_contexts(above_ec, left_ec);
Use above/left (instead of previous in scan-order) as token context. Pearson correlation for above or left is significantly higher than for previous-in-scan-order (absolute values depend on position in scan, but in general, we gain about 0.1-0.2 by using either above or left; using both basically just makes this even better). For eob branch skipping, we continue to use the previous token in scan order. This helps about 0.9% on derf after re-training on a limited data set. Full re-training and results on larger-resolution clips are pending. Note that this commit breaks trellis, so we can probably get further gains out of it by fixing trellis at some later point. Change-Id: Iead68e296fc3a105cca746b5e3da9555d6010cfe
2013-03-27 00:46:09 +01:00
nb = vp9_get_coef_neighbors_handle(scan, &pad);
default_eob = seg_eob;
make cost_coeffs() and tokenize_b() consistent Change-Id: I7cdb5c32a1400f88ec36d08ea982e38b77731602
2013-01-03 18:00:30 +01:00
Removes the code_nonzerocount experiment This patch does not seem to give any benefits. Change-Id: I9d2b4091d6af3dfc0875f24db86c01e2de57f8db
2013-04-22 19:58:49 +02:00
if (vp9_segfeature_active(xd, segment_id, SEG_LVL_SKIP))
seg_eob = 0;
make cost_coeffs() and tokenize_b() consistent Change-Id: I7cdb5c32a1400f88ec36d08ea982e38b77731602
2013-01-03 18:00:30 +01:00
Bugfix in encode_inter_mb_segment_8x8 Fixes an indexing bug. Looks like the bug has been there for a while. Change-Id: I9fc04b0c30754bcb47366ad94a08112925600c4d
2013-04-04 20:07:19 +02:00
/* sanity check to ensure that we do not have spurious non-zero q values */
if (eob < seg_eob)
assert(qcoeff_ptr[scan[eob]] == 0);
Split coefficient token tables intra vs. inter. Change-Id: I5416455f8f129ca0f450d00e48358d2012605072
2013-02-19 22:36:38 +01:00
{
End of orientation zero group experiment Adds an experiment that codes an end-of-orientation symbol for every eligible zero encountered in scan order. This cleans out various other sub-experiments that were part of the origiinal patch, which will be later included if found useful. Results are slightly positive on all sets (0.1 - 0.2% range). Change-Id: I57765c605fefc7fb9d1b57f1b356843602abefaf
2013-03-28 18:42:23 +01:00
for (c = 0; c < eob; c++) {
Entropy coding for hybrid transform Separates the entropy coding context models for 4x4, 8x8 and 16x16 ADST variants. There is a small improvement for HD (hd/std-hd) by about 0.1-0.2%. Results on derf/yt are about the same, probably because there is not enough statistics. Results may improve somewhat once the initial probability tables are updated for the hybrid transforms which is coming soon. Change-Id: Ic7c0c62dacc68ef551054fdb575be8b8507d32a8
2012-09-10 07:42:35 +02:00
int v = qcoeff_ptr[scan[c]];
Fixing member names inside TOKENVALUE and TOKENEXTRA structs. Change-Id: I183ec5819d4d80966c92db36db75b8c3be0d381d
2013-04-19 01:18:08 +02:00
int t = vp9_dct_value_tokens_ptr[v].token;
Change to band calculation. Change band calculation back to simpler model based on the order in which coefficients are coded in scan order not the absolute coefficient positions. With the scatter scan experiment enabled the results were appear broadly neutral on derf (-0.028) but up a little on std-hd +0.134). Without the scatterscan experiment on the results were up derf as well. Change-Id: Ie9ef03ce42a6b24b849a4bebe950d4a5dffa6791
2013-05-09 19:47:58 +02:00
int band = get_coef_band(band_translate, c);
End of orientation zero group experiment Adds an experiment that codes an end-of-orientation symbol for every eligible zero encountered in scan order. This cleans out various other sub-experiments that were part of the origiinal patch, which will be later included if found useful. Results are slightly positive on all sets (0.1 - 0.2% range). Change-Id: I57765c605fefc7fb9d1b57f1b356843602abefaf
2013-03-28 18:42:23 +01:00
if (c)
pt = vp9_get_coef_context(scan, nb, pad, token_cache, c, default_eob);
Deprecate code_zerogroup experiment. Delete code under the CONFIG_CODE_ZEROGROUP flag. Change-Id: I5fe6c7b42a5da9b73118e33594301da4129f320a
2013-05-07 18:24:21 +02:00
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#if CONFIG_BALANCED_COEFTREE
if (!c || token_cache[scan[c - 1]]) // do not skip eob
cost += token_costs_noskip[band][pt][t] + vp9_dct_value_cost_ptr[v];
else
cost += token_costs[band][pt][t] + vp9_dct_value_cost_ptr[v];
#else
End of orientation zero group experiment Adds an experiment that codes an end-of-orientation symbol for every eligible zero encountered in scan order. This cleans out various other sub-experiments that were part of the origiinal patch, which will be later included if found useful. Results are slightly positive on all sets (0.1 - 0.2% range). Change-Id: I57765c605fefc7fb9d1b57f1b356843602abefaf
2013-03-28 18:42:23 +01:00
cost += token_costs[band][pt][t] + vp9_dct_value_cost_ptr[v];
Removes the code_nonzerocount experiment This patch does not seem to give any benefits. Change-Id: I9d2b4091d6af3dfc0875f24db86c01e2de57f8db
2013-04-22 19:58:49 +02:00
if (!c || token_cache[scan[c - 1]])
Updating the model coef experiment Cleans up the experiment. Actually uses reduced counts for backward updates, and reduced number of probabilities in the context. No change in bitstream when the experiment is on. Between expt on and off: derfraw300 is down only -0.062% (which is better than when expts were run previously). Change-Id: I55285a049a0c22810bdb42914212ab5a4f8521b5
2013-05-17 15:40:25 +02:00
cost += vp9_cost_bit(coef_probs[band][pt][0], 1);
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#endif
Residual coding to cache energy class of tokens. Proposal for tuning the residual coding by changing how the context from previous tokens is calculated. Storing the energy class of previous tokens instead of the token itself eases the critical path of HW implementations. Change-Id: I6d71d856b84518f6c88de771ddd818436f794bab
2013-05-23 12:08:00 +02:00
token_cache[scan[c]] = vp9_pt_energy_class[t];
Entropy coding for hybrid transform Separates the entropy coding context models for 4x4, 8x8 and 16x16 ADST variants. There is a small improvement for HD (hd/std-hd) by about 0.1-0.2%. Results on derf/yt are about the same, probably because there is not enough statistics. Results may improve somewhat once the initial probability tables are updated for the hybrid transforms which is coming soon. Change-Id: Ic7c0c62dacc68ef551054fdb575be8b8507d32a8
2012-09-10 07:42:35 +02:00
}
Removes the code_nonzerocount experiment This patch does not seem to give any benefits. Change-Id: I9d2b4091d6af3dfc0875f24db86c01e2de57f8db
2013-04-22 19:58:49 +02:00
if (c < seg_eob) {
if (c)
pt = vp9_get_coef_context(scan, nb, pad, token_cache, c, default_eob);
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#if CONFIG_BALANCED_COEFTREE
cost += mb->token_costs_noskip[tx_size][type][ref]
[get_coef_band(band_translate, c)]
[pt][DCT_EOB_TOKEN];
#else
Removes the code_nonzerocount experiment This patch does not seem to give any benefits. Change-Id: I9d2b4091d6af3dfc0875f24db86c01e2de57f8db
2013-04-22 19:58:49 +02:00
cost += mb->token_costs[tx_size][type][ref]
Change to band calculation. Change band calculation back to simpler model based on the order in which coefficients are coded in scan order not the absolute coefficient positions. With the scatter scan experiment enabled the results were appear broadly neutral on derf (-0.028) but up a little on std-hd +0.134). Without the scatterscan experiment on the results were up derf as well. Change-Id: Ie9ef03ce42a6b24b849a4bebe950d4a5dffa6791
2013-05-09 19:47:58 +02:00
[get_coef_band(band_translate, c)]
Removes the code_nonzerocount experiment This patch does not seem to give any benefits. Change-Id: I9d2b4091d6af3dfc0875f24db86c01e2de57f8db
2013-04-22 19:58:49 +02:00
[pt][DCT_EOB_TOKEN];
Balancing coef-tree to reduce bool decodes This patch changes the coefficient tree to move the EOB to below the ZERO node in order to save number of bool decodes. The advantages of moving EOB one step down as opposed to two steps down in the other parallel patch are: 1. The coef modeling based on the One-node becomes independent of the tree structure above it, and 2. Fewer conext/counter increases are needed. The drawback is that the potential savings in bool decodes will be less, but assuming that 0s are much more predominant than 1's the potential savings is still likely to be substantial. Results on derf300: -0.237% Change-Id: Ie784be13dc98291306b338e8228703a4c2ea2242
2013-05-08 19:04:14 +02:00
#endif
Removes the code_nonzerocount experiment This patch does not seem to give any benefits. Change-Id: I9d2b4091d6af3dfc0875f24db86c01e2de57f8db
2013-04-22 19:58:49 +02:00
}
More refactoring of RD cost to make later changes easier. Change-Id: Ia6a290a6a53fdb88c971bea2de47ac73dcb7a364
2012-08-06 20:22:28 +02:00
}
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
// is eob first coefficient;
for (pt = 0; pt < (1 << tx_size); pt++) {
A[pt] = L[pt] = c > 0;
make cost_coeffs to use combined context Change-Id: Ia15f4244595fab49bffda0c651a750a8a9481d28
2013-02-26 02:01:30 +01:00
}
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
More refactoring of RD cost to make later changes easier. Change-Id: Ia6a290a6a53fdb88c971bea2de47ac73dcb7a364
2012-08-06 20:22:28 +02:00
return cost;
}
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
static void choose_txfm_size_from_rd(VP9_COMP *cpi, MACROBLOCK *x,
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
int (*r)[2], int *rate,
int *d, int *distortion,
int *s, int *skip,
int64_t txfm_cache[NB_TXFM_MODES],
TX_SIZE max_txfm_size) {
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
VP9_COMMON *const cm = &cpi->common;
MACROBLOCKD *const xd = &x->e_mbd;
MB_MODE_INFO *const mbmi = &xd->mode_info_context->mbmi;
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
vp9_prob skip_prob = vp9_get_pred_prob(cm, xd, PRED_MBSKIP);
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
int64_t rd[TX_SIZE_MAX_SB][2];
int n, m;
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
int s0, s1;
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
for (n = TX_4X4; n <= max_txfm_size; n++) {
r[n][1] = r[n][0];
for (m = 0; m <= n - (n == max_txfm_size); m++) {
if (m == n)
r[n][1] += vp9_cost_zero(cm->prob_tx[m]);
else
r[n][1] += vp9_cost_one(cm->prob_tx[m]);
}
}
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
assert(skip_prob > 0);
s0 = vp9_cost_bit(skip_prob, 0);
s1 = vp9_cost_bit(skip_prob, 1);
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
for (n = TX_4X4; n <= max_txfm_size; n++) {
if (s[n]) {
rd[n][0] = rd[n][1] = RDCOST(x->rdmult, x->rddiv, s1, d[n]);
} else {
rd[n][0] = RDCOST(x->rdmult, x->rddiv, r[n][0] + s0, d[n]);
rd[n][1] = RDCOST(x->rdmult, x->rddiv, r[n][1] + s0, d[n]);
Selectable transform size try 2. Results: derf (vanilla or +hybridtx) +0.2% and (+hybrid16x16 or +tx16x16) +0.7%-0.8%; HD (vanilla or +hybridtx) +0.1-0.2% and (+hybrid16x16 or +tx16x16) +1.4%, STD/HD (vanilla or +hybridtx) about even, and (+hybrid16x16 or +tx16x16) +0.8-1.0%. Change-Id: I03899e2f7a64e725a863f32e55366035ba77aa62
2012-10-09 18:18:21 +02:00
}
}
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
if (max_txfm_size == TX_32X32 &&
(cm->txfm_mode == ALLOW_32X32 ||
(cm->txfm_mode == TX_MODE_SELECT &&
rd[TX_32X32][1] < rd[TX_16X16][1] && rd[TX_32X32][1] < rd[TX_8X8][1] &&
rd[TX_32X32][1] < rd[TX_4X4][1]))) {
mbmi->txfm_size = TX_32X32;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
} else if (max_txfm_size >= TX_16X16 &&
(cm->txfm_mode == ALLOW_16X16 ||
cm->txfm_mode == ALLOW_32X32 ||
(cm->txfm_mode == TX_MODE_SELECT &&
rd[TX_16X16][1] < rd[TX_8X8][1] &&
rd[TX_16X16][1] < rd[TX_4X4][1]))) {
Selectable transform size try 2. Results: derf (vanilla or +hybridtx) +0.2% and (+hybrid16x16 or +tx16x16) +0.7%-0.8%; HD (vanilla or +hybridtx) +0.1-0.2% and (+hybrid16x16 or +tx16x16) +1.4%, STD/HD (vanilla or +hybridtx) about even, and (+hybrid16x16 or +tx16x16) +0.8-1.0%. Change-Id: I03899e2f7a64e725a863f32e55366035ba77aa62
2012-10-09 18:18:21 +02:00
mbmi->txfm_size = TX_16X16;
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
} else if (cm->txfm_mode == ALLOW_8X8 ||
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
cm->txfm_mode == ALLOW_16X16 ||
cm->txfm_mode == ALLOW_32X32 ||
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
(cm->txfm_mode == TX_MODE_SELECT && rd[TX_8X8][1] < rd[TX_4X4][1])) {
Selectable transform size try 2. Results: derf (vanilla or +hybridtx) +0.2% and (+hybrid16x16 or +tx16x16) +0.7%-0.8%; HD (vanilla or +hybridtx) +0.1-0.2% and (+hybrid16x16 or +tx16x16) +1.4%, STD/HD (vanilla or +hybridtx) about even, and (+hybrid16x16 or +tx16x16) +0.8-1.0%. Change-Id: I03899e2f7a64e725a863f32e55366035ba77aa62
2012-10-09 18:18:21 +02:00
mbmi->txfm_size = TX_8X8;
} else {
mbmi->txfm_size = TX_4X4;
}
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
*distortion = d[mbmi->txfm_size];
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
*rate = r[mbmi->txfm_size][cm->txfm_mode == TX_MODE_SELECT];
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
*skip = s[mbmi->txfm_size];
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
txfm_cache[ONLY_4X4] = rd[TX_4X4][0];
txfm_cache[ALLOW_8X8] = rd[TX_8X8][0];
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
txfm_cache[ALLOW_16X16] = rd[MIN(max_txfm_size, TX_16X16)][0];
txfm_cache[ALLOW_32X32] = rd[MIN(max_txfm_size, TX_32X32)][0];
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
if (max_txfm_size == TX_32X32 &&
rd[TX_32X32][1] < rd[TX_16X16][1] && rd[TX_32X32][1] < rd[TX_8X8][1] &&
rd[TX_32X32][1] < rd[TX_4X4][1])
txfm_cache[TX_MODE_SELECT] = rd[TX_32X32][1];
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
else if (max_txfm_size >= TX_16X16 &&
rd[TX_16X16][1] < rd[TX_8X8][1] && rd[TX_16X16][1] < rd[TX_4X4][1])
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
txfm_cache[TX_MODE_SELECT] = rd[TX_16X16][1];
Selectable transform size try 2. Results: derf (vanilla or +hybridtx) +0.2% and (+hybrid16x16 or +tx16x16) +0.7%-0.8%; HD (vanilla or +hybridtx) +0.1-0.2% and (+hybrid16x16 or +tx16x16) +1.4%, STD/HD (vanilla or +hybridtx) about even, and (+hybrid16x16 or +tx16x16) +0.8-1.0%. Change-Id: I03899e2f7a64e725a863f32e55366035ba77aa62
2012-10-09 18:18:21 +02:00
else
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
txfm_cache[TX_MODE_SELECT] = rd[TX_4X4][1] < rd[TX_8X8][1] ?
rd[TX_4X4][1] : rd[TX_8X8][1];
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
}
Convert coeff to per-plane MACROBLOCK data This commit moves the coeff storage from the MACROBLOCK struct to its per-plane part. The next commit will remove the coeff member from the BLOCK structure so that it is consistently accessed per-plane. Also refactors vp9_sb_block_error_c and vp9_sb_uv_block_error_c to be variable subsampling aware. Change-Id: I18c30f87f27c3a012119b6c1970d5fa499804455
2013-04-23 18:51:09 +02:00
static int block_error(int16_t *coeff, int16_t *dqcoeff,
int block_size, int shift) {
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
int i;
int64_t error = 0;
for (i = 0; i < block_size; i++) {
Convert coeff to per-plane MACROBLOCK data This commit moves the coeff storage from the MACROBLOCK struct to its per-plane part. The next commit will remove the coeff member from the BLOCK structure so that it is consistently accessed per-plane. Also refactors vp9_sb_block_error_c and vp9_sb_uv_block_error_c to be variable subsampling aware. Change-Id: I18c30f87f27c3a012119b6c1970d5fa499804455
2013-04-23 18:51:09 +02:00
int this_diff = coeff[i] - dqcoeff[i];
Fix overflow in RD error calculation code. Change-Id: I61ef1f198c876f9f79787ea7d7385a862cfbae19
2013-05-03 19:33:07 +02:00
error += (unsigned)this_diff * this_diff;
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
}
Re-add support for ADST in superblocks. This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-06 00:18:06 +01:00
error >>= shift;
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
libvpx: Fix some warnings. Change-Id: If8be8b9d28a29631f29c46daea8a226ab3580610
2013-01-18 18:44:23 +01:00
return error > INT_MAX ? INT_MAX : (int)error;
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
}
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
static int block_error_sby(MACROBLOCK *x, BLOCK_SIZE_TYPE bsize, int shift) {
const int bwl = b_width_log2(bsize), bhl = b_height_log2(bsize);
Convert coeff to per-plane MACROBLOCK data This commit moves the coeff storage from the MACROBLOCK struct to its per-plane part. The next commit will remove the coeff member from the BLOCK structure so that it is consistently accessed per-plane. Also refactors vp9_sb_block_error_c and vp9_sb_uv_block_error_c to be variable subsampling aware. Change-Id: I18c30f87f27c3a012119b6c1970d5fa499804455
2013-04-23 18:51:09 +02:00
return block_error(x->plane[0].coeff, x->e_mbd.plane[0].dqcoeff,
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
16 << (bwl + bhl), shift);
Convert coeff to per-plane MACROBLOCK data This commit moves the coeff storage from the MACROBLOCK struct to its per-plane part. The next commit will remove the coeff member from the BLOCK structure so that it is consistently accessed per-plane. Also refactors vp9_sb_block_error_c and vp9_sb_uv_block_error_c to be variable subsampling aware. Change-Id: I18c30f87f27c3a012119b6c1970d5fa499804455
2013-04-23 18:51:09 +02:00
}
Move qcoeff, dqcoeff from BLOCKD to per-plane data Start grouping data per-plane, as part of refactoring to support additional planes, and chroma planes with other-than 4:2:0 subsampling. Change-Id: Idb76a0e23ab239180c818025bae1f36f1608bb23
2013-04-02 23:50:40 +02:00
Convert coeff to per-plane MACROBLOCK data This commit moves the coeff storage from the MACROBLOCK struct to its per-plane part. The next commit will remove the coeff member from the BLOCK structure so that it is consistently accessed per-plane. Also refactors vp9_sb_block_error_c and vp9_sb_uv_block_error_c to be variable subsampling aware. Change-Id: I18c30f87f27c3a012119b6c1970d5fa499804455
2013-04-23 18:51:09 +02:00
static int block_error_sbuv(MACROBLOCK *x, BLOCK_SIZE_TYPE bsize, int shift) {
const int bwl = b_width_log2(bsize), bhl = b_height_log2(bsize);
int64_t sum = 0;
int plane;
Move qcoeff, dqcoeff from BLOCKD to per-plane data Start grouping data per-plane, as part of refactoring to support additional planes, and chroma planes with other-than 4:2:0 subsampling. Change-Id: Idb76a0e23ab239180c818025bae1f36f1608bb23
2013-04-02 23:50:40 +02:00
Convert coeff to per-plane MACROBLOCK data This commit moves the coeff storage from the MACROBLOCK struct to its per-plane part. The next commit will remove the coeff member from the BLOCK structure so that it is consistently accessed per-plane. Also refactors vp9_sb_block_error_c and vp9_sb_uv_block_error_c to be variable subsampling aware. Change-Id: I18c30f87f27c3a012119b6c1970d5fa499804455
2013-04-23 18:51:09 +02:00
for (plane = 1; plane < MAX_MB_PLANE; plane++) {
const int subsampling = x->e_mbd.plane[plane].subsampling_x +
x->e_mbd.plane[plane].subsampling_y;
sum += block_error(x->plane[plane].coeff, x->e_mbd.plane[plane].dqcoeff,
16 << (bwl + bhl - subsampling), 0);
}
sum >>= shift;
return sum > INT_MAX ? INT_MAX : (int)sum;
Move qcoeff, dqcoeff from BLOCKD to per-plane data Start grouping data per-plane, as part of refactoring to support additional planes, and chroma planes with other-than 4:2:0 subsampling. Change-Id: Idb76a0e23ab239180c818025bae1f36f1608bb23
2013-04-02 23:50:40 +02:00
}
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
static int rdcost_plane(VP9_COMMON *const cm, MACROBLOCK *x,
int plane, BLOCK_SIZE_TYPE bsize, TX_SIZE tx_size) {
Re-add support for ADST in superblocks. This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-06 00:18:06 +01:00
MACROBLOCKD *const xd = &x->e_mbd;
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
const int bwl = b_width_log2(bsize) - xd->plane[plane].subsampling_x;
const int bhl = b_height_log2(bsize) - xd->plane[plane].subsampling_y;
const int bw = 1 << bwl, bh = 1 << bhl;
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
ENTROPY_CONTEXT t_above[16], t_left[16];
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
int block, cost;
Re-add support for ADST in superblocks. This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-06 00:18:06 +01:00
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
vpx_memcpy(&t_above, xd->plane[plane].above_context,
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
sizeof(ENTROPY_CONTEXT) * bw);
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
vpx_memcpy(&t_left, xd->plane[plane].left_context,
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
sizeof(ENTROPY_CONTEXT) * bh);
Re-add support for ADST in superblocks. This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-06 00:18:06 +01:00
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
cost = 0;
for (block = 0; block < bw * bh; block += 1 << (tx_size * 2)) {
int x_idx, y_idx;
Re-add support for ADST in superblocks. This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-06 00:18:06 +01:00
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
txfrm_block_to_raster_xy(xd, bsize, plane, block, tx_size * 2,
&x_idx, &y_idx);
Re-add support for ADST in superblocks. This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-06 00:18:06 +01:00
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
cost += cost_coeffs(cm, x, plane, block, xd->plane[plane].plane_type,
t_above + x_idx, t_left + y_idx,
tx_size, bw * bh);
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
}
Re-add support for ADST in superblocks. This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-06 00:18:06 +01:00
return cost;
}
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
static int rdcost_uv(VP9_COMMON *const cm, MACROBLOCK *x,
BLOCK_SIZE_TYPE bsize, TX_SIZE tx_size) {
int cost = 0, plane;
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
for (plane = 1; plane < MAX_MB_PLANE; plane++) {
cost += rdcost_plane(cm, x, plane, bsize, tx_size);
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
}
return cost;
Re-add support for ADST in superblocks. This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-06 00:18:06 +01:00
}
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
static void super_block_yrd_for_txfm(VP9_COMMON *const cm, MACROBLOCK *x,
int *rate, int *distortion, int *skippable,
BLOCK_SIZE_TYPE bsize, TX_SIZE tx_size) {
Make superblocks independent of macroblock code and data. Split macroblock and superblock tokenization and detokenization functions and coefficient-related data structs so that the bitstream layout and related code of superblock coefficients looks less like it's a hack to fit macroblocks in superblocks. In addition, unify chroma transform size selection from luma transform size (i.e. always use the same size, as long as it fits the predictor); in practice, this means 32x32 and 64x64 superblocks using the 16x16 luma transform will now use the 16x16 (instead of the 8x8) chroma transform, and 64x64 superblocks using the 32x32 luma transform will now use the 32x32 (instead of the 16x16) chroma transform. Lastly, add a trellis optimize function for 32x32 transform blocks. HD gains about 0.3%, STDHD about 0.15% and derf about 0.1%. There's a few negative points here and there that I might want to analyze a little closer. Change-Id: Ibad7c3ddfe1acfc52771dfc27c03e9783e054430
2013-03-04 23:12:17 +01:00
MACROBLOCKD *const xd = &x->e_mbd;
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
xd->mode_info_context->mbmi.txfm_size = tx_size;
vp9_xform_quant_sby(cm, x, bsize);
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
*distortion = block_error_sby(x, bsize, tx_size == TX_32X32 ? 0 : 2);
*rate = rdcost_plane(cm, x, 0, bsize, tx_size);
tokenize: convert skippable functions Use the common block walker to calculate skippability. Change-Id: I6721e42f065df237426c91c1d871ec226ba7cdcb
2013-04-11 20:14:31 +02:00
*skippable = vp9_sby_is_skippable(xd, bsize);
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
}
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
static void super_block_yrd(VP9_COMP *cpi,
MACROBLOCK *x, int *rate, int *distortion,
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
int *skip, BLOCK_SIZE_TYPE bs,
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
int64_t txfm_cache[NB_TXFM_MODES]) {
Re-add support for ADST in superblocks. This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-06 00:18:06 +01:00
VP9_COMMON *const cm = &cpi->common;
int r[TX_SIZE_MAX_SB][2], d[TX_SIZE_MAX_SB], s[TX_SIZE_MAX_SB];
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Move src_diff to per-plane MACROBLOCK data First in a series of commits making certain MACROBLOCK members addressable per-plane. This commit also refactors the block subtraction functions vp9_subtract_b, vp9_subtract_sby_c, etc to be loops-over-planes and variable subsampling aware. Change-Id: I371d092b914ae0a495dfd852ea1a3d2467be6ec3
2013-04-23 17:26:10 +02:00
vp9_subtract_sby(x, bs);
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
if (bs >= BLOCK_SIZE_SB32X32)
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
super_block_yrd_for_txfm(cm, x, &r[TX_32X32][0], &d[TX_32X32], &s[TX_32X32],
bs, TX_32X32);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
if (bs >= BLOCK_SIZE_MB16X16)
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
super_block_yrd_for_txfm(cm, x, &r[TX_16X16][0], &d[TX_16X16], &s[TX_16X16],
bs, TX_16X16);
super_block_yrd_for_txfm(cm, x, &r[TX_8X8][0], &d[TX_8X8], &s[TX_8X8], bs,
TX_8X8);
super_block_yrd_for_txfm(cm, x, &r[TX_4X4][0], &d[TX_4X4], &s[TX_4X4], bs,
TX_4X4);
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
choose_txfm_size_from_rd(cpi, x, r, rate, d, distortion, s, skip, txfm_cache,
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
TX_32X32 - (bs < BLOCK_SIZE_SB32X32)
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
- (bs < BLOCK_SIZE_MB16X16));
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
}
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Move qcoeff, dqcoeff from BLOCKD to per-plane data Start grouping data per-plane, as part of refactoring to support additional planes, and chroma planes with other-than 4:2:0 subsampling. Change-Id: Idb76a0e23ab239180c818025bae1f36f1608bb23
2013-04-02 23:50:40 +02:00
static int64_t rd_pick_intra4x4block(VP9_COMP *cpi, MACROBLOCK *x, int ib,
a few clean-ups 1. remove prediction mode conversion 2. unified bmode, same for key and non-key frame 3. set I4X4_PRED count for pdf to 0, as I4X4_PRED is no longer coded ever. It is determined by ref_frame and block partition Change-Id: If5b282957c24339b241acdb9f2afef85658fe47d
2013-05-25 01:13:54 +02:00
MB_PREDICTION_MODE *best_mode,
Minor cosmetic changes to rdopt.c. Change-Id: Ib7b972486ccbbcb82fa4c05a5f6d7f8aaec32155
2012-08-07 01:21:23 +02:00
int *bmode_costs,
ENTROPY_CONTEXT *a, ENTROPY_CONTEXT *l,
int *bestrate, int *bestratey,
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
int *bestdistortion,
BLOCK_SIZE_TYPE bsize) {
MB_PREDICTION_MODE mode;
Some cleanups and fixes. Separates the logic on transform type selection previously spread out over a number of files into a separate function. Currently the tx_type field in b_mode_info is not used, but still left in there to eventually use for signaling the transform type in the bitstream. Also, now for tx_type = DCT_DCT, the regular integer DCT is used, as opposed to the floating point DCT used in conjuction with hybrid transform. Results change somewhat due to the transform change, but are within reasonable limits. The hd/std-hd sets are slightly up, while derf/yt are slightly down. Change-Id: I5776840c2239ca2da31ca6cfd7fd1148dc5f9e0f
2012-10-16 01:41:41 +02:00
MACROBLOCKD *xd = &x->e_mbd;
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
int64_t best_rd = INT64_MAX;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
int rate = 0;
int distortion;
Coding con-zero count rather than EOB for coeffs This patch revamps the entropy coding of coefficients to code first a non-zero count per coded block and correspondingly remove the EOB token from the token set. STATUS: Main encode/decode code achieving encode/decode sync - done. Forward and backward probability updates to the nzcs - done. Rd costing updates for nzcs - done. Note: The dynamic progrmaming apporach used in trellis quantization is not exactly compatible with nzcs. A suboptimal approach has been used instead where branch costs are updated to account for changes in the nzcs. TODO: Training the default probs/counts for nzcs Change-Id: I951bc1e22f47885077a7453a09b0493daa77883d
2013-02-20 19:16:24 +01:00
VP9_COMMON *const cm = &cpi->common;
Remove BLOCK structure All members can be referenced from their per-plane counterparts, and removes assumptions about 24 blocks per macroblock. Change-Id: I593fb0715e74cd84b48facd1c9b18c3ae1185d4b
2013-04-24 01:22:47 +02:00
const int src_stride = x->plane[0].src.stride;
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
uint8_t *src, *dst;
int16_t *src_diff, *coeff;
ENTROPY_CONTEXT ta[2], tempa[2];
ENTROPY_CONTEXT tl[2], templ[2];
Some cleanups and fixes. Separates the logic on transform type selection previously spread out over a number of files into a separate function. Currently the tx_type field in b_mode_info is not used, but still left in there to eventually use for signaling the transform type in the bitstream. Also, now for tx_type = DCT_DCT, the regular integer DCT is used, as opposed to the floating point DCT used in conjuction with hybrid transform. Results change somewhat due to the transform change, but are within reasonable limits. The hd/std-hd sets are slightly up, while derf/yt are slightly down. Change-Id: I5776840c2239ca2da31ca6cfd7fd1148dc5f9e0f
2012-10-16 01:41:41 +02:00
TX_TYPE tx_type = DCT_DCT;
TX_TYPE best_tx_type = DCT_DCT;
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
int bw = 1 << b_width_log2(bsize);
int bh = 1 << b_height_log2(bsize);
int idx, idy, block;
DECLARE_ALIGNED(16, int16_t, best_dqcoeff[4][16]);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
assert(ib < 4);
Deprecate the newbintramode experiment. Clean out code relating to newbintramode. Change-Id: Ie91f4f156cdf60ce0da8ca407c1c9cb00c7d0705
2013-05-07 14:58:36 +02:00
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
vpx_memcpy(ta, a, sizeof(ta));
vpx_memcpy(tl, l, sizeof(tl));
Coding con-zero count rather than EOB for coeffs This patch revamps the entropy coding of coefficients to code first a non-zero count per coded block and correspondingly remove the EOB token from the token set. STATUS: Main encode/decode code achieving encode/decode sync - done. Forward and backward probability updates to the nzcs - done. Rd costing updates for nzcs - done. Note: The dynamic progrmaming apporach used in trellis quantization is not exactly compatible with nzcs. A suboptimal approach has been used instead where branch costs are updated to account for changes in the nzcs. TODO: Training the default probs/counts for nzcs Change-Id: I951bc1e22f47885077a7453a09b0493daa77883d
2013-02-20 19:16:24 +01:00
xd->mode_info_context->mbmi.txfm_size = TX_4X4;
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
for (mode = DC_PRED; mode <= TM_PRED; ++mode) {
Remove compound intra-intra experiment. This experiment gives little gains and adds relatively much code complexity (and it hinders other experiments), so let's get rid of it. Change-Id: Id25e79a137a1b8a01138aa27a1fa0ba4a2df274a
2013-01-14 23:37:53 +01:00
int64_t this_rd;
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
int ratey = 0;
New b-intra mode where direction is contextual Preliminary patch on a new 4x4 intra mode B_CONTEXT_PRED where the dominant direction from the context is used to encode. Various decoder changes are needed to support decoding of B_CONTEXT_PRED in conjunction with hybrid transforms since the scan order and tokenization depends on the actual direction of prediction obtained from the context. Currently the traditional directional modes are used in conjunction with the B_CONTEXT_PRED, which also seems to provide the best results. The gains are small - in the 0.1% range. Change-Id: I5a7ea80b5218f42a9c0dfb42d3f79a68c7f0cdc2
2012-10-09 22:19:15 +02:00
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
if (cm->frame_type == KEY_FRAME)
rate = bmode_costs[mode];
else
rate = x->mbmode_cost[cm->frame_type][mode];
distortion = 0;
vpx_memcpy(tempa, ta, sizeof(ta));
vpx_memcpy(templ, tl, sizeof(tl));
for (idy = 0; idy < bh; ++idy) {
for (idx = 0; idx < bw; ++idx) {
block = ib + idy * 2 + idx;
xd->mode_info_context->bmi[block].as_mode.first = mode;
src = raster_block_offset_uint8(xd, BLOCK_SIZE_SB8X8, 0, block,
x->plane[0].src.buf, src_stride);
src_diff = raster_block_offset_int16(xd, BLOCK_SIZE_SB8X8, 0, block,
x->plane[0].src_diff);
coeff = BLOCK_OFFSET(x->plane[0].coeff, block, 16);
dst = raster_block_offset_uint8(xd, BLOCK_SIZE_SB8X8, 0, block,
xd->plane[0].dst.buf,
xd->plane[0].dst.stride);
Fix a bug in intra4x4 level rd loop This commit fixed a uninitialized value use in the intra 4x4/8x4/4x8 rate-distortion loop. Change-Id: I5c25b3536b59e4f5fbb23cf85baf93b2ccec7d72
2013-05-24 00:12:54 +02:00
vp9_intra4x4_predict(xd, block, BLOCK_SIZE_SB8X8, mode,
dst, xd->plane[0].dst.stride);
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
vp9_subtract_block(4, 4, src_diff, 8,
src, src_stride,
dst, xd->plane[0].dst.stride);
tx_type = get_tx_type_4x4(xd, block);
if (tx_type != DCT_DCT) {
vp9_short_fht4x4(src_diff, coeff, 8, tx_type);
x->quantize_b_4x4(x, block, tx_type, 16);
} else {
x->fwd_txm4x4(src_diff, coeff, 16);
x->quantize_b_4x4(x, block, tx_type, 16);
}
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
ratey += cost_coeffs(cm, x, 0, block, PLANE_TYPE_Y_WITH_DC,
tempa + idx, templ + idy, TX_4X4, 16);
distortion += vp9_block_error(coeff, BLOCK_OFFSET(xd->plane[0].dqcoeff,
block, 16), 16) >> 2;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
if (best_tx_type != DCT_DCT)
vp9_short_iht4x4_add(BLOCK_OFFSET(xd->plane[0].dqcoeff, block, 16),
dst, xd->plane[0].dst.stride, best_tx_type);
else
xd->inv_txm4x4_add(BLOCK_OFFSET(xd->plane[0].dqcoeff, block, 16),
dst, xd->plane[0].dst.stride);
}
}
Adds hybrid transform Adds ADST/DCT hybrid transform coding for Intra4x4 mode. The ADST is applied to directions in which the boundary pixels are used for prediction, while DCT applied to directions without corresponding boundary prediction. Adds enum TX_TYPE in b_mode_infor to indicate the transform type used. Make coding style consistent with google style. Fixed the commented issues. Experimental results in terms of bit-rate reduction: derf: 0.731% yt: 0.982% std-hd: 0.459% hd: 0.725% Will be looking at 8x8 transforms next. Change-Id: I46dbd7b80dbb3e8856e9c34fbc58cb3764a12fcf
2012-06-25 21:26:09 +02:00
Remove compound intra-intra experiment. This experiment gives little gains and adds relatively much code complexity (and it hinders other experiments), so let's get rid of it. Change-Id: Id25e79a137a1b8a01138aa27a1fa0ba4a2df274a
2013-01-14 23:37:53 +01:00
rate += ratey;
this_rd = RDCOST(x->rdmult, x->rddiv, rate, distortion);
Adds hybrid transform Adds ADST/DCT hybrid transform coding for Intra4x4 mode. The ADST is applied to directions in which the boundary pixels are used for prediction, while DCT applied to directions without corresponding boundary prediction. Adds enum TX_TYPE in b_mode_infor to indicate the transform type used. Make coding style consistent with google style. Fixed the commented issues. Experimental results in terms of bit-rate reduction: derf: 0.731% yt: 0.982% std-hd: 0.459% hd: 0.725% Will be looking at 8x8 transforms next. Change-Id: I46dbd7b80dbb3e8856e9c34fbc58cb3764a12fcf
2012-06-25 21:26:09 +02:00
Remove compound intra-intra experiment. This experiment gives little gains and adds relatively much code complexity (and it hinders other experiments), so let's get rid of it. Change-Id: Id25e79a137a1b8a01138aa27a1fa0ba4a2df274a
2013-01-14 23:37:53 +01:00
if (this_rd < best_rd) {
*bestrate = rate;
*bestratey = ratey;
*bestdistortion = distortion;
best_rd = this_rd;
*best_mode = mode;
best_tx_type = tx_type;
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
vpx_memcpy(a, tempa, sizeof(tempa));
vpx_memcpy(l, templ, sizeof(templ));
for (idy = 0; idy < bh; ++idy) {
for (idx = 0; idx < bw; ++idx) {
block = ib + idy * 2 + idx;
vpx_memcpy(best_dqcoeff[idy * 2 + idx],
BLOCK_OFFSET(xd->plane[0].dqcoeff, block, 16),
sizeof(best_dqcoeff[0]));
}
}
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
}
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
for (idy = 0; idy < bh; ++idy) {
for (idx = 0; idx < bw; ++idx) {
block = ib + idy * 2 + idx;
xd->mode_info_context->bmi[block].as_mode.first = *best_mode;
dst = raster_block_offset_uint8(xd, BLOCK_SIZE_SB8X8, 0, block,
xd->plane[0].dst.buf,
xd->plane[0].dst.stride);
vp9_intra4x4_predict(xd, block, BLOCK_SIZE_SB8X8, *best_mode,
dst, xd->plane[0].dst.stride);
// inverse transform
if (best_tx_type != DCT_DCT)
vp9_short_iht4x4_add(best_dqcoeff[idy * 2 + idx], dst,
xd->plane[0].dst.stride, best_tx_type);
else
xd->inv_txm4x4_add(best_dqcoeff[idy * 2 + idx], dst,
xd->plane[0].dst.stride);
}
WIP: 4x4 idct/recon merge This patch eliminates the intermediate diff buffer usage by combining the short idct and the add residual into one function. The encoder can use the same code as well. Change-Id: I296604bf73579c45105de0dd1adbcc91bcc53c22
2013-05-20 19:03:17 +02:00
}
Initial WebM release
2010-05-18 17:58:33 +02:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
return best_rd;
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Remove compound intra-intra experiment. This experiment gives little gains and adds relatively much code complexity (and it hinders other experiments), so let's get rid of it. Change-Id: Id25e79a137a1b8a01138aa27a1fa0ba4a2df274a
2013-01-14 23:37:53 +01:00
static int64_t rd_pick_intra4x4mby_modes(VP9_COMP *cpi, MACROBLOCK *mb,
int *Rate, int *rate_y,
Remove "update_context" variable from VP9_COMP context. The variable is always zero. Change-Id: Id5cdbecad543bca465a5b1d471badaec7e112c8d
2013-01-25 01:28:53 +01:00
int *Distortion, int64_t best_rd) {
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
int i, j;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
MACROBLOCKD *const xd = &mb->e_mbd;
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
BLOCK_SIZE_TYPE bsize = xd->mode_info_context->mbmi.sb_type;
int bw = 1 << b_width_log2(bsize);
int bh = 1 << b_height_log2(bsize);
int idx, idy;
Enable recursive partition down to 4x4 This commit allows the rate-distortion optimization recursion at encoder to go down to 4x4 block size. It deprecates the use of I4X4_PRED and SPLITMV syntax elements from bit-stream writing/reading. Will remove the unused probability models in the next patch. The partition type search and bit-stream are now capable of supporting the rectangular partition of 8x8 block, i.e., 8x4 and 4x8. Need to revise the rate-distortion parts to get these two partition tested in the rd loop. Change-Id: I0dfe3b90a1507ad6138db10cc58e6e237a06a9d6
2013-05-11 02:06:37 +02:00
int cost = 0;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
int distortion = 0;
int tot_rate_y = 0;
int64_t total_rd = 0;
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
ENTROPY_CONTEXT t_above[4], t_left[4];
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
int *bmode_costs;
Change above/left_context to use an 8x8 basis. Output changes slightly because of a minor bug in (at least) the sb32x16 block2above tx16x16 tables that previously existed in vp9_blockd.c. Change-Id: I624af28ac200a8322d64454cf05c79e9502968cc
2013-04-29 19:37:25 +02:00
vpx_memcpy(t_above, xd->plane[0].above_context, sizeof(t_above));
vpx_memcpy(t_left, xd->plane[0].left_context, sizeof(t_left));
Initial WebM release
2010-05-18 17:58:33 +02:00
Rename B_PRED to I4X4_PRED So it is consistent with I8x8_PRED. Change-Id: Iefa65124b2419690d83e526c611129c0ede29d11
2013-04-11 21:16:35 +02:00
xd->mode_info_context->mbmi.mode = I4X4_PRED;
further clean-ups on intra4x4 coding Removed one 4x4 prediction step that was unnessary in the rd loop. Removed a unused modecosts estimate from encoder side. Change-Id: I65221a52719d6876492996955ef04142d2752d86
2013-05-25 00:21:01 +02:00
bmode_costs = mb->mbmode_cost[cpi->common.frame_type];
Improved rd_pick_intra4x4block Eliminated unnecessary calculations. Improved performance by 10% on keyframes and 1.6% overall for the test clip used. Change-Id: I87671b26af5e2cc439e81d0fee3b15c7cd2a3309
2011-02-14 19:32:58 +01:00
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
for (idy = 0; idy < 2; idy += bh) {
for (idx = 0; idx < 2; idx += bw) {
MODE_INFO *const mic = xd->mode_info_context;
const int mis = xd->mode_info_stride;
a few clean-ups 1. remove prediction mode conversion 2. unified bmode, same for key and non-key frame 3. set I4X4_PRED count for pdf to 0, as I4X4_PRED is no longer coded ever. It is determined by ref_frame and block partition Change-Id: If5b282957c24339b241acdb9f2afef85658fe47d
2013-05-25 01:13:54 +02:00
MB_PREDICTION_MODE UNINITIALIZED_IS_SAFE(best_mode);
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
int UNINITIALIZED_IS_SAFE(r), UNINITIALIZED_IS_SAFE(ry);
int UNINITIALIZED_IS_SAFE(d);
i = idy * 2 + idx;
if (xd->frame_type == KEY_FRAME) {
const MB_PREDICTION_MODE A = above_block_mode(mic, i, mis);
const MB_PREDICTION_MODE L = (xd->left_available || idx) ?
left_block_mode(mic, i) : DC_PRED;
bmode_costs = mb->bmode_costs[A][L];
}
Improved rd_pick_intra4x4block Eliminated unnecessary calculations. Improved performance by 10% on keyframes and 1.6% overall for the test clip used. Change-Id: I87671b26af5e2cc439e81d0fee3b15c7cd2a3309
2011-02-14 19:32:58 +01:00
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
total_rd += rd_pick_intra4x4block(cpi, mb, i, &best_mode, bmode_costs,
t_above + idx, t_left + idy,
&r, &ry, &d, bsize);
cost += r;
distortion += d;
tot_rate_y += ry;
mic->bmi[i].as_mode.first = best_mode;
for (j = 1; j < bh; ++j)
mic->bmi[i + j * 2].as_mode.first = best_mode;
for (j = 1; j < bw; ++j)
mic->bmi[i + j].as_mode.first = best_mode;
if (total_rd >= best_rd)
break;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
}
}
Initial WebM release
2010-05-18 17:58:33 +02:00
Make RDCOST() results 64-bit. This prevents various integer overflows that several of us have been seeing in the past few days. Change-Id: If0020613a608e13d311b7838796c7ad53cb97255
2012-08-02 19:07:33 +02:00
if (total_rd >= best_rd)
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
return INT64_MAX;
Added early breakout for vp8_rd_pick_intra4x4mby_modes Improved performance of good quality, speed 0 (3% average) with no average quality loss. Change-Id: Ica34473f99bd74260eaebde6b132185e09e3c09d
2011-02-08 22:50:43 +01:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
*Rate = cost;
rdopt: fix use of uninitialized value in addition rd_pick_intra4x4mby_modes / rd_pick_intra8x8mby_modes would both use the input value of 'rate_y' in the return calculation. In many places this value is uninitialized. Remove the unneeded sum. Change-Id: Icbd3df685303000301e69291c0ebc06f74bd548d
2012-11-05 21:50:16 +01:00
*rate_y = tot_rate_y;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
*Distortion = distortion;
Initial WebM release
2010-05-18 17:58:33 +02:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
return RDCOST(mb->rdmult, mb->rddiv, cost, distortion);
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Move RD intra block mode selection to rdopt.c This change is analogous to I0b67dae1f8a74902378da7bdf565e39ab832dda7, which made the move for the non-RD path. Change-Id: If63fc1b0cd1eb7f932e710f83ff24d91454f8ed1
2011-06-08 18:05:05 +02:00
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
static int64_t rd_pick_intra_sby_mode(VP9_COMP *cpi, MACROBLOCK *x,
int *rate, int *rate_tokenonly,
int *distortion, int *skippable,
BLOCK_SIZE_TYPE bsize,
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
int64_t txfm_cache[NB_TXFM_MODES]) {
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
MB_PREDICTION_MODE mode;
MB_PREDICTION_MODE UNINITIALIZED_IS_SAFE(mode_selected);
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
MACROBLOCKD *xd = &x->e_mbd;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
int this_rate, this_rate_tokenonly;
Add proper skip support to intra frames. Change-Id: I7524883fb29f42303fb46a5bc6772fbcf8781d1d
2012-08-29 19:43:20 +02:00
int this_distortion, s;
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
int64_t best_rd = INT64_MAX, this_rd;
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
TX_SIZE UNINITIALIZED_IS_SAFE(best_tx);
int i;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Enable recursive partition down to 4x4 This commit allows the rate-distortion optimization recursion at encoder to go down to 4x4 block size. It deprecates the use of I4X4_PRED and SPLITMV syntax elements from bit-stream writing/reading. Will remove the unused probability models in the next patch. The partition type search and bit-stream are now capable of supporting the rectangular partition of 8x8 block, i.e., 8x4 and 4x8. Need to revise the rate-distortion parts to get these two partition tested in the rd loop. Change-Id: I0dfe3b90a1507ad6138db10cc58e6e237a06a9d6
2013-05-11 02:06:37 +02:00
if (bsize < BLOCK_SIZE_SB8X8) {
x->e_mbd.mode_info_context->mbmi.txfm_size = TX_4X4;
return best_rd;
}
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
for (i = 0; i < NB_TXFM_MODES; i++)
txfm_cache[i] = INT64_MAX;
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
/* Y Search for 32x32 intra prediction mode */
for (mode = DC_PRED; mode <= TM_PRED; mode++) {
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
int64_t local_txfm_cache[NB_TXFM_MODES];
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
MODE_INFO *const mic = xd->mode_info_context;
const int mis = xd->mode_info_stride;
const MB_PREDICTION_MODE A = above_block_mode(mic, 0, mis);
const MB_PREDICTION_MODE L = xd->left_available ?
left_block_mode(mic, 0) : DC_PRED;
int *bmode_costs = x->bmode_costs[A][L];
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
x->e_mbd.mode_info_context->mbmi.mode = mode;
Make the use of pred buffers consistent in MB/SB Use in-place buffers (dst of MACROBLOCKD) for macroblock prediction. This makes the macroblock buffer handling consistent with those of superblock. Remove predictor buffer MACROBLOCKD. Change-Id: Id1bcd898961097b1e6230c10f0130753a59fc6df
2013-04-15 18:31:27 +02:00
vp9_build_intra_predictors_sby_s(&x->e_mbd, bsize);
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
super_block_yrd(cpi, x, &this_rate_tokenonly, &this_distortion, &s,
bsize, local_txfm_cache);
Make the intra rd search support 8x4/4x8 This commit allows the rate-distortion optimization of intra coding capable of supporting 8x4 and 4x8 partition settings. It enables the entropy coding of intra modes in key frame using a unified contextual probability model conditioned on its above/left prediction modes. Coding performance: derf 0.464% Change-Id: Ieed055084e11fcb64d5d5faeb0e706d30268ba18
2013-05-21 01:04:28 +02:00
this_rate = this_rate_tokenonly + bmode_costs[mode];
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
this_rd = RDCOST(x->rdmult, x->rddiv, this_rate, this_distortion);
if (this_rd < best_rd) {
mode_selected = mode;
best_rd = this_rd;
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
best_tx = x->e_mbd.mode_info_context->mbmi.txfm_size;
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
*rate = this_rate;
*rate_tokenonly = this_rate_tokenonly;
*distortion = this_distortion;
*skippable = s;
}
Selectable transform size try 2. Results: derf (vanilla or +hybridtx) +0.2% and (+hybrid16x16 or +tx16x16) +0.7%-0.8%; HD (vanilla or +hybridtx) +0.1-0.2% and (+hybrid16x16 or +tx16x16) +1.4%, STD/HD (vanilla or +hybridtx) about even, and (+hybrid16x16 or +tx16x16) +0.8-1.0%. Change-Id: I03899e2f7a64e725a863f32e55366035ba77aa62
2012-10-09 18:18:21 +02:00
Remove compound intra-intra experiment. This experiment gives little gains and adds relatively much code complexity (and it hinders other experiments), so let's get rid of it. Change-Id: Id25e79a137a1b8a01138aa27a1fa0ba4a2df274a
2013-01-14 23:37:53 +01:00
for (i = 0; i < NB_TXFM_MODES; i++) {
int64_t adj_rd = this_rd + local_txfm_cache[i] -
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
local_txfm_cache[cpi->common.txfm_mode];
Remove compound intra-intra experiment. This experiment gives little gains and adds relatively much code complexity (and it hinders other experiments), so let's get rid of it. Change-Id: Id25e79a137a1b8a01138aa27a1fa0ba4a2df274a
2013-01-14 23:37:53 +01:00
if (adj_rd < txfm_cache[i]) {
txfm_cache[i] = adj_rd;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
}
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
}
Initial WebM release
2010-05-18 17:58:33 +02:00
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
x->e_mbd.mode_info_context->mbmi.mode = mode_selected;
x->e_mbd.mode_info_context->mbmi.txfm_size = best_tx;
hybrid transform of 16x16 dimension Enable ADST/DCT of dimension 16x16 for I16X16 modes. This change provides benefits mostly for hd sequences. Set up the framework for selectable transform dimension. Also allowing quantization parameter threshold to control the use of hybrid transform (This is currently disabled by setting threshold always above the quantization parameter. Adaptive thresholding can be built upon this, which will further improve the coding performance.) The coding performance gains (with respect to the codec that has all other configuration settings turned on) are derf: 0.013 yt: 0.086 hd: 0.198 std-hd: 0.501 Change-Id: Ibb4263a61fc74e0b3c345f54d73e8c73552bf926
2012-08-29 20:25:38 +02:00
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
return best_rd;
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Adds hybrid transform Adds ADST/DCT hybrid transform coding for Intra4x4 mode. The ADST is applied to directions in which the boundary pixels are used for prediction, while DCT applied to directions without corresponding boundary prediction. Adds enum TX_TYPE in b_mode_infor to indicate the transform type used. Make coding style consistent with google style. Fixed the commented issues. Experimental results in terms of bit-rate reduction: derf: 0.731% yt: 0.982% std-hd: 0.459% hd: 0.725% Will be looking at 8x8 transforms next. Change-Id: I46dbd7b80dbb3e8856e9c34fbc58cb3764a12fcf
2012-06-25 21:26:09 +02:00
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
static void super_block_uvrd_for_txfm(VP9_COMMON *const cm, MACROBLOCK *x,
int *rate, int *distortion,
int *skippable, BLOCK_SIZE_TYPE bsize,
TX_SIZE uv_tx_size) {
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
MACROBLOCKD *const xd = &x->e_mbd;
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
vp9_xform_quant_sbuv(cm, x, bsize);
revised the rate distortion computation for UV this commit changed the UV r/d calculation in the mode decision process to properly account for the rate of 8x8 transform coefficients. Change-Id: I485f8f35f2b61db0b6539beb32e83481b1cf083b
2012-01-21 00:30:31 +01:00
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
*distortion = block_error_sbuv(x, bsize, uv_tx_size == TX_32X32 ? 0 : 2);
*rate = rdcost_uv(cm, x, bsize, uv_tx_size);
*skippable = vp9_sbuv_is_skippable(xd, bsize);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
}
static void super_block_uvrd(VP9_COMMON *const cm, MACROBLOCK *x,
int *rate, int *distortion, int *skippable,
BLOCK_SIZE_TYPE bsize) {
MACROBLOCKD *const xd = &x->e_mbd;
MB_MODE_INFO *const mbmi = &xd->mode_info_context->mbmi;
vp9_subtract_sbuv(x, bsize);
if (mbmi->txfm_size >= TX_32X32 && bsize >= BLOCK_SIZE_SB64X64) {
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
super_block_uvrd_for_txfm(cm, x, rate, distortion, skippable, bsize,
TX_32X32);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
} else if (mbmi->txfm_size >= TX_16X16 && bsize >= BLOCK_SIZE_SB32X32) {
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
super_block_uvrd_for_txfm(cm, x, rate, distortion, skippable, bsize,
TX_16X16);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
} else if (mbmi->txfm_size >= TX_8X8 && bsize >= BLOCK_SIZE_MB16X16) {
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
super_block_uvrd_for_txfm(cm, x, rate, distortion, skippable, bsize,
TX_8X8);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
} else {
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
super_block_uvrd_for_txfm(cm, x, rate, distortion, skippable, bsize,
TX_4X4);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
}
}
static int64_t rd_pick_intra_sbuv_mode(VP9_COMP *cpi, MACROBLOCK *x,
int *rate, int *rate_tokenonly,
int *distortion, int *skippable,
BLOCK_SIZE_TYPE bsize) {
MB_PREDICTION_MODE mode;
MB_PREDICTION_MODE UNINITIALIZED_IS_SAFE(mode_selected);
int64_t best_rd = INT64_MAX, this_rd;
int this_rate_tokenonly, this_rate;
int this_distortion, s;
for (mode = DC_PRED; mode <= TM_PRED; mode++) {
x->e_mbd.mode_info_context->mbmi.uv_mode = mode;
vp9_build_intra_predictors_sbuv_s(&x->e_mbd, bsize);
super_block_uvrd(&cpi->common, x, &this_rate_tokenonly,
&this_distortion, &s, bsize);
this_rate = this_rate_tokenonly +
x->intra_uv_mode_cost[x->e_mbd.frame_type][mode];
this_rd = RDCOST(x->rdmult, x->rddiv, this_rate, this_distortion);
if (this_rd < best_rd) {
mode_selected = mode;
best_rd = this_rd;
*rate = this_rate;
*rate_tokenonly = this_rate_tokenonly;
*distortion = this_distortion;
*skippable = s;
}
}
x->e_mbd.mode_info_context->mbmi.uv_mode = mode_selected;
return best_rd;
}
int vp9_cost_mv_ref(VP9_COMP *cpi,
MB_PREDICTION_MODE m,
const int mode_context) {
MACROBLOCKD *xd = &cpi->mb.e_mbd;
int segment_id = xd->mode_info_context->mbmi.segment_id;
// Dont account for mode here if segment skip is enabled.
if (!vp9_segfeature_active(xd, segment_id, SEG_LVL_SKIP)) {
VP9_COMMON *pc = &cpi->common;
vp9_prob p [VP9_MVREFS - 1];
assert(NEARESTMV <= m && m <= SPLITMV);
vp9_mv_ref_probs(pc, p, mode_context);
Remove some unused code related to macroblock/splitmv coding. Change-Id: Ic40d56fb162f4e201547dfae33e62ccd9e865889
2013-05-29 15:29:56 +02:00
return cost_token(vp9_sb_mv_ref_tree, p,
vp9_sb_mv_ref_encoding_array - NEARESTMV + m);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
} else
return 0;
}
void vp9_set_mbmode_and_mvs(MACROBLOCK *x, MB_PREDICTION_MODE mb, int_mv *mv) {
x->e_mbd.mode_info_context->mbmi.mode = mb;
x->e_mbd.mode_info_context->mbmi.mv[0].as_int = mv->as_int;
}
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
static int labels2mode(MACROBLOCK *x, int i,
a few clean-ups 1. remove prediction mode conversion 2. unified bmode, same for key and non-key frame 3. set I4X4_PRED count for pdf to 0, as I4X4_PRED is no longer coded ever. It is determined by ref_frame and block partition Change-Id: If5b282957c24339b241acdb9f2afef85658fe47d
2013-05-25 01:13:54 +02:00
MB_PREDICTION_MODE this_mode,
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
int_mv *this_mv, int_mv *this_second_mv,
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
int_mv frame_mv[MB_MODE_COUNT][MAX_REF_FRAMES],
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
int_mv seg_mvs[MAX_REF_FRAMES - 1],
int_mv *best_ref_mv,
int_mv *second_best_ref_mv,
int *mvjcost, int *mvcost[2], VP9_COMP *cpi) {
MACROBLOCKD *const xd = &x->e_mbd;
MODE_INFO *const mic = xd->mode_info_context;
MB_MODE_INFO * mbmi = &mic->mbmi;
Refactor rd loop for inter modes This commit pulls the iterative motion search for compound inter- inter out from handle_inter_mode_ as a separate function. Hence, it is applicable to 4x4/4x8/8x4 level compound inter search to be enabled later. Also edit the rd loop for 4x4 inter block sizes for cosmetic purpose. Change-Id: Ibc71a11cbe5a26cd52faba01026cf8446cf4d2b4
2013-05-28 23:02:29 +02:00
int cost = 0, thismvcost = 0;
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
int idx, idy;
int bw = 1 << b_width_log2(mbmi->sb_type);
int bh = 1 << b_height_log2(mbmi->sb_type);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
/* We have to be careful retrieving previously-encoded motion vectors.
Ones from this macroblock have to be pulled from the BLOCKD array
as they have not yet made it to the bmi array in our MB_MODE_INFO. */
Refactor rd loop for inter modes This commit pulls the iterative motion search for compound inter- inter out from handle_inter_mode_ as a separate function. Hence, it is applicable to 4x4/4x8/8x4 level compound inter search to be enabled later. Also edit the rd loop for 4x4 inter block sizes for cosmetic purpose. Change-Id: Ibc71a11cbe5a26cd52faba01026cf8446cf4d2b4
2013-05-28 23:02:29 +02:00
MB_PREDICTION_MODE m;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
Refactor rd loop for inter modes This commit pulls the iterative motion search for compound inter- inter out from handle_inter_mode_ as a separate function. Hence, it is applicable to 4x4/4x8/8x4 level compound inter search to be enabled later. Also edit the rd loop for 4x4 inter block sizes for cosmetic purpose. Change-Id: Ibc71a11cbe5a26cd52faba01026cf8446cf4d2b4
2013-05-28 23:02:29 +02:00
// the only time we should do costing for new motion vector or mode
// is when we are on a new label (jbb May 08, 2007)
switch (m = this_mode) {
case NEWMV:
if (mbmi->second_ref_frame > 0) {
this_mv->as_int = seg_mvs[mbmi->ref_frame - 1].as_int;
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
this_second_mv->as_int = seg_mvs[mbmi->second_ref_frame - 1].as_int;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
}
Refactor rd loop for inter modes This commit pulls the iterative motion search for compound inter- inter out from handle_inter_mode_ as a separate function. Hence, it is applicable to 4x4/4x8/8x4 level compound inter search to be enabled later. Also edit the rd loop for 4x4 inter block sizes for cosmetic purpose. Change-Id: Ibc71a11cbe5a26cd52faba01026cf8446cf4d2b4
2013-05-28 23:02:29 +02:00
thismvcost = vp9_mv_bit_cost(this_mv, best_ref_mv, mvjcost, mvcost,
102, xd->allow_high_precision_mv);
if (mbmi->second_ref_frame > 0) {
thismvcost += vp9_mv_bit_cost(this_second_mv, second_best_ref_mv,
mvjcost, mvcost, 102,
xd->allow_high_precision_mv);
}
break;
case NEARESTMV:
this_mv->as_int = frame_mv[NEARESTMV][mbmi->ref_frame].as_int;
if (mbmi->second_ref_frame > 0)
this_second_mv->as_int =
frame_mv[NEARESTMV][mbmi->second_ref_frame].as_int;
break;
case NEARMV:
this_mv->as_int = frame_mv[NEARMV][mbmi->ref_frame].as_int;
if (mbmi->second_ref_frame > 0)
this_second_mv->as_int =
frame_mv[NEARMV][mbmi->second_ref_frame].as_int;
break;
case ZEROMV:
this_mv->as_int = 0;
if (mbmi->second_ref_frame > 0)
this_second_mv->as_int = 0;
break;
default:
break;
}
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
Refactor rd loop for inter modes This commit pulls the iterative motion search for compound inter- inter out from handle_inter_mode_ as a separate function. Hence, it is applicable to 4x4/4x8/8x4 level compound inter search to be enabled later. Also edit the rd loop for 4x4 inter block sizes for cosmetic purpose. Change-Id: Ibc71a11cbe5a26cd52faba01026cf8446cf4d2b4
2013-05-28 23:02:29 +02:00
cost = vp9_cost_mv_ref(cpi, this_mode,
mbmi->mb_mode_context[mbmi->ref_frame]);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
Refactor rd loop for inter modes This commit pulls the iterative motion search for compound inter- inter out from handle_inter_mode_ as a separate function. Hence, it is applicable to 4x4/4x8/8x4 level compound inter search to be enabled later. Also edit the rd loop for 4x4 inter block sizes for cosmetic purpose. Change-Id: Ibc71a11cbe5a26cd52faba01026cf8446cf4d2b4
2013-05-28 23:02:29 +02:00
mic->bmi[i].as_mv[0].as_int = this_mv->as_int;
if (mbmi->second_ref_frame > 0)
mic->bmi[i].as_mv[1].as_int = this_second_mv->as_int;
x->partition_info->bmi[i].mode = m;
x->partition_info->bmi[i].mv.as_int = this_mv->as_int;
if (mbmi->second_ref_frame > 0)
x->partition_info->bmi[i].second_mv.as_int = this_second_mv->as_int;
for (idy = 0; idy < bh; ++idy) {
for (idx = 0; idx < bw; ++idx) {
vpx_memcpy(&mic->bmi[i + idy * 2 + idx],
&mic->bmi[i], sizeof(mic->bmi[i]));
vpx_memcpy(&x->partition_info->bmi[i + idy * 2 + idx],
&x->partition_info->bmi[i],
sizeof(x->partition_info->bmi[i]));
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
}
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
}
cost += thismvcost;
return cost;
}
static int64_t encode_inter_mb_segment(VP9_COMMON *const cm,
MACROBLOCK *x,
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
int i,
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
int *labelyrate,
int *distortion,
ENTROPY_CONTEXT *ta,
ENTROPY_CONTEXT *tl) {
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
int k;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
MACROBLOCKD *xd = &x->e_mbd;
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
BLOCK_SIZE_TYPE bsize = xd->mode_info_context->mbmi.sb_type;
int bwl = b_width_log2(bsize), bw = 1 << bwl;
int bhl = b_height_log2(bsize), bh = 1 << bhl;
int idx, idy;
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
const int src_stride = x->plane[0].src.stride;
uint8_t* const src =
raster_block_offset_uint8(xd, BLOCK_SIZE_SB8X8, 0, i,
x->plane[0].src.buf, src_stride);
int16_t* src_diff =
raster_block_offset_int16(xd, BLOCK_SIZE_SB8X8, 0, i,
x->plane[0].src_diff);
int16_t* coeff = BLOCK_OFFSET(x->plane[0].coeff, 16, i);
uint8_t* const pre =
raster_block_offset_uint8(xd, BLOCK_SIZE_SB8X8, 0, i,
xd->plane[0].pre[0].buf,
xd->plane[0].pre[0].stride);
uint8_t* const dst =
raster_block_offset_uint8(xd, BLOCK_SIZE_SB8X8, 0, i,
xd->plane[0].dst.buf,
xd->plane[0].dst.stride);
int thisdistortion = 0;
int thisrate = 0;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
*labelyrate = 0;
*distortion = 0;
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
vp9_build_inter_predictor(pre,
xd->plane[0].pre[0].stride,
dst,
xd->plane[0].dst.stride,
&xd->mode_info_context->bmi[i].as_mv[0],
&xd->scale_factor[0],
4 * bw, 4 * bh, 0 /* no avg */, &xd->subpix);
// TODO(debargha): Make this work properly with the
// implicit-compoundinter-weight experiment when implicit
// weighting for splitmv modes is turned on.
if (xd->mode_info_context->mbmi.second_ref_frame > 0) {
uint8_t* const second_pre =
raster_block_offset_uint8(xd, BLOCK_SIZE_SB8X8, 0, i,
xd->plane[0].pre[1].buf,
xd->plane[0].pre[1].stride);
vp9_build_inter_predictor(second_pre, xd->plane[0].pre[1].stride,
dst, xd->plane[0].dst.stride,
&xd->mode_info_context->bmi[i].as_mv[1],
&xd->scale_factor[1], 4 * bw, 4 * bh, 1,
&xd->subpix);
}
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
vp9_subtract_block(4 * bh, 4 * bw, src_diff, 8,
src, src_stride,
dst, xd->plane[0].dst.stride);
k = i;
for (idy = 0; idy < bh; ++idy) {
for (idx = 0; idx < bw; ++idx) {
k += (idy * 2 + idx);
src_diff = raster_block_offset_int16(xd, BLOCK_SIZE_SB8X8, 0, k,
x->plane[0].src_diff);
coeff = BLOCK_OFFSET(x->plane[0].coeff, 16, k);
x->fwd_txm4x4(src_diff, coeff, 16);
x->quantize_b_4x4(x, k, DCT_DCT, 16);
thisdistortion += vp9_block_error(coeff,
BLOCK_OFFSET(xd->plane[0].dqcoeff,
k, 16), 16);
thisrate += cost_coeffs(cm, x, 0, k, PLANE_TYPE_Y_WITH_DC,
ta + (k & 1),
tl + (k >> 1), TX_4X4, 16);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
}
}
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
*distortion += thisdistortion;
*labelyrate += thisrate;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
*distortion >>= 2;
return RDCOST(x->rdmult, x->rddiv, *labelyrate, *distortion);
}
typedef struct {
int_mv *ref_mv, *second_ref_mv;
int_mv mvp;
int64_t segment_rd;
int r;
int d;
int segment_yrate;
a few clean-ups 1. remove prediction mode conversion 2. unified bmode, same for key and non-key frame 3. set I4X4_PRED count for pdf to 0, as I4X4_PRED is no longer coded ever. It is determined by ref_frame and block partition Change-Id: If5b282957c24339b241acdb9f2afef85658fe47d
2013-05-25 01:13:54 +02:00
MB_PREDICTION_MODE modes[4];
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
int_mv mvs[4], second_mvs[4];
int eobs[4];
int mvthresh;
} BEST_SEG_INFO;
static INLINE int mv_check_bounds(MACROBLOCK *x, int_mv *mv) {
int r = 0;
r |= (mv->as_mv.row >> 3) < x->mv_row_min;
r |= (mv->as_mv.row >> 3) > x->mv_row_max;
r |= (mv->as_mv.col >> 3) < x->mv_col_min;
r |= (mv->as_mv.col >> 3) > x->mv_col_max;
return r;
}
Add building blocks for 4x8/8x4 rd search These building blocks enable rate-distortion optimization search over block sizes of 8x4 and 4x8. Need to convert them into mmx/sse forms. Change-Id: I570ea2d22d14ceec3fe3575128d7dfa172a577de
2013-05-15 21:19:59 +02:00
static enum BlockSize get_block_size(int bw, int bh) {
if (bw == 4 && bh == 4)
return BLOCK_4X4;
if (bw == 4 && bh == 8)
return BLOCK_4X8;
if (bw == 8 && bh == 4)
return BLOCK_8X4;
if (bw == 8 && bh == 8)
return BLOCK_8X8;
if (bw == 8 && bh == 16)
return BLOCK_8X16;
if (bw == 16 && bh == 8)
return BLOCK_16X8;
if (bw == 16 && bh == 16)
return BLOCK_16X16;
if (bw == 32 && bh == 32)
return BLOCK_32X32;
if (bw == 32 && bh == 16)
return BLOCK_32X16;
if (bw == 16 && bh == 32)
return BLOCK_16X32;
if (bw == 64 && bh == 32)
return BLOCK_64X32;
if (bw == 32 && bh == 64)
return BLOCK_32X64;
if (bw == 64 && bh == 64)
return BLOCK_64X64;
assert(0);
return -1;
}
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
static void rd_check_segment_txsize(VP9_COMP *cpi, MACROBLOCK *x,
BEST_SEG_INFO *bsi,
int_mv seg_mvs[4][MAX_REF_FRAMES - 1]) {
int i, j;
int br = 0, bd = 0;
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
MB_PREDICTION_MODE this_mode;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
MB_MODE_INFO * mbmi = &x->e_mbd.mode_info_context->mbmi;
const int label_count = 4;
int64_t this_segment_rd = 0, other_segment_rd;
int label_mv_thresh;
int rate = 0;
int sbr = 0, sbd = 0;
int segmentyrate = 0;
int best_eobs[4] = { 0 };
Add building blocks for 4x8/8x4 rd search These building blocks enable rate-distortion optimization search over block sizes of 8x4 and 4x8. Need to convert them into mmx/sse forms. Change-Id: I570ea2d22d14ceec3fe3575128d7dfa172a577de
2013-05-15 21:19:59 +02:00
BLOCK_SIZE_TYPE bsize = mbmi->sb_type;
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
int bwl = b_width_log2(bsize), bw = 1 << bwl;
int bhl = b_height_log2(bsize), bh = 1 << bhl;
int idx, idy;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
vp9_variance_fn_ptr_t *v_fn_ptr;
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
ENTROPY_CONTEXT t_above[4], t_left[4];
ENTROPY_CONTEXT t_above_b[4], t_left_b[4];
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
vpx_memcpy(t_above, x->e_mbd.plane[0].above_context, sizeof(t_above));
vpx_memcpy(t_left, x->e_mbd.plane[0].left_context, sizeof(t_left));
Add building blocks for 4x8/8x4 rd search These building blocks enable rate-distortion optimization search over block sizes of 8x4 and 4x8. Need to convert them into mmx/sse forms. Change-Id: I570ea2d22d14ceec3fe3575128d7dfa172a577de
2013-05-15 21:19:59 +02:00
v_fn_ptr = &cpi->fn_ptr[get_block_size(4 << bwl, 4 << bhl)];
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
// 64 makes this threshold really big effectively
// making it so that we very rarely check mvs on
// segments. setting this to 1 would make mv thresh
// roughly equal to what it is for macroblocks
label_mv_thresh = 1 * bsi->mvthresh / label_count;
// Segmentation method overheads
other_segment_rd = this_segment_rd;
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
for (idy = 0; idy < 2; idy += bh) {
for (idx = 0; idx < 2; idx += bw) {
// TODO(jingning,rbultje): rewrite the rate-distortion optimization
Merge 4x4 block level partition into codebase Move 4x4/4x8/8x4 partition coding out of experimental list. This commit fixed the unit test failure issues. It also resolved the merge conflicts between 4x4 block level partition and iterative motion search for comp_inter_inter. Change-Id: I898671f0631f5ddc4f5cc68d4c62ead7de9c5a58
2013-05-22 06:28:42 +02:00
// loop for 4x4/4x8/8x4 block coding. to be replaced with new rd loop
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
int_mv mode_mv[MB_MODE_COUNT], second_mode_mv[MB_MODE_COUNT];
int_mv frame_mv[MB_MODE_COUNT][MAX_REF_FRAMES];
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
int64_t best_label_rd = INT64_MAX, best_other_rd = INT64_MAX;
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
MB_PREDICTION_MODE mode_selected = ZEROMV;
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
int bestlabelyrate = 0;
i = idy * 2 + idx;
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
frame_mv[ZEROMV][mbmi->ref_frame].as_int = 0;
frame_mv[ZEROMV][mbmi->second_ref_frame].as_int = 0;
vp9_append_sub8x8_mvs_for_idx(&cpi->common, &x->e_mbd,
&frame_mv[NEARESTMV][mbmi->ref_frame],
&frame_mv[NEARMV][mbmi->ref_frame],
i, 0);
if (mbmi->second_ref_frame > 0)
vp9_append_sub8x8_mvs_for_idx(&cpi->common, &x->e_mbd,
&frame_mv[NEARESTMV][mbmi->second_ref_frame],
&frame_mv[NEARMV][mbmi->second_ref_frame],
i, 1);
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
// search for the best motion vector on this segment
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
for (this_mode = NEARESTMV; this_mode <= NEWMV; ++this_mode) {
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
int64_t this_rd;
int distortion;
int labelyrate;
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
ENTROPY_CONTEXT t_above_s[4], t_left_s[4];
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
vpx_memcpy(t_above_s, t_above, sizeof(t_above_s));
vpx_memcpy(t_left_s, t_left, sizeof(t_left_s));
// motion search for newmv (single predictor case only)
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
if (mbmi->second_ref_frame <= 0 && this_mode == NEWMV) {
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
int step_param = 0;
int further_steps;
int thissme, bestsme = INT_MAX;
const struct buf_2d orig_src = x->plane[0].src;
const struct buf_2d orig_pre = x->e_mbd.plane[0].pre[0];
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
int sadpb = x->sadperbit4;
int_mv mvp_full;
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
/* Is the best so far sufficiently good that we cant justify doing
* and new motion search. */
if (best_label_rd < label_mv_thresh)
break;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
if (cpi->compressor_speed) {
// use previous block's result as next block's MV predictor.
if (i > 0) {
bsi->mvp.as_int =
x->e_mbd.mode_info_context->bmi[i - 1].as_mv[0].as_int;
if (i == 2)
bsi->mvp.as_int =
x->e_mbd.mode_info_context->bmi[i - 2].as_mv[0].as_int;
step_param = 2;
}
}
further_steps = (MAX_MVSEARCH_STEPS - 1) - step_param;
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
mvp_full.as_mv.row = bsi->mvp.as_mv.row >> 3;
mvp_full.as_mv.col = bsi->mvp.as_mv.col >> 3;
// adjust src pointer for this segment
x->plane[0].src.buf =
raster_block_offset_uint8(&x->e_mbd, BLOCK_SIZE_SB8X8, 0, i,
x->plane[0].src.buf,
x->plane[0].src.stride);
assert(((intptr_t)x->e_mbd.plane[0].pre[0].buf & 0x7) == 0);
x->e_mbd.plane[0].pre[0].buf =
raster_block_offset_uint8(&x->e_mbd, BLOCK_SIZE_SB8X8, 0, i,
x->e_mbd.plane[0].pre[0].buf,
x->e_mbd.plane[0].pre[0].stride);
bestsme = vp9_full_pixel_diamond(cpi, x, &mvp_full, step_param,
sadpb, further_steps, 0, v_fn_ptr,
bsi->ref_mv, &mode_mv[NEWMV]);
// Should we do a full search (best quality only)
if (cpi->compressor_speed == 0) {
/* Check if mvp_full is within the range. */
clamp_mv(&mvp_full, x->mv_col_min, x->mv_col_max,
x->mv_row_min, x->mv_row_max);
thissme = cpi->full_search_sad(x, &mvp_full,
sadpb, 16, v_fn_ptr,
x->nmvjointcost, x->mvcost,
bsi->ref_mv, i);
if (thissme < bestsme) {
bestsme = thissme;
mode_mv[NEWMV].as_int =
x->e_mbd.mode_info_context->bmi[i].as_mv[0].as_int;
} else {
/* The full search result is actually worse so re-instate the
* previous best vector */
x->e_mbd.mode_info_context->bmi[i].as_mv[0].as_int =
mode_mv[NEWMV].as_int;
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
}
}
if (bestsme < INT_MAX) {
int distortion;
unsigned int sse;
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
cpi->find_fractional_mv_step(x, &mode_mv[NEWMV],
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
bsi->ref_mv, x->errorperbit, v_fn_ptr,
x->nmvjointcost, x->mvcost,
&distortion, &sse);
// safe motion search result for use in compound prediction
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
seg_mvs[i][mbmi->ref_frame - 1].as_int = mode_mv[NEWMV].as_int;
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
}
// restore src pointers
x->plane[0].src = orig_src;
x->e_mbd.plane[0].pre[0] = orig_pre;
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
} else if (mbmi->second_ref_frame > 0 && this_mode == NEWMV) {
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
if (seg_mvs[i][mbmi->second_ref_frame - 1].as_int == INVALID_MV ||
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
seg_mvs[i][mbmi->ref_frame - 1].as_int == INVALID_MV)
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
continue;
}
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
rate = labels2mode(x, i, this_mode, &mode_mv[this_mode],
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
&second_mode_mv[this_mode], frame_mv, seg_mvs[i],
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
bsi->ref_mv, bsi->second_ref_mv, x->nmvjointcost,
x->mvcost, cpi);
// Trap vectors that reach beyond the UMV borders
if (((mode_mv[this_mode].as_mv.row >> 3) < x->mv_row_min) ||
((mode_mv[this_mode].as_mv.row >> 3) > x->mv_row_max) ||
((mode_mv[this_mode].as_mv.col >> 3) < x->mv_col_min) ||
((mode_mv[this_mode].as_mv.col >> 3) > x->mv_col_max)) {
continue;
}
if (mbmi->second_ref_frame > 0 &&
mv_check_bounds(x, &second_mode_mv[this_mode]))
continue;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
this_rd = encode_inter_mb_segment(&cpi->common,
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
x, i, &labelyrate,
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
&distortion, t_above_s, t_left_s);
this_rd += RDCOST(x->rdmult, x->rddiv, rate, 0);
rate += labelyrate;
if (this_rd < best_label_rd) {
sbr = rate;
sbd = distortion;
bestlabelyrate = labelyrate;
mode_selected = this_mode;
best_label_rd = this_rd;
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
best_eobs[i] = x->e_mbd.plane[0].eobs[i];
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
vpx_memcpy(t_above_b, t_above_s, sizeof(t_above_s));
vpx_memcpy(t_left_b, t_left_s, sizeof(t_left_s));
}
} /*for each 4x4 mode*/
vpx_memcpy(t_above, t_above_b, sizeof(t_above));
vpx_memcpy(t_left, t_left_b, sizeof(t_left));
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
labels2mode(x, i, mode_selected, &mode_mv[mode_selected],
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
&second_mode_mv[mode_selected], frame_mv, seg_mvs[i],
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
bsi->ref_mv, bsi->second_ref_mv, x->nmvjointcost,
x->mvcost, cpi);
br += sbr;
bd += sbd;
segmentyrate += bestlabelyrate;
this_segment_rd += best_label_rd;
other_segment_rd += best_other_rd;
for (j = 1; j < bh; ++j)
vpx_memcpy(&x->partition_info->bmi[i + j * 2],
&x->partition_info->bmi[i],
sizeof(x->partition_info->bmi[i]));
for (j = 1; j < bw; ++j)
vpx_memcpy(&x->partition_info->bmi[i + j],
&x->partition_info->bmi[i],
sizeof(x->partition_info->bmi[i]));
}
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
} /* for each label */
if (this_segment_rd < bsi->segment_rd) {
bsi->r = br;
bsi->d = bd;
bsi->segment_yrate = segmentyrate;
bsi->segment_rd = this_segment_rd;
// store everything needed to come back to this!!
for (i = 0; i < 4; i++) {
bsi->mvs[i].as_mv = x->partition_info->bmi[i].mv.as_mv;
if (mbmi->second_ref_frame > 0)
bsi->second_mvs[i].as_mv = x->partition_info->bmi[i].second_mv.as_mv;
bsi->modes[i] = x->partition_info->bmi[i].mode;
bsi->eobs[i] = best_eobs[i];
}
Make superblocks independent of macroblock code and data. Split macroblock and superblock tokenization and detokenization functions and coefficient-related data structs so that the bitstream layout and related code of superblock coefficients looks less like it's a hack to fit macroblocks in superblocks. In addition, unify chroma transform size selection from luma transform size (i.e. always use the same size, as long as it fits the predictor); in practice, this means 32x32 and 64x64 superblocks using the 16x16 luma transform will now use the 16x16 (instead of the 8x8) chroma transform, and 64x64 superblocks using the 32x32 luma transform will now use the 32x32 (instead of the 16x16) chroma transform. Lastly, add a trellis optimize function for 32x32 transform blocks. HD gains about 0.3%, STDHD about 0.15% and derf about 0.1%. There's a few negative points here and there that I might want to analyze a little closer. Change-Id: Ibad7c3ddfe1acfc52771dfc27c03e9783e054430
2013-03-04 23:12:17 +01:00
}
}
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
static void rd_check_segment(VP9_COMP *cpi, MACROBLOCK *x,
BEST_SEG_INFO *bsi,
int_mv seg_mvs[4][MAX_REF_FRAMES - 1]) {
rd_check_segment_txsize(cpi, x, bsi, seg_mvs);
Make superblocks independent of macroblock code and data. Split macroblock and superblock tokenization and detokenization functions and coefficient-related data structs so that the bitstream layout and related code of superblock coefficients looks less like it's a hack to fit macroblocks in superblocks. In addition, unify chroma transform size selection from luma transform size (i.e. always use the same size, as long as it fits the predictor); in practice, this means 32x32 and 64x64 superblocks using the 16x16 luma transform will now use the 16x16 (instead of the 8x8) chroma transform, and 64x64 superblocks using the 32x32 luma transform will now use the 32x32 (instead of the 16x16) chroma transform. Lastly, add a trellis optimize function for 32x32 transform blocks. HD gains about 0.3%, STDHD about 0.15% and derf about 0.1%. There's a few negative points here and there that I might want to analyze a little closer. Change-Id: Ibad7c3ddfe1acfc52771dfc27c03e9783e054430
2013-03-04 23:12:17 +01:00
}
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
static int rd_pick_best_mbsegmentation(VP9_COMP *cpi, MACROBLOCK *x,
int_mv *best_ref_mv,
int_mv *second_best_ref_mv,
int64_t best_rd,
int *returntotrate,
int *returnyrate,
int *returndistortion,
int *skippable, int mvthresh,
int_mv seg_mvs[4][MAX_REF_FRAMES - 1]) {
int i;
BEST_SEG_INFO bsi;
MB_MODE_INFO * mbmi = &x->e_mbd.mode_info_context->mbmi;
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
vpx_memset(&bsi, 0, sizeof(bsi));
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
bsi.segment_rd = best_rd;
bsi.ref_mv = best_ref_mv;
bsi.second_ref_mv = second_best_ref_mv;
bsi.mvp.as_int = best_ref_mv->as_int;
bsi.mvthresh = mvthresh;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
for (i = 0; i < 4; i++)
Remove splitmv. Also do per-partition motion vector referencing in <sb8x8 partitions, and adjust mvref finding for sub8x8 partitions. Change-Id: Id3ed1ed4d2a8910d11d327db6cc63b8eb79f941f
2013-05-26 23:40:49 +02:00
bsi.modes[i] = ZEROMV;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
rd_check_segment(cpi, x, &bsi, seg_mvs);
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
/* set it to the best */
for (i = 0; i < 4; i++) {
x->e_mbd.mode_info_context->bmi[i].as_mv[0].as_int = bsi.mvs[i].as_int;
if (mbmi->second_ref_frame > 0)
x->e_mbd.mode_info_context->bmi[i].as_mv[1].as_int =
bsi.second_mvs[i].as_int;
x->e_mbd.plane[0].eobs[i] = bsi.eobs[i];
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
/* save partitions */
x->partition_info->count = 4;
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
for (i = 0; i < x->partition_info->count; i++) {
x->partition_info->bmi[i].mode = bsi.modes[i];
x->partition_info->bmi[i].mv.as_mv = bsi.mvs[i].as_mv;
if (mbmi->second_ref_frame > 0)
x->partition_info->bmi[i].second_mv.as_mv = bsi.second_mvs[i].as_mv;
}
/*
* used to set mbmi->mv.as_int
*/
x->partition_info->bmi[3].mv.as_int = bsi.mvs[3].as_int;
if (mbmi->second_ref_frame > 0)
x->partition_info->bmi[3].second_mv.as_int = bsi.second_mvs[3].as_int;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
*returntotrate = bsi.r;
*returndistortion = bsi.d;
*returnyrate = bsi.segment_yrate;
*skippable = vp9_sby_is_skippable(&x->e_mbd, BLOCK_SIZE_SB8X8);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
return (int)(bsi.segment_rd);
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
static void mv_pred(VP9_COMP *cpi, MACROBLOCK *x,
uint8_t *ref_y_buffer, int ref_y_stride,
int ref_frame, enum BlockSize block_size ) {
MACROBLOCKD *xd = &x->e_mbd;
MB_MODE_INFO *mbmi = &xd->mode_info_context->mbmi;
int_mv this_mv;
int i;
int zero_seen = 0;
int best_index = 0;
int best_sad = INT_MAX;
int this_sad = INT_MAX;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
uint8_t *src_y_ptr = x->plane[0].src.buf;
uint8_t *ref_y_ptr;
int row_offset, col_offset;
Compound prediction for splitmv macroblocks. Change-Id: I0af3395500b1cb0ed629249eb6636a0c9322cb18
2012-04-18 22:51:58 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Get the sad for each candidate reference mv
for (i = 0; i < MAX_MV_REF_CANDIDATES; i++) {
this_mv.as_int = mbmi->ref_mvs[ref_frame][i].as_int;
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// The list is at an end if we see 0 for a second time.
if (!this_mv.as_int && zero_seen)
break;
zero_seen = zero_seen || !this_mv.as_int;
MODE_INFO size reduction Declared the bmi in MODE_INFO as a union instead of B_MODE_INFO. This reduced the memory footprint by 518,400 bytes for 1080 resolutions. The decoder performance improved by ~4% for the clip used and the encoder showed very small improvements. (0.5%) This reduction was first mentioned to me by John K. and in a later discussion by Yaowu. This is WIP. Change-Id: I8e175fdbc46d28c35277302a04bee4540efc8d29
2011-05-24 19:24:52 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
row_offset = this_mv.as_mv.row >> 3;
col_offset = this_mv.as_mv.col >> 3;
ref_y_ptr = ref_y_buffer + (ref_y_stride * row_offset) + col_offset;
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Find sad for current vector.
this_sad = cpi->fn_ptr[block_size].sdf(src_y_ptr, x->plane[0].src.stride,
ref_y_ptr, ref_y_stride,
0x7fffffff);
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Note if it is the best so far.
if (this_sad < best_sad) {
best_sad = this_sad;
best_index = i;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
}
Minor cosmetic changes to rdopt.c. Change-Id: Ib7b972486ccbbcb82fa4c05a5f6d7f8aaec32155
2012-08-07 01:21:23 +02:00
}
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Note the index of the mv that worked best in the reference list.
x->mv_best_ref_index[ref_frame] = best_index;
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
extern void vp9_calc_ref_probs(int *count, vp9_prob *probs);
static void estimate_curframe_refprobs(VP9_COMP *cpi,
vp9_prob mod_refprobs[3],
int pred_ref) {
int norm_cnt[MAX_REF_FRAMES];
const int *const rfct = cpi->count_mb_ref_frame_usage;
int intra_count = rfct[INTRA_FRAME];
int last_count = rfct[LAST_FRAME];
int gf_count = rfct[GOLDEN_FRAME];
int arf_count = rfct[ALTREF_FRAME];
Refactor inter recon functions to support scaling Ensure that all inter prediction goes through a common code path that takes scaling into account. Removes a bunch of duplicate 1st/2nd predictor code. Also introduces a 16x8 mode for 8x8 MVs, similar to the 8x4 trick we were doing before. This has an unexpected effect with EIGHTTAP_SMOOTH, so it's disabled in that case for now. Change-Id: Ia053e823a8bc616a988a0af30452e1e75a739cba
2013-02-09 02:49:44 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Work out modified reference frame probabilities to use where prediction
// of the reference frame fails
if (pred_ref == INTRA_FRAME) {
norm_cnt[0] = 0;
norm_cnt[1] = last_count;
norm_cnt[2] = gf_count;
norm_cnt[3] = arf_count;
vp9_calc_ref_probs(norm_cnt, mod_refprobs);
mod_refprobs[0] = 0; // This branch implicit
} else if (pred_ref == LAST_FRAME) {
norm_cnt[0] = intra_count;
norm_cnt[1] = 0;
norm_cnt[2] = gf_count;
norm_cnt[3] = arf_count;
vp9_calc_ref_probs(norm_cnt, mod_refprobs);
mod_refprobs[1] = 0; // This branch implicit
} else if (pred_ref == GOLDEN_FRAME) {
norm_cnt[0] = intra_count;
norm_cnt[1] = last_count;
norm_cnt[2] = 0;
norm_cnt[3] = arf_count;
vp9_calc_ref_probs(norm_cnt, mod_refprobs);
mod_refprobs[2] = 0; // This branch implicit
} else {
norm_cnt[0] = intra_count;
norm_cnt[1] = last_count;
norm_cnt[2] = gf_count;
norm_cnt[3] = 0;
vp9_calc_ref_probs(norm_cnt, mod_refprobs);
mod_refprobs[2] = 0; // This branch implicit
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
}
Initial WebM release
2010-05-18 17:58:33 +02:00
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
static INLINE unsigned weighted_cost(vp9_prob *tab0, vp9_prob *tab1,
int idx, int val, int weight) {
unsigned cost0 = tab0[idx] ? vp9_cost_bit(tab0[idx], val) : 0;
unsigned cost1 = tab1[idx] ? vp9_cost_bit(tab1[idx], val) : 0;
// weight is 16-bit fixed point, so this basically calculates:
// 0.5 + weight * cost1 + (1.0 - weight) * cost0
return (0x8000 + weight * cost1 + (0x10000 - weight) * cost0) >> 16;
}
static void estimate_ref_frame_costs(VP9_COMP *cpi, int segment_id,
unsigned int *ref_costs) {
VP9_COMMON *cm = &cpi->common;
MACROBLOCKD *xd = &cpi->mb.e_mbd;
vp9_prob *mod_refprobs;
unsigned int cost;
int pred_ref;
int pred_flag;
int pred_ctx;
int i;
vp9_prob pred_prob, new_pred_prob;
int seg_ref_active;
int seg_ref_count = 0;
seg_ref_active = vp9_segfeature_active(xd,
segment_id,
SEG_LVL_REF_FRAME);
8x8 transform support in splitmv. For splitmv, where partitioning is 8x16, 16x8 or 8x8, this patch uses the 8x8 transform (instead of the 4x4) if txfm_mode is ALLOW_8X8 or ALLOW_16X16. For TX_MODE_SELECT, splitmv can indicate which of the 2 transform sizes (4x4 or 8x8) it wants to use. Gains (with hybridtx4x4/8x8/16x16 and tx_select experiments enabled) on derf: +0.9%, HD: +0.4%, STD/HD: +0.8% (SSIM or overall PSNR, both metrics show similar improvements). Change-Id: Ide954b000b415548ed92a7ac78e24f36e60fcf06
2012-10-22 20:49:00 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if (seg_ref_active) {
seg_ref_count = vp9_check_segref(xd, segment_id, INTRA_FRAME) +
vp9_check_segref(xd, segment_id, LAST_FRAME) +
vp9_check_segref(xd, segment_id, GOLDEN_FRAME) +
vp9_check_segref(xd, segment_id, ALTREF_FRAME);
8x8 transform support in splitmv. For splitmv, where partitioning is 8x16, 16x8 or 8x8, this patch uses the 8x8 transform (instead of the 4x4) if txfm_mode is ALLOW_8X8 or ALLOW_16X16. For TX_MODE_SELECT, splitmv can indicate which of the 2 transform sizes (4x4 or 8x8) it wants to use. Gains (with hybridtx4x4/8x8/16x16 and tx_select experiments enabled) on derf: +0.9%, HD: +0.4%, STD/HD: +0.8% (SSIM or overall PSNR, both metrics show similar improvements). Change-Id: Ide954b000b415548ed92a7ac78e24f36e60fcf06
2012-10-22 20:49:00 +02:00
}
Use 8x8 block bases for non-4x4 splitmv RD. Change-Id: I76095d5a02edcc5d404ea830157780b24fd509d7
2012-10-17 20:40:00 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Get the predicted reference for this mb
pred_ref = vp9_get_pred_ref(cm, xd);
Use 8x8 block bases for non-4x4 splitmv RD. Change-Id: I76095d5a02edcc5d404ea830157780b24fd509d7
2012-10-17 20:40:00 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Get the context probability for the prediction flag (based on last frame)
pred_prob = vp9_get_pred_prob(cm, xd, PRED_REF);
Refactor inter recon functions to support scaling Ensure that all inter prediction goes through a common code path that takes scaling into account. Removes a bunch of duplicate 1st/2nd predictor code. Also introduces a 16x8 mode for 8x8 MVs, similar to the 8x4 trick we were doing before. This has an unexpected effect with EIGHTTAP_SMOOTH, so it's disabled in that case for now. Change-Id: Ia053e823a8bc616a988a0af30452e1e75a739cba
2013-02-09 02:49:44 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Predict probability for current frame based on stats so far
pred_ctx = vp9_get_pred_context(cm, xd, PRED_REF);
new_pred_prob = get_binary_prob(cpi->ref_pred_count[pred_ctx][0],
cpi->ref_pred_count[pred_ctx][1]);
Use 8x8 block bases for non-4x4 splitmv RD. Change-Id: I76095d5a02edcc5d404ea830157780b24fd509d7
2012-10-17 20:40:00 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Get the set of probabilities to use if prediction fails
mod_refprobs = cm->mod_refprobs[pred_ref];
// For each possible selected reference frame work out a cost.
for (i = 0; i < MAX_REF_FRAMES; i++) {
if (seg_ref_active && seg_ref_count == 1) {
cost = 0;
} else {
pred_flag = (i == pred_ref);
// Get the prediction for the current mb
cost = weighted_cost(&pred_prob, &new_pred_prob, 0,
pred_flag, cpi->seg0_progress);
if (cost > 1024) cost = 768; // i.e. account for 4 bits max.
// for incorrectly predicted cases
if (!pred_flag) {
vp9_prob curframe_mod_refprobs[3];
if (cpi->seg0_progress) {
estimate_curframe_refprobs(cpi, curframe_mod_refprobs, pred_ref);
} else {
vpx_memset(curframe_mod_refprobs, 0, sizeof(curframe_mod_refprobs));
8x8 transform support in splitmv. For splitmv, where partitioning is 8x16, 16x8 or 8x8, this patch uses the 8x8 transform (instead of the 4x4) if txfm_mode is ALLOW_8X8 or ALLOW_16X16. For TX_MODE_SELECT, splitmv can indicate which of the 2 transform sizes (4x4 or 8x8) it wants to use. Gains (with hybridtx4x4/8x8/16x16 and tx_select experiments enabled) on derf: +0.9%, HD: +0.4%, STD/HD: +0.8% (SSIM or overall PSNR, both metrics show similar improvements). Change-Id: Ide954b000b415548ed92a7ac78e24f36e60fcf06
2012-10-22 20:49:00 +02:00
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
cost += weighted_cost(mod_refprobs, curframe_mod_refprobs, 0,
(i != INTRA_FRAME), cpi->seg0_progress);
if (i != INTRA_FRAME) {
cost += weighted_cost(mod_refprobs, curframe_mod_refprobs, 1,
(i != LAST_FRAME), cpi->seg0_progress);
if (i != LAST_FRAME) {
cost += weighted_cost(mod_refprobs, curframe_mod_refprobs, 2,
(i != GOLDEN_FRAME), cpi->seg0_progress);
8x8 transform support in splitmv. For splitmv, where partitioning is 8x16, 16x8 or 8x8, this patch uses the 8x8 transform (instead of the 4x4) if txfm_mode is ALLOW_8X8 or ALLOW_16X16. For TX_MODE_SELECT, splitmv can indicate which of the 2 transform sizes (4x4 or 8x8) it wants to use. Gains (with hybridtx4x4/8x8/16x16 and tx_select experiments enabled) on derf: +0.9%, HD: +0.4%, STD/HD: +0.8% (SSIM or overall PSNR, both metrics show similar improvements). Change-Id: Ide954b000b415548ed92a7ac78e24f36e60fcf06
2012-10-22 20:49:00 +02:00
}
}
Use 8x8 block bases for non-4x4 splitmv RD. Change-Id: I76095d5a02edcc5d404ea830157780b24fd509d7
2012-10-17 20:40:00 +02:00
}
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
ref_costs[i] = cost;
Use 8x8 block bases for non-4x4 splitmv RD. Change-Id: I76095d5a02edcc5d404ea830157780b24fd509d7
2012-10-17 20:40:00 +02:00
}
}
fix build issues for experimental branch experimental branch build was broken from some merge artifacts, this commit fixes those issues to enable the experimental branch to build. Change-Id: Ic52b2d2f1d1b80abb7ecaa4c0927bcf887ac0c2a
2011-06-23 00:07:04 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
static void store_coding_context(MACROBLOCK *x, PICK_MODE_CONTEXT *ctx,
int mode_index,
PARTITION_INFO *partition,
int_mv *ref_mv,
int_mv *second_ref_mv,
int64_t comp_pred_diff[NB_PREDICTION_TYPES],
int64_t txfm_size_diff[NB_TXFM_MODES]) {
MACROBLOCKD *const xd = &x->e_mbd;
vp8_rd_pick_best_mbsegmentation code restructure Moved the code from the segmentation loop into a function which is now called for each segment. This will allow us to change the segment order checking more easily. Change-Id: I9510d26f0acae5a73043fcca8f1984b121d3e052
2010-12-06 22:42:52 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Take a snapshot of the coding context so it can be
// restored if we decide to encode this way
ctx->skip = x->skip;
ctx->best_mode_index = mode_index;
vpx_memcpy(&ctx->mic, xd->mode_info_context,
sizeof(MODE_INFO));
if (partition)
vpx_memcpy(&ctx->partition_info, partition,
sizeof(PARTITION_INFO));
ctx->best_ref_mv.as_int = ref_mv->as_int;
ctx->second_best_ref_mv.as_int = second_ref_mv->as_int;
vp8_rd_pick_best_mbsegmentation code restructure Moved the code from the segmentation loop into a function which is now called for each segment. This will allow us to change the segment order checking more easily. Change-Id: I9510d26f0acae5a73043fcca8f1984b121d3e052
2010-12-06 22:42:52 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
ctx->single_pred_diff = (int)comp_pred_diff[SINGLE_PREDICTION_ONLY];
ctx->comp_pred_diff = (int)comp_pred_diff[COMP_PREDICTION_ONLY];
ctx->hybrid_pred_diff = (int)comp_pred_diff[HYBRID_PREDICTION];
vp8_rd_pick_best_mbsegmentation code restructure Moved the code from the segmentation loop into a function which is now called for each segment. This will allow us to change the segment order checking more easily. Change-Id: I9510d26f0acae5a73043fcca8f1984b121d3e052
2010-12-06 22:42:52 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
memcpy(ctx->txfm_rd_diff, txfm_size_diff, sizeof(ctx->txfm_rd_diff));
}
vp8_rd_pick_best_mbsegmentation code restructure Moved the code from the segmentation loop into a function which is now called for each segment. This will allow us to change the segment order checking more easily. Change-Id: I9510d26f0acae5a73043fcca8f1984b121d3e052
2010-12-06 22:42:52 +01:00
Make setup_pred_block subsampling-aware. Code previously set up the pointers by scaling by MI_UV_SIZE, which is 4:2:0 only. Change-Id: Ic13a92895cff018ec1345736746ed84cb31e6e31
2013-05-07 20:31:12 +02:00
static void setup_pred_block(const MACROBLOCKD *xd,
struct buf_2d dst[MAX_MB_PLANE],
const YV12_BUFFER_CONFIG *src,
int mi_row, int mi_col,
const struct scale_factors *scale,
const struct scale_factors *scale_uv) {
int i;
dst[0].buf = src->y_buffer;
dst[0].stride = src->y_stride;
dst[1].buf = src->u_buffer;
dst[2].buf = src->v_buffer;
dst[1].stride = dst[2].stride = src->uv_stride;
Initial version of alpha channel support This is a mostly-working implementation of an extra channel in the bitstream. Configure with --enable-alpha to test. Notable TODOs: - Add extra channel to all mismatch tests, PSNR, SSIM, etc - Configurable subsampling - Variable number of planes (currently always uses all 4) - Loop filtering - Per-plane lossless quantizer - ARNR support This implementation just uses the same contents as the Y channel for the A channel, due to lack of content and general pain in playing back 4 channel content. A later patch will use the actual alpha channel passed in from outside the codec. Change-Id: Ibf81f023b1c570bd84b3064e9b4b8ae52e087592
2013-05-16 02:55:08 +02:00
#if CONFIG_ALPHA
dst[3].buf = src->alpha_buffer;
dst[3].stride = src->alpha_stride;
#endif
Make setup_pred_block subsampling-aware. Code previously set up the pointers by scaling by MI_UV_SIZE, which is 4:2:0 only. Change-Id: Ic13a92895cff018ec1345736746ed84cb31e6e31
2013-05-07 20:31:12 +02:00
// TODO(jkoleszar): Make scale factors per-plane data
for (i = 0; i < MAX_MB_PLANE; i++) {
setup_pred_plane(dst + i, dst[i].buf, dst[i].stride, mi_row, mi_col,
i ? scale_uv : scale,
xd->plane[i].subsampling_x, xd->plane[i].subsampling_y);
}
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
static void setup_buffer_inter(VP9_COMP *cpi, MACROBLOCK *x,
int idx, MV_REFERENCE_FRAME frame_type,
enum BlockSize block_size,
int mi_row, int mi_col,
int_mv frame_nearest_mv[MAX_REF_FRAMES],
int_mv frame_near_mv[MAX_REF_FRAMES],
Make setup_pred_block subsampling-aware. Code previously set up the pointers by scaling by MI_UV_SIZE, which is 4:2:0 only. Change-Id: Ic13a92895cff018ec1345736746ed84cb31e6e31
2013-05-07 20:31:12 +02:00
struct buf_2d yv12_mb[4][MAX_MB_PLANE],
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
struct scale_factors scale[MAX_REF_FRAMES]) {
VP9_COMMON *cm = &cpi->common;
YV12_BUFFER_CONFIG *yv12 = &cm->yv12_fb[cpi->common.ref_frame_map[idx]];
MACROBLOCKD *const xd = &x->e_mbd;
MB_MODE_INFO *const mbmi = &xd->mode_info_context->mbmi;
int use_prev_in_find_mv_refs;
Modify motion estimation for SPLITMV mode 1. Search for block8x16/block16x8 uses block8x8's search results. 2. Check block4x4 only if block8x8 is chosen. (This hurts quality, which will be improved in another check-in.) 3. In block4x4 search, the previous block's result is used as MV predictor for next block. This change improves performance. Change-Id: I9dc089007ca08129fb6c11fe7692777ebb8647b0
2010-12-23 17:23:03 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// set up scaling factors
scale[frame_type] = cpi->common.active_ref_scale[frame_type - 1];
scale[frame_type].x_offset_q4 =
(mi_col * MI_SIZE * scale[frame_type].x_num /
scale[frame_type].x_den) & 0xf;
scale[frame_type].y_offset_q4 =
(mi_row * MI_SIZE * scale[frame_type].y_num /
scale[frame_type].y_den) & 0xf;
vp8_rd_pick_best_mbsegmentation code restructure Moved the code from the segmentation loop into a function which is now called for each segment. This will allow us to change the segment order checking more easily. Change-Id: I9510d26f0acae5a73043fcca8f1984b121d3e052
2010-12-06 22:42:52 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// TODO(jkoleszar): Is the UV buffer ever used here? If so, need to make this
// use the UV scaling factors.
Make setup_pred_block subsampling-aware. Code previously set up the pointers by scaling by MI_UV_SIZE, which is 4:2:0 only. Change-Id: Ic13a92895cff018ec1345736746ed84cb31e6e31
2013-05-07 20:31:12 +02:00
setup_pred_block(xd, yv12_mb[frame_type], yv12, mi_row, mi_col,
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
&scale[frame_type], &scale[frame_type]);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Gets an initial list of candidate vectors from neighbours and orders them
use_prev_in_find_mv_refs = cm->width == cm->last_width &&
cm->height == cm->last_height &&
!cpi->common.error_resilient_mode;
vp9_find_mv_refs(&cpi->common, xd, xd->mode_info_context,
use_prev_in_find_mv_refs ? xd->prev_mode_info_context : NULL,
frame_type,
mbmi->ref_mvs[frame_type],
cpi->common.ref_frame_sign_bias);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Candidate refinement carried out at encoder and decoder
vp9_find_best_ref_mvs(xd,
mbmi->ref_mvs[frame_type],
&frame_nearest_mv[frame_type],
&frame_near_mv[frame_type]);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Further refinement that is encode side only to test the top few candidates
// in full and choose the best as the centre point for subsequent searches.
// The current implementation doesn't support scaling.
if (scale[frame_type].x_num == scale[frame_type].x_den &&
scale[frame_type].y_num == scale[frame_type].y_den)
Make setup_pred_block subsampling-aware. Code previously set up the pointers by scaling by MI_UV_SIZE, which is 4:2:0 only. Change-Id: Ic13a92895cff018ec1345736746ed84cb31e6e31
2013-05-07 20:31:12 +02:00
mv_pred(cpi, x, yv12_mb[frame_type][0].buf, yv12->y_stride,
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
frame_type, block_size);
}
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
static void model_rd_from_var_lapndz(int var, int n, int qstep,
int *rate, int *dist) {
// This function models the rate and distortion for a Laplacian
// source with given variance when quantized with a uniform quantizer
// with given stepsize. The closed form expressions are in:
// Hang and Chen, "Source Model for transform video coder and its
// application - Part I: Fundamental Theory", IEEE Trans. Circ.
// Sys. for Video Tech., April 1997.
// The function is implemented as piecewise approximation to the
// exact computation.
// TODO(debargha): Implement the functions by interpolating from a
// look-up table
vp9_clear_system_state();
{
double D, R;
double s2 = (double) var / n;
double s = sqrt(s2);
double x = qstep / s;
if (x > 1.0) {
double y = exp(-x / 2);
double y2 = y * y;
D = 2.069981728764738 * y2 - 2.764286806516079 * y + 1.003956960819275;
R = 0.924056758535089 * y2 + 2.738636469814024 * y - 0.005169662030017;
} else {
double x2 = x * x;
D = 0.075303187668830 * x2 + 0.004296954321112 * x - 0.000413209252807;
if (x > 0.125)
R = 1 / (-0.03459733614226 * x2 + 0.36561675733603 * x +
0.1626989668625);
else
R = -1.442252874826093 * log(x) + 1.944647760719664;
}
if (R < 0) {
*rate = 0;
*dist = var;
} else {
*rate = (n * R * 256 + 0.5);
*dist = (n * D * s2 + 0.5);
}
}
vp9_clear_system_state();
}
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Removing y_to_uv_block_size and y_bsizet_to_block_size functions. Change-Id: I49527ff8dd8bef1074c18a964fed2a575f0b118a
2013-05-09 00:23:42 +02:00
static enum BlockSize get_plane_block_size(BLOCK_SIZE_TYPE bsize,
struct macroblockd_plane *pd) {
const int bwl = b_width_log2(bsize) - pd->subsampling_x;
const int bhl = b_height_log2(bsize) - pd->subsampling_y;
return get_block_size(4 << bwl, 4 << bhl);
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
static void model_rd_for_sb(VP9_COMP *cpi, BLOCK_SIZE_TYPE bsize,
MACROBLOCK *x, MACROBLOCKD *xd,
int *out_rate_sum, int *out_dist_sum) {
// Note our transform coeffs are 8 times an orthogonal transform.
// Hence quantizer step is also 8 times. To get effective quantizer
// we need to divide by 8 before sending to modeling function.
unsigned int sse, var;
int i, rate_sum = 0, dist_sum = 0;
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
for (i = 0; i < MAX_MB_PLANE; ++i) {
struct macroblock_plane *const p = &x->plane[i];
struct macroblockd_plane *const pd = &xd->plane[i];
Add half-pixel variance RTCD functions NEON has optimized 16x16 half-pixel variance functions, but they were not part of the RTCD framework. Add these functions to RTCD, so that other platforms can make use of this optimization in the future and special-case ARM code can be removed. A number of functions were taking two variance functions as parameters. These functions were changed to take a single parameter, a pointer to a struct containing all the variance functions for that block size. This provides additional flexibility for calling additional variance functions (the half-pixel special case, for example) and by initializing the table for all block sizes, we don't have to construct this function pointer table for each macroblock. Change-Id: I78289ff36b2715f9a7aa04d5f6fbe3d23acdc29c
2010-10-26 21:34:16 +02:00
Removing y_to_uv_block_size and y_bsizet_to_block_size functions. Change-Id: I49527ff8dd8bef1074c18a964fed2a575f0b118a
2013-05-09 00:23:42 +02:00
// TODO(dkovalev) the same code in get_plane_block_size
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
const int bwl = b_width_log2(bsize) - pd->subsampling_x;
const int bhl = b_height_log2(bsize) - pd->subsampling_y;
const enum BlockSize bs = get_block_size(4 << bwl, 4 << bhl);
int rate, dist;
var = cpi->fn_ptr[bs].vf(p->src.buf, p->src.stride,
pd->dst.buf, pd->dst.stride, &sse);
model_rd_from_var_lapndz(var, 16 << (bwl + bhl),
pd->dequant[1] >> 3, &rate, &dist);
vp8_rd_pick_best_mbsegmentation code restructure Moved the code from the segmentation loop into a function which is now called for each segment. This will allow us to change the segment order checking more easily. Change-Id: I9510d26f0acae5a73043fcca8f1984b121d3e052
2010-12-06 22:42:52 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
rate_sum += rate;
dist_sum += dist;
}
vp8_rd_pick_best_mbsegmentation code restructure Moved the code from the segmentation loop into a function which is now called for each segment. This will allow us to change the segment order checking more easily. Change-Id: I9510d26f0acae5a73043fcca8f1984b121d3e052
2010-12-06 22:42:52 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
*out_rate_sum = rate_sum;
*out_dist_sum = dist_sum;
}
Initial WebM release
2010-05-18 17:58:33 +02:00
Adding get_switchable_rate function. Change-Id: I71311a14f8d7f48508b250f25d1d0914c6a1ac72
2013-05-07 21:23:14 +02:00
static INLINE int get_switchable_rate(VP9_COMMON *cm, MACROBLOCK *x) {
MACROBLOCKD *xd = &x->e_mbd;
MB_MODE_INFO *const mbmi = &xd->mode_info_context->mbmi;
const int c = vp9_get_pred_context(cm, xd, PRED_SWITCHABLE_INTERP);
const int m = vp9_switchable_interp_map[mbmi->interp_filter];
return SWITCHABLE_INTERP_RATE_FACTOR * x->switchable_interp_costs[c][m];
}
Refactor rd loop for inter modes This commit pulls the iterative motion search for compound inter- inter out from handle_inter_mode_ as a separate function. Hence, it is applicable to 4x4/4x8/8x4 level compound inter search to be enabled later. Also edit the rd loop for 4x4 inter block sizes for cosmetic purpose. Change-Id: Ibc71a11cbe5a26cd52faba01026cf8446cf4d2b4
2013-05-28 23:02:29 +02:00
static void iterative_motion_search(VP9_COMP *cpi, MACROBLOCK *x,
BLOCK_SIZE_TYPE bsize,
int_mv *frame_mv,
YV12_BUFFER_CONFIG **scaled_ref_frame,
int mi_row, int mi_col,
int_mv single_newmv[MAX_REF_FRAMES]) {
int pw = 4 << b_width_log2(bsize), ph = 4 << b_height_log2(bsize);
MACROBLOCKD *xd = &x->e_mbd;
MB_MODE_INFO *mbmi = &xd->mode_info_context->mbmi;
int refs[2] = { mbmi->ref_frame,
(mbmi->second_ref_frame < 0 ? 0 : mbmi->second_ref_frame) };
int_mv ref_mv[2];
const enum BlockSize block_size = get_plane_block_size(bsize, &xd->plane[0]);
int ite;
// Prediction buffer from second frame.
uint8_t *second_pred = vpx_memalign(16, pw * ph * sizeof(uint8_t));
// Do joint motion search in compound mode to get more accurate mv.
struct buf_2d backup_yv12[MAX_MB_PLANE] = {{0}};
struct buf_2d backup_second_yv12[MAX_MB_PLANE] = {{0}};
struct buf_2d scaled_first_yv12;
int last_besterr[2] = {INT_MAX, INT_MAX};
ref_mv[0] = mbmi->ref_mvs[refs[0]][0];
ref_mv[1] = mbmi->ref_mvs[refs[1]][0];
if (scaled_ref_frame[0]) {
int i;
// Swap out the reference frame for a version that's been scaled to
// match the resolution of the current frame, allowing the existing
// motion search code to be used without additional modifications.
for (i = 0; i < MAX_MB_PLANE; i++)
backup_yv12[i] = xd->plane[i].pre[0];
setup_pre_planes(xd, scaled_ref_frame[0], NULL, mi_row, mi_col,
NULL, NULL);
}
if (scaled_ref_frame[1]) {
int i;
for (i = 0; i < MAX_MB_PLANE; i++)
backup_second_yv12[i] = xd->plane[i].pre[1];
setup_pre_planes(xd, scaled_ref_frame[1], NULL, mi_row, mi_col,
NULL, NULL);
}
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
Refactor rd loop for inter modes This commit pulls the iterative motion search for compound inter- inter out from handle_inter_mode_ as a separate function. Hence, it is applicable to 4x4/4x8/8x4 level compound inter search to be enabled later. Also edit the rd loop for 4x4 inter block sizes for cosmetic purpose. Change-Id: Ibc71a11cbe5a26cd52faba01026cf8446cf4d2b4
2013-05-28 23:02:29 +02:00
xd->scale_factor[0].set_scaled_offsets(&xd->scale_factor[0],
mi_row, mi_col);
xd->scale_factor[1].set_scaled_offsets(&xd->scale_factor[1],
mi_row, mi_col);
scaled_first_yv12 = xd->plane[0].pre[0];
// Initialize mv using single prediction mode result.
frame_mv[refs[0]].as_int = single_newmv[refs[0]].as_int;
frame_mv[refs[1]].as_int = single_newmv[refs[1]].as_int;
// Allow joint search multiple times iteratively for each ref frame
// and break out the search loop if it couldn't find better mv.
for (ite = 0; ite < 4; ite++) {
struct buf_2d ref_yv12[2];
int bestsme = INT_MAX;
int sadpb = x->sadperbit16;
int_mv tmp_mv;
int search_range = 3;
int tmp_col_min = x->mv_col_min;
int tmp_col_max = x->mv_col_max;
int tmp_row_min = x->mv_row_min;
int tmp_row_max = x->mv_row_max;
int id = ite % 2;
// Initialized here because of compiler problem in Visual Studio.
ref_yv12[0] = xd->plane[0].pre[0];
ref_yv12[1] = xd->plane[0].pre[1];
// Get pred block from second frame.
vp9_build_inter_predictor(ref_yv12[!id].buf,
ref_yv12[!id].stride,
second_pred, pw,
&frame_mv[refs[!id]],
&xd->scale_factor[!id],
pw, ph, 0,
&xd->subpix);
// Compound motion search on first ref frame.
if (id)
xd->plane[0].pre[0] = ref_yv12[id];
vp9_clamp_mv_min_max(x, &ref_mv[id]);
// Use mv result from single mode as mvp.
tmp_mv.as_int = frame_mv[refs[id]].as_int;
tmp_mv.as_mv.col >>= 3;
tmp_mv.as_mv.row >>= 3;
// Small-range full-pixel motion search
bestsme = vp9_refining_search_8p_c(x, &tmp_mv, sadpb,
search_range,
&cpi->fn_ptr[block_size],
x->nmvjointcost, x->mvcost,
&ref_mv[id], second_pred,
pw, ph);
x->mv_col_min = tmp_col_min;
x->mv_col_max = tmp_col_max;
x->mv_row_min = tmp_row_min;
x->mv_row_max = tmp_row_max;
if (bestsme < INT_MAX) {
int dis; /* TODO: use dis in distortion calculation later. */
unsigned int sse;
bestsme = vp9_find_best_sub_pixel_comp(x, &tmp_mv,
&ref_mv[id],
x->errorperbit,
&cpi->fn_ptr[block_size],
x->nmvjointcost, x->mvcost,
&dis, &sse, second_pred,
pw, ph);
}
if (id)
xd->plane[0].pre[0] = scaled_first_yv12;
if (bestsme < last_besterr[id]) {
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
frame_mv[refs[id]].as_int = tmp_mv.as_int;
Refactor rd loop for inter modes This commit pulls the iterative motion search for compound inter- inter out from handle_inter_mode_ as a separate function. Hence, it is applicable to 4x4/4x8/8x4 level compound inter search to be enabled later. Also edit the rd loop for 4x4 inter block sizes for cosmetic purpose. Change-Id: Ibc71a11cbe5a26cd52faba01026cf8446cf4d2b4
2013-05-28 23:02:29 +02:00
last_besterr[id] = bestsme;
} else {
break;
}
}
// restore the predictor
if (scaled_ref_frame[0]) {
int i;
for (i = 0; i < MAX_MB_PLANE; i++)
xd->plane[i].pre[0] = backup_yv12[i];
}
if (scaled_ref_frame[1]) {
int i;
for (i = 0; i < MAX_MB_PLANE; i++)
xd->plane[i].pre[1] = backup_second_yv12[i];
}
vpx_free(second_pred);
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
static int64_t handle_inter_mode(VP9_COMP *cpi, MACROBLOCK *x,
BLOCK_SIZE_TYPE bsize,
Remove unused mdcounts mdcounts seems no longer used. Change-Id: Idd8162e8acfa3f5be7a18767156cc79ccbc2bdee
2013-05-09 01:27:35 +02:00
int64_t txfm_cache[],
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
int *rate2, int *distortion, int *skippable,
int *compmode_cost,
int *rate_y, int *distortion_y,
int *rate_uv, int *distortion_uv,
int *mode_excluded, int *disable_skip,
INTERPOLATIONFILTERTYPE *best_filter,
Decrease scope of frame_mv argument to handle_inter_mode(). Change-Id: I81c637c61ecc33cb66beb59a2a33166d66b9a0a2
2013-05-27 23:16:45 +02:00
int_mv *frame_mv,
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
YV12_BUFFER_CONFIG **scaled_ref_frame,
int mi_row, int mi_col,
int_mv single_newmv[MAX_REF_FRAMES]) {
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
const int bw = 1 << mi_width_log2(bsize), bh = 1 << mi_height_log2(bsize);
Removing y_to_uv_block_size and y_bsizet_to_block_size functions. Change-Id: I49527ff8dd8bef1074c18a964fed2a575f0b118a
2013-05-09 00:23:42 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
VP9_COMMON *cm = &cpi->common;
MACROBLOCKD *xd = &x->e_mbd;
Removing y_to_uv_block_size and y_bsizet_to_block_size functions. Change-Id: I49527ff8dd8bef1074c18a964fed2a575f0b118a
2013-05-09 00:23:42 +02:00
const enum BlockSize block_size = get_plane_block_size(bsize, &xd->plane[0]);
const enum BlockSize uv_block_size = get_plane_block_size(bsize,
&xd->plane[1]);
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
MB_MODE_INFO *mbmi = &xd->mode_info_context->mbmi;
const int is_comp_pred = (mbmi->second_ref_frame > 0);
const int num_refs = is_comp_pred ? 2 : 1;
const int this_mode = mbmi->mode;
int i;
int refs[2] = { mbmi->ref_frame,
(mbmi->second_ref_frame < 0 ? 0 : mbmi->second_ref_frame) };
int_mv cur_mv[2];
int_mv ref_mv[2];
int64_t this_rd = 0;
Make switchable filter search subsampling-aware Makes the temporary storage of the filtered data agnostic to the number of planes and how they're subsampled. Change-Id: I12f352cd69a47ebe1ac622af30db29b49becb7f4
2013-05-07 20:49:21 +02:00
unsigned char tmp_buf[MAX_MB_PLANE][64 * 64];
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
int pred_exists = 0;
int interpolating_intpel_seen = 0;
int intpel_mv;
int64_t rd, best_rd = INT64_MAX;
vp8_rd_pick_best_mbsegmentation code restructure Moved the code from the segmentation loop into a function which is now called for each segment. This will allow us to change the segment order checking more easily. Change-Id: I9510d26f0acae5a73043fcca8f1984b121d3e052
2010-12-06 22:42:52 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
switch (this_mode) {
case NEWMV:
ref_mv[0] = mbmi->ref_mvs[refs[0]][0];
ref_mv[1] = mbmi->ref_mvs[refs[1]][0];
Fix compiler warnings Change-Id: I2a97f08cc3c7808ce5be39e910cc5147ecf03a1d
2010-06-11 20:33:49 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if (is_comp_pred) {
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
// Initialize mv using single prediction mode result.
frame_mv[refs[0]].as_int = single_newmv[refs[0]].as_int;
frame_mv[refs[1]].as_int = single_newmv[refs[1]].as_int;
Refactor rd loop for inter modes This commit pulls the iterative motion search for compound inter- inter out from handle_inter_mode_ as a separate function. Hence, it is applicable to 4x4/4x8/8x4 level compound inter search to be enabled later. Also edit the rd loop for 4x4 inter block sizes for cosmetic purpose. Change-Id: Ibc71a11cbe5a26cd52faba01026cf8446cf4d2b4
2013-05-28 23:02:29 +02:00
if (cpi->sf.comp_inter_joint_serach)
iterative_motion_search(cpi, x, bsize, frame_mv, scaled_ref_frame,
mi_row, mi_col, single_newmv);
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
Decrease scope of frame_mv argument to handle_inter_mode(). Change-Id: I81c637c61ecc33cb66beb59a2a33166d66b9a0a2
2013-05-27 23:16:45 +02:00
if (frame_mv[refs[0]].as_int == INVALID_MV ||
frame_mv[refs[1]].as_int == INVALID_MV)
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
return INT64_MAX;
Decrease scope of frame_mv argument to handle_inter_mode(). Change-Id: I81c637c61ecc33cb66beb59a2a33166d66b9a0a2
2013-05-27 23:16:45 +02:00
*rate2 += vp9_mv_bit_cost(&frame_mv[refs[0]],
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
&ref_mv[0],
x->nmvjointcost, x->mvcost, 96,
x->e_mbd.allow_high_precision_mv);
Decrease scope of frame_mv argument to handle_inter_mode(). Change-Id: I81c637c61ecc33cb66beb59a2a33166d66b9a0a2
2013-05-27 23:16:45 +02:00
*rate2 += vp9_mv_bit_cost(&frame_mv[refs[1]],
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
&ref_mv[1],
x->nmvjointcost, x->mvcost, 96,
x->e_mbd.allow_high_precision_mv);
Refactor SPLITMV in RD to a special case. SPLITMV shares very little code with the other inter modes. Change-Id: I0beee95de1fbedeea1f71650951ffc7db3a118eb
2012-08-08 20:49:56 +02:00
} else {
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
struct buf_2d backup_yv12[MAX_MB_PLANE] = {{0}};
int bestsme = INT_MAX;
int further_steps, step_param = cpi->sf.first_step;
int sadpb = x->sadperbit16;
int_mv mvp_full, tmp_mv;
int sr = 0;
Selectable transform size try 2. Results: derf (vanilla or +hybridtx) +0.2% and (+hybrid16x16 or +tx16x16) +0.7%-0.8%; HD (vanilla or +hybridtx) +0.1-0.2% and (+hybrid16x16 or +tx16x16) +1.4%, STD/HD (vanilla or +hybridtx) about even, and (+hybrid16x16 or +tx16x16) +0.8-1.0%. Change-Id: I03899e2f7a64e725a863f32e55366035ba77aa62
2012-10-09 18:18:21 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
int tmp_col_min = x->mv_col_min;
int tmp_col_max = x->mv_col_max;
int tmp_row_min = x->mv_row_min;
int tmp_row_max = x->mv_row_max;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
if (scaled_ref_frame[0]) {
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
int i;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Swap out the reference frame for a version that's been scaled to
// match the resolution of the current frame, allowing the existing
// motion search code to be used without additional modifications.
for (i = 0; i < MAX_MB_PLANE; i++)
backup_yv12[i] = xd->plane[i].pre[0];
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
setup_pre_planes(xd, scaled_ref_frame[0], NULL, mi_row, mi_col,
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
NULL, NULL);
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
}
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
vp9_clamp_mv_min_max(x, &ref_mv[0]);
Dual 16x16 inter prediction. This patch introduces the concept of dual inter16x16 prediction. A 16x16 inter-predicted macroblock can use 2 references instead of 1, where both references use the same mvmode (new, near/est, zero). In the case of newmv, this means that two MVs are coded instead of one. The frame can be encoded in 3 ways: all MBs single-prediction, all MBs dual prediction, or per-MB single/dual prediction selection ("hybrid"), in which case a single bit is coded per-MB to indicate whether the MB uses single or dual inter prediction. In the future, we can (maybe?) get further gains by mixing this with Adrian's 32x32 work, per-segment dual prediction settings, or adding support for dual splitmv/8x8mv inter prediction. Gain (on derf-set, CQ mode) is ~2.8% (SSIM) or ~3.6% (glb PSNR). Most gain is at medium/high bitrates, but there's minor gains at low bitrates also. Output was confirmed to match between encoder and decoder. Note for optimization people: this patch introduces a 2nd version of 16x16/8x8 sixtap/bilin functions, which does an avg instead of a store. They may want to look and make sure this is implemented to their satisfaction so we can optimize it best in the future. Change-ID: I59dc84b07cbb3ccf073ac0f756d03d294cb19281
2011-12-06 20:53:02 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
sr = vp9_init_search_range(cpi->common.width, cpi->common.height);
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// mvp_full.as_int = ref_mv[0].as_int;
mvp_full.as_int =
mbmi->ref_mvs[refs[0]][x->mv_best_ref_index[refs[0]]].as_int;
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
mvp_full.as_mv.col >>= 3;
mvp_full.as_mv.row >>= 3;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// adjust search range according to sr from mv prediction
step_param = MAX(step_param, sr);
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Further step/diamond searches as necessary
further_steps = (cpi->sf.max_step_search_steps - 1) - step_param;
Added Prediction Filter to Mode Selection Added the ability to optionally filter the prediction data when inter modes are selected (excludes SPLITMV, for now). The mode selection loop considers both the filtered and non-filtered prediction data when choosing mode. The filter can be turned on/off at the frame-level, or signaled for each MB. Change-Id: I1b783c71d95a361ab36c761b07e8a6b06bc36822
2012-06-26 01:23:58 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
bestsme = vp9_full_pixel_diamond(cpi, x, &mvp_full, step_param,
sadpb, further_steps, 1,
&cpi->fn_ptr[block_size],
&ref_mv[0], &tmp_mv);
x->mv_col_min = tmp_col_min;
x->mv_col_max = tmp_col_max;
x->mv_row_min = tmp_row_min;
x->mv_row_max = tmp_row_max;
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if (bestsme < INT_MAX) {
int dis; /* TODO: use dis in distortion calculation later. */
unsigned int sse;
cpi->find_fractional_mv_step(x, &tmp_mv,
&ref_mv[0],
x->errorperbit,
&cpi->fn_ptr[block_size],
x->nmvjointcost, x->mvcost,
&dis, &sse);
}
Refactor 4x4 block level rd loop This commit makes clean-ups in the rate-distortion loop for 4x4, 4x8, and 8x4 block sizes for the use of iterative motion search. Removed unnecessary use of bmi in handle_inter_mode. Deprecated loop over labels in the 4x4/4x8/8x4 block rd search. Change-Id: I71203dbb68b65e66f073b37abd90d82ef5ae6826
2013-05-29 21:52:57 +02:00
frame_mv[refs[0]].as_int = tmp_mv.as_int;
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
single_newmv[refs[0]].as_int = tmp_mv.as_int;
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Add the new motion vector cost to our rolling cost variable
*rate2 += vp9_mv_bit_cost(&tmp_mv, &ref_mv[0],
x->nmvjointcost, x->mvcost,
96, xd->allow_high_precision_mv);
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// restore the predictor, if required
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
if (scaled_ref_frame[0]) {
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
int i;
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
for (i = 0; i < MAX_MB_PLANE; i++)
xd->plane[i].pre[0] = backup_yv12[i];
}
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
break;
case NEARMV:
case NEARESTMV:
case ZEROMV:
default:
break;
}
for (i = 0; i < num_refs; ++i) {
Decrease scope of frame_mv argument to handle_inter_mode(). Change-Id: I81c637c61ecc33cb66beb59a2a33166d66b9a0a2
2013-05-27 23:16:45 +02:00
cur_mv[i] = frame_mv[refs[i]];
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Clip "next_nearest" so that it does not extend to far out of image
if (this_mode == NEWMV)
assert(!clamp_mv2(&cur_mv[i], xd));
else
clamp_mv2(&cur_mv[i], xd);
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if (mv_check_bounds(x, &cur_mv[i]))
return INT64_MAX;
mbmi->mv[i].as_int = cur_mv[i].as_int;
}
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
/* We don't include the cost of the second reference here, because there
* are only three options: Last/Golden, ARF/Last or Golden/ARF, or in other
* words if you present them in that order, the second one is always known
* if the first is known */
*compmode_cost = vp9_cost_bit(vp9_get_pred_prob(cm, xd, PRED_COMP),
is_comp_pred);
*rate2 += vp9_cost_mv_ref(cpi, this_mode,
mbmi->mb_mode_context[mbmi->ref_frame]);
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
pred_exists = 0;
interpolating_intpel_seen = 0;
// Are all MVs integer pel for Y and UV
intpel_mv = (mbmi->mv[0].as_mv.row & 15) == 0 &&
(mbmi->mv[0].as_mv.col & 15) == 0;
if (is_comp_pred)
intpel_mv &= (mbmi->mv[1].as_mv.row & 15) == 0 &&
(mbmi->mv[1].as_mv.col & 15) == 0;
// Search for best switchable filter by checking the variance of
// pred error irrespective of whether the filter will be used
if (1) {
int i, newbest;
int tmp_rate_sum = 0, tmp_dist_sum = 0;
for (i = 0; i < VP9_SWITCHABLE_FILTERS; ++i) {
int rs = 0;
const INTERPOLATIONFILTERTYPE filter = vp9_switchable_interp[i];
const int is_intpel_interp = intpel_mv &&
vp9_is_interpolating_filter[filter];
mbmi->interp_filter = filter;
vp9_setup_interp_filters(xd, mbmi->interp_filter, cm);
Initial WebM release
2010-05-18 17:58:33 +02:00
Adding get_switchable_rate function. Change-Id: I71311a14f8d7f48508b250f25d1d0914c6a1ac72
2013-05-07 21:23:14 +02:00
if (cm->mcomp_filter_type == SWITCHABLE)
rs = get_switchable_rate(cm, x);
enabled 8x8 intra prediction modes on inter frames This commit enabled the usage of 8x8 intra prediction modes on inter frames. There are a few TODO items related to this: 1)baseline entropy need be calibrated; 2)cost of UV need to be done more properly rather than using decision only relying on Y; 3)Threshold for allowing picking 8x8 intra prediction should be lowered to lower than the B_PRED. Even with all the TODOs, tests showed consistent gain on derf set ~0.1% (PSNR:0.08% and SSIM:0.14%). It is assumed that 8x8 intra prediction will help more on large resolution clips, especially with above TODOs addressed. Change-Id: I398ada49dfc32575cfab962a569c2885111ae3ba
2011-12-01 01:25:00 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if (interpolating_intpel_seen && is_intpel_interp) {
rd = RDCOST(x->rdmult, x->rddiv, rs + tmp_rate_sum, tmp_dist_sum);
} else {
int rate_sum = 0, dist_sum = 0;
vp9_build_inter_predictors_sb(xd, mi_row, mi_col, bsize);
model_rd_for_sb(cpi, bsize, x, xd, &rate_sum, &dist_sum);
rd = RDCOST(x->rdmult, x->rddiv, rs + rate_sum, dist_sum);
if (!interpolating_intpel_seen && is_intpel_interp) {
tmp_rate_sum = rate_sum;
tmp_dist_sum = dist_sum;
}
}
newbest = i == 0 || rd < best_rd;
Initial WebM release
2010-05-18 17:58:33 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if (newbest) {
best_rd = rd;
*best_filter = mbmi->interp_filter;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
}
Selectable transform size try 2. Results: derf (vanilla or +hybridtx) +0.2% and (+hybrid16x16 or +tx16x16) +0.7%-0.8%; HD (vanilla or +hybridtx) +0.1-0.2% and (+hybrid16x16 or +tx16x16) +1.4%, STD/HD (vanilla or +hybridtx) about even, and (+hybrid16x16 or +tx16x16) +0.8-1.0%. Change-Id: I03899e2f7a64e725a863f32e55366035ba77aa62
2012-10-09 18:18:21 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if ((cm->mcomp_filter_type == SWITCHABLE && newbest) ||
(cm->mcomp_filter_type != SWITCHABLE &&
cm->mcomp_filter_type == mbmi->interp_filter)) {
Make switchable filter search subsampling-aware Makes the temporary storage of the filtered data agnostic to the number of planes and how they're subsampled. Change-Id: I12f352cd69a47ebe1ac622af30db29b49becb7f4
2013-05-07 20:49:21 +02:00
int p;
for (p = 0; p < MAX_MB_PLANE; p++) {
const int y = (MI_SIZE * bh) >> xd->plane[p].subsampling_y;
const int x = (MI_SIZE * bw) >> xd->plane[p].subsampling_x;
int i;
for (i = 0; i < y; i++)
vpx_memcpy(&tmp_buf[p][64 * i],
xd->plane[p].dst.buf + i * xd->plane[p].dst.stride, x);
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
pred_exists = 1;
Selectable transform size try 2. Results: derf (vanilla or +hybridtx) +0.2% and (+hybrid16x16 or +tx16x16) +0.7%-0.8%; HD (vanilla or +hybridtx) +0.1-0.2% and (+hybrid16x16 or +tx16x16) +1.4%, STD/HD (vanilla or +hybridtx) about even, and (+hybrid16x16 or +tx16x16) +0.8-1.0%. Change-Id: I03899e2f7a64e725a863f32e55366035ba77aa62
2012-10-09 18:18:21 +02:00
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
interpolating_intpel_seen |= is_intpel_interp;
Using partition_info instead of blockd info for splitmv The partition_info struct contains info just for SPLITMV, so it should be used instead of BLOCKD. Eventually, I want to reduce the size of B_MODE_INFO struct found in BLOCKD, so this is the first step toward that goal. Also, since SPLITMV is not supported in vp8_pick_inter_mode(), the unnecessary mem copies and checks were removed. For rt encodes, this gave a slight performance improvement. Change-Id: I5585c98fa9d5acbde1c7e0f452a01d9ecc080574
2011-05-19 21:03:36 +02:00
}
Refactoring of switchable filter search for speed Refactors the switchable filter search in the rd loop to improve encode speed. Uses a piecewise approximation to a closed form expression to estimate rd cost for a Laplacian source with a given variance and quantization step-size. About 40% encode time reduction is achieved. Results (on a feb 12 baseline) show a slight drop: derf: -0.019% yt: +0.010% std-hd: -0.162% hd: -0.050% Change-Id: Ie861badf5bba1e3b1052e29a0ef1b7e256edbcd0
2013-02-12 02:08:52 +01:00
}
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Set the appripriate filter
mbmi->interp_filter = cm->mcomp_filter_type != SWITCHABLE ?
cm->mcomp_filter_type : *best_filter;
vp9_setup_interp_filters(xd, mbmi->interp_filter, cm);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if (pred_exists) {
Make switchable filter search subsampling-aware Makes the temporary storage of the filtered data agnostic to the number of planes and how they're subsampled. Change-Id: I12f352cd69a47ebe1ac622af30db29b49becb7f4
2013-05-07 20:49:21 +02:00
int p;
for (p = 0; p < MAX_MB_PLANE; p++) {
const int y = (MI_SIZE * bh) >> xd->plane[p].subsampling_y;
const int x = (MI_SIZE * bw) >> xd->plane[p].subsampling_x;
int i;
for (i = 0; i < y; i++)
vpx_memcpy(xd->plane[p].dst.buf + i * xd->plane[p].dst.stride,
&tmp_buf[p][64 * i], x);
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
} else {
// Handles the special case when a filter that is not in the
// switchable list (ex. bilinear, 6-tap) is indicated at the frame level
vp9_build_inter_predictors_sb(xd, mi_row, mi_col, bsize);
}
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Adding get_switchable_rate function. Change-Id: I71311a14f8d7f48508b250f25d1d0914c6a1ac72
2013-05-07 21:23:14 +02:00
if (cpi->common.mcomp_filter_type == SWITCHABLE)
*rate2 += get_switchable_rate(cm, x);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if (cpi->active_map_enabled && x->active_ptr[0] == 0)
x->skip = 1;
else if (x->encode_breakout) {
unsigned int var, sse;
int threshold = (xd->plane[0].dequant[1]
* xd->plane[0].dequant[1] >> 4);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if (threshold < x->encode_breakout)
threshold = x->encode_breakout;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Add building blocks for partition down to 4x4 Macro ab4x4 contains experiments for recursive partition down to 4x4 block size. Change-Id: Ic727842fa98a4df9fd51e0025a545dc76a5c76c1
2013-05-07 19:45:31 +02:00
var = cpi->fn_ptr[block_size].vf(x->plane[0].src.buf,
x->plane[0].src.stride,
xd->plane[0].dst.buf,
xd->plane[0].dst.stride,
&sse);
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if ((int)sse < threshold) {
unsigned int q2dc = xd->plane[0].dequant[0];
/* If there is no codeable 2nd order dc
or a very small uniform pixel change change */
if ((sse - var < q2dc * q2dc >> 4) ||
(sse / 2 > var && sse - var < 64)) {
// Check u and v to make sure skip is ok
int sse2;
Add building blocks for partition down to 4x4 Macro ab4x4 contains experiments for recursive partition down to 4x4 block size. Change-Id: Ic727842fa98a4df9fd51e0025a545dc76a5c76c1
2013-05-07 19:45:31 +02:00
unsigned int sse2u, sse2v;
var = cpi->fn_ptr[uv_block_size].vf(x->plane[1].src.buf,
x->plane[1].src.stride,
xd->plane[1].dst.buf,
xd->plane[1].dst.stride, &sse2u);
var = cpi->fn_ptr[uv_block_size].vf(x->plane[2].src.buf,
x->plane[1].src.stride,
xd->plane[2].dst.buf,
xd->plane[1].dst.stride, &sse2v);
sse2 = sse2u + sse2v;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if (sse2 * 2 < threshold) {
x->skip = 1;
*distortion = sse + sse2;
*rate2 = 500;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
/* for best_yrd calculation */
*rate_uv = 0;
*distortion_uv = sse2;
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
*disable_skip = 1;
this_rd = RDCOST(x->rdmult, x->rddiv, *rate2, *distortion);
}
}
}
Refactor to remove some arguments from vp8_rd_pick_inter_mode Change-Id: I8e72279cc68f34d269705f06cdaf8f3d06eed635
2012-08-10 03:25:29 +02:00
}
Selectable transform size try 2. Results: derf (vanilla or +hybridtx) +0.2% and (+hybrid16x16 or +tx16x16) +0.7%-0.8%; HD (vanilla or +hybridtx) +0.1-0.2% and (+hybrid16x16 or +tx16x16) +1.4%, STD/HD (vanilla or +hybridtx) about even, and (+hybrid16x16 or +tx16x16) +0.8-1.0%. Change-Id: I03899e2f7a64e725a863f32e55366035ba77aa62
2012-10-09 18:18:21 +02:00
if (!x->skip) {
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
int skippable_y, skippable_uv;
Selectable transform size try 2. Results: derf (vanilla or +hybridtx) +0.2% and (+hybrid16x16 or +tx16x16) +0.7%-0.8%; HD (vanilla or +hybridtx) +0.1-0.2% and (+hybrid16x16 or +tx16x16) +1.4%, STD/HD (vanilla or +hybridtx) about even, and (+hybrid16x16 or +tx16x16) +0.8-1.0%. Change-Id: I03899e2f7a64e725a863f32e55366035ba77aa62
2012-10-09 18:18:21 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// Y cost and distortion
super_block_yrd(cpi, x, rate_y, distortion_y, &skippable_y,
bsize, txfm_cache);
*rate2 += *rate_y;
*distortion += *distortion_y;
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
super_block_uvrd(cm, x, rate_uv, distortion_uv,
&skippable_uv, bsize);
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
*rate2 += *rate_uv;
*distortion += *distortion_uv;
*skippable = skippable_y && skippable_uv;
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
if (!(*mode_excluded)) {
if (is_comp_pred) {
*mode_excluded = (cpi->common.comp_pred_mode == SINGLE_PREDICTION_ONLY);
} else {
*mode_excluded = (cpi->common.comp_pred_mode == COMP_PREDICTION_ONLY);
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
}
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
return this_rd; // if 0, this will be re-calculated by caller
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
Restyle code Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-14 00:21:29 +02:00
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
void vp9_rd_pick_intra_mode_sb(VP9_COMP *cpi, MACROBLOCK *x,
int *returnrate, int *returndist,
Merge pick_sb_modes and pick_sb64_modes. Change-Id: Iad69e7a3b7e470acf6094f6a52e7da69066fd552
2013-04-11 18:33:49 +02:00
BLOCK_SIZE_TYPE bsize,
PICK_MODE_CONTEXT *ctx) {
Change name of common top-level structures from VP8 to VP9. This change encompasses VP8_PTR, VP8_COMP, VP8D_COMP, VP8_COMMON, VP8Decompressor and VP8Common. Change-Id: I514ef4ad4e682370f36d656af1c09ee20da216ad
2012-10-31 01:53:32 +01:00
VP9_COMMON *cm = &cpi->common;
Add proper skip support to intra frames. Change-Id: I7524883fb29f42303fb46a5bc6772fbcf8781d1d
2012-08-29 19:43:20 +02:00
MACROBLOCKD *xd = &x->e_mbd;
fix a number issues that cause failures During master jenkins verification proces Change-Id: I3722b8753eaf39f99b45979ce407a8ea0bea0b89
2013-01-15 03:03:34 +01:00
int rate_y = 0, rate_uv;
int rate_y_tokenonly = 0, rate_uv_tokenonly;
int dist_y = 0, dist_uv;
int y_skip = 0, uv_skip;
Prevent filling transform size cache with uninitialized values. The 32x32 value in case of splitmv was uninitialized. this leads to all kind of erratic behaviour down the line. Also fill in dummy values for superblocks in keyframes (the values are currently unused, but we run into integer overflows anyway, which makes detecting bad cases harder). Lastly, in case we did not find any RD value at all, don't set tx_diff to INT_MIN, but instead set it to zero (since if we couldn't find a mode, it's unlikely that any particular transform would have made that worse or better; rather, it's likely equally bad for all tx_sizes). Change-Id: If236fd3aa2037e5b398d03f3b1978fbbc5ce740e
2013-02-16 00:55:31 +01:00
int64_t txfm_cache[NB_TXFM_MODES], err;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
MB_PREDICTION_MODE mode;
TX_SIZE txfm_size;
int rate4x4_y, rate4x4_y_tokenonly, dist4x4_y;
int64_t err4x4 = INT64_MAX;
Prevent filling transform size cache with uninitialized values. The 32x32 value in case of splitmv was uninitialized. this leads to all kind of erratic behaviour down the line. Also fill in dummy values for superblocks in keyframes (the values are currently unused, but we run into integer overflows anyway, which makes detecting bad cases harder). Lastly, in case we did not find any RD value at all, don't set tx_diff to INT_MIN, but instead set it to zero (since if we couldn't find a mode, it's unlikely that any particular transform would have made that worse or better; rather, it's likely equally bad for all tx_sizes). Change-Id: If236fd3aa2037e5b398d03f3b1978fbbc5ce740e
2013-02-16 00:55:31 +01:00
int i;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Fix lingering x->skip settings if static_threshold is used. Keyframes don't set this variable, so it would use the last set values from inter frames. Change-Id: Ie1ef45ece2c44b21b5d55f6cea9f7d6e7a445692
2013-04-15 21:50:32 +02:00
ctx->skip = 0;
Coding con-zero count rather than EOB for coeffs This patch revamps the entropy coding of coefficients to code first a non-zero count per coded block and correspondingly remove the EOB token from the token set. STATUS: Main encode/decode code achieving encode/decode sync - done. Forward and backward probability updates to the nzcs - done. Rd costing updates for nzcs - done. Note: The dynamic progrmaming apporach used in trellis quantization is not exactly compatible with nzcs. A suboptimal approach has been used instead where branch costs are updated to account for changes in the nzcs. TODO: Training the default probs/counts for nzcs Change-Id: I951bc1e22f47885077a7453a09b0493daa77883d
2013-02-20 19:16:24 +01:00
xd->mode_info_context->mbmi.mode = DC_PRED;
Prevent filling transform size cache with uninitialized values. The 32x32 value in case of splitmv was uninitialized. this leads to all kind of erratic behaviour down the line. Also fill in dummy values for superblocks in keyframes (the values are currently unused, but we run into integer overflows anyway, which makes detecting bad cases harder). Lastly, in case we did not find any RD value at all, don't set tx_diff to INT_MIN, but instead set it to zero (since if we couldn't find a mode, it's unlikely that any particular transform would have made that worse or better; rather, it's likely equally bad for all tx_sizes). Change-Id: If236fd3aa2037e5b398d03f3b1978fbbc5ce740e
2013-02-16 00:55:31 +01:00
err = rd_pick_intra_sby_mode(cpi, x, &rate_y, &rate_y_tokenonly,
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
&dist_y, &y_skip, bsize, txfm_cache);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
mode = xd->mode_info_context->mbmi.mode;
txfm_size = xd->mode_info_context->mbmi.txfm_size;
Fix compiler warnings The warnings caused verify failure with gerrit for several commits Change-Id: I030df8638bd69b8783a3ac58e720ff9f0bfd546c
2013-01-14 22:56:52 +01:00
rd_pick_intra_sbuv_mode(cpi, x, &rate_uv, &rate_uv_tokenonly,
Force bsize for UV in I4X4 and SPLITMV Use 4x4 block coding for UV components arbitrarily in I4X4_PRED and SPLITMV coding modes. This is a temporary solution to enable bit-stream support for recursive partition down to 4x4 block size. Will separate the functionalities of 4x4 block coding rate-distortion out from those of superblocks. Change-Id: I03dc15d5897014f175f3f2c91e9b266091d56797
2013-05-11 22:33:13 +02:00
&dist_uv, &uv_skip,
(bsize < BLOCK_SIZE_SB8X8) ? BLOCK_SIZE_SB8X8 :
bsize);
Enable recursive partition down to 4x4 This commit allows the rate-distortion optimization recursion at encoder to go down to 4x4 block size. It deprecates the use of I4X4_PRED and SPLITMV syntax elements from bit-stream writing/reading. Will remove the unused probability models in the next patch. The partition type search and bit-stream are now capable of supporting the rectangular partition of 8x8 block, i.e., 8x4 and 4x8. Need to revise the rate-distortion parts to get these two partition tested in the rd loop. Change-Id: I0dfe3b90a1507ad6138db10cc58e6e237a06a9d6
2013-05-11 02:06:37 +02:00
if (bsize < BLOCK_SIZE_SB8X8)
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
err4x4 = rd_pick_intra4x4mby_modes(cpi, x, &rate4x4_y,
&rate4x4_y_tokenonly,
&dist4x4_y, err);
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
if (y_skip && uv_skip) {
Add proper skip support to intra frames. Change-Id: I7524883fb29f42303fb46a5bc6772fbcf8781d1d
2012-08-29 19:43:20 +02:00
*returnrate = rate_y + rate_uv - rate_y_tokenonly - rate_uv_tokenonly +
Remove vp8 in local symbols. For non-static functions, change the prefix to vp9_. For static functions, remove the prefix. Also fix some comments, remove unused code or unused function prototypes. Change-Id: I1f8be05362f66060fe421c3d4c9a906fdf835de5
2012-10-31 22:40:53 +01:00
vp9_cost_bit(vp9_get_pred_prob(cm, xd, PRED_MBSKIP), 1);
Add proper skip support to intra frames. Change-Id: I7524883fb29f42303fb46a5bc6772fbcf8781d1d
2012-08-29 19:43:20 +02:00
*returndist = dist_y + (dist_uv >> 2);
Fix the transform type selection in 4x4 partition This commit allows proper transform type (DCT/ADST) selection in the settings of partition 4x4 level. Change-Id: Iec6f922a46480d777e7ca9142a99e8c131f0077b
2013-05-15 19:19:40 +02:00
memset(ctx->txfm_rd_diff, 0, sizeof(ctx->txfm_rd_diff));
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
xd->mode_info_context->mbmi.mode = mode;
xd->mode_info_context->mbmi.txfm_size = txfm_size;
Enable recursive partition down to 4x4 This commit allows the rate-distortion optimization recursion at encoder to go down to 4x4 block size. It deprecates the use of I4X4_PRED and SPLITMV syntax elements from bit-stream writing/reading. Will remove the unused probability models in the next patch. The partition type search and bit-stream are now capable of supporting the rectangular partition of 8x8 block, i.e., 8x4 and 4x8. Need to revise the rate-distortion parts to get these two partition tested in the rd loop. Change-Id: I0dfe3b90a1507ad6138db10cc58e6e237a06a9d6
2013-05-11 02:06:37 +02:00
} else if (bsize < BLOCK_SIZE_SB8X8 && err4x4 < err) {
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
*returnrate = rate4x4_y + rate_uv +
vp9_cost_bit(vp9_get_pred_prob(cm, xd, PRED_MBSKIP), 0);
*returndist = dist4x4_y + (dist_uv >> 2);
for (i = 0; i < NB_TXFM_MODES; i++) {
ctx->txfm_rd_diff[i] = MIN(err4x4, err - txfm_cache[i]);
}
xd->mode_info_context->mbmi.txfm_size = TX_4X4;
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
} else {
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
*returnrate = rate_y + rate_uv +
vp9_cost_bit(vp9_get_pred_prob(cm, xd, PRED_MBSKIP), 0);
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
*returndist = dist_y + (dist_uv >> 2);
Prevent filling transform size cache with uninitialized values. The 32x32 value in case of splitmv was uninitialized. this leads to all kind of erratic behaviour down the line. Also fill in dummy values for superblocks in keyframes (the values are currently unused, but we run into integer overflows anyway, which makes detecting bad cases harder). Lastly, in case we did not find any RD value at all, don't set tx_diff to INT_MIN, but instead set it to zero (since if we couldn't find a mode, it's unlikely that any particular transform would have made that worse or better; rather, it's likely equally bad for all tx_sizes). Change-Id: If236fd3aa2037e5b398d03f3b1978fbbc5ce740e
2013-02-16 00:55:31 +01:00
for (i = 0; i < NB_TXFM_MODES; i++) {
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
ctx->txfm_rd_diff[i] = MIN(err4x4, err - txfm_cache[i]);
Prevent filling transform size cache with uninitialized values. The 32x32 value in case of splitmv was uninitialized. this leads to all kind of erratic behaviour down the line. Also fill in dummy values for superblocks in keyframes (the values are currently unused, but we run into integer overflows anyway, which makes detecting bad cases harder). Lastly, in case we did not find any RD value at all, don't set tx_diff to INT_MIN, but instead set it to zero (since if we couldn't find a mode, it's unlikely that any particular transform would have made that worse or better; rather, it's likely equally bad for all tx_sizes). Change-Id: If236fd3aa2037e5b398d03f3b1978fbbc5ce740e
2013-02-16 00:55:31 +01:00
}
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
xd->mode_info_context->mbmi.txfm_size = txfm_size;
xd->mode_info_context->mbmi.mode = mode;
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
}
Merge pick_sb_modes and pick_sb64_modes. Change-Id: Iad69e7a3b7e470acf6094f6a52e7da69066fd552
2013-04-11 18:33:49 +02:00
vpx_memcpy(&ctx->mic, xd->mode_info_context, sizeof(MODE_INFO));
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
}
Move RD intra block mode selection to rdopt.c This change is analogous to I0b67dae1f8a74902378da7bdf565e39ab832dda7, which made the move for the non-RD path. Change-Id: If63fc1b0cd1eb7f932e710f83ff24d91454f8ed1
2011-06-08 18:05:05 +02:00
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
int64_t vp9_rd_pick_inter_mode_sb(VP9_COMP *cpi, MACROBLOCK *x,
Grow MODE_INFO array to use an 8x8 basis. Change-Id: I087e08e7909a406b71715b8525c104208daa6889
2013-04-26 20:57:17 +02:00
int mi_row, int mi_col,
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
int *returnrate,
int *returndistortion,
Merge pick_sb_modes and pick_sb64_modes. Change-Id: Iad69e7a3b7e470acf6094f6a52e7da69066fd552
2013-04-11 18:33:49 +02:00
BLOCK_SIZE_TYPE bsize,
PICK_MODE_CONTEXT *ctx) {
Change name of common top-level structures from VP8 to VP9. This change encompasses VP8_PTR, VP8_COMP, VP8D_COMP, VP8_COMMON, VP8Decompressor and VP8Common. Change-Id: I514ef4ad4e682370f36d656af1c09ee20da216ad
2012-10-31 01:53:32 +01:00
VP9_COMMON *cm = &cpi->common;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
MACROBLOCKD *xd = &x->e_mbd;
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
MB_MODE_INFO *mbmi = &xd->mode_info_context->mbmi;
Removing y_to_uv_block_size and y_bsizet_to_block_size functions. Change-Id: I49527ff8dd8bef1074c18a964fed2a575f0b118a
2013-05-09 00:23:42 +02:00
const enum BlockSize block_size = get_plane_block_size(bsize, &xd->plane[0]);
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
MB_PREDICTION_MODE this_mode;
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
MB_PREDICTION_MODE best_mode = DC_PRED;
Make setup_pred_block subsampling-aware. Code previously set up the pointers by scaling by MI_UV_SIZE, which is 4:2:0 only. Change-Id: Ic13a92895cff018ec1345736746ed84cb31e6e31
2013-05-07 20:31:12 +02:00
MV_REFERENCE_FRAME ref_frame, second_ref = INTRA_FRAME;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
unsigned char segment_id = xd->mode_info_context->mbmi.segment_id;
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
int comp_pred, i;
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
int_mv frame_mv[MB_MODE_COUNT][MAX_REF_FRAMES];
Make setup_pred_block subsampling-aware. Code previously set up the pointers by scaling by MI_UV_SIZE, which is 4:2:0 only. Change-Id: Ic13a92895cff018ec1345736746ed84cb31e6e31
2013-05-07 20:31:12 +02:00
struct buf_2d yv12_mb[4][MAX_MB_PLANE];
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
int_mv single_newmv[MAX_REF_FRAMES];
Remove vp8 in local symbols. For non-static functions, change the prefix to vp9_. For static functions, remove the prefix. Also fix some comments, remove unused code or unused function prototypes. Change-Id: I1f8be05362f66060fe421c3d4c9a906fdf835de5
2012-10-31 22:40:53 +01:00
static const int flag_list[4] = { 0, VP9_LAST_FLAG, VP9_GOLD_FLAG,
VP9_ALT_FLAG };
Start to anonymize reference frames Remove lst_fb_idx, gld_fb_idx, alt_fb_idx, refresh_last_frame, refresh_golden_frame, refresh_alt_ref_frame from common. Gold/Alt are encode side conventions. From the decoder's perspective, we want to be dealing with numbered references. Updates to active_ref 2 signal mode context switches, vestigial from refresh_alt_ref_frame. This needs some clean up to make sense with increased numbers of reference frames, as well as reimplementing the swapping of alt/golden which was previously done using the buffer-to-buffer copy mechanism removed in an earlier commit. Change-Id: I7334445158b7666f9295d2a2dd22aa03f4485f58
2013-01-15 22:49:44 +01:00
int idx_list[4] = {0,
Spatial resamping of ZEROMV predictors This patch allows coding frames using references of different resolution, in ZEROMV mode. For compound prediction, either reference may be scaled. To test, I use the resize_test and enable WRITE_RECON_BUFFER in vp9_onyxd_if.c. It's also useful to apply this patch to test/i420_video_source.h: --- a/test/i420_video_source.h +++ b/test/i420_video_source.h @@ -93,6 +93,7 @@ class I420VideoSource : public VideoSource { virtual void FillFrame() { // Read a frame from input_file. + if (frame_ != 3) if (fread(img_->img_data, raw_sz_, 1, input_file_) == 0) { limit_ = frame_; } This forces the frame that the resolution changes on to be coded with no motion, only scaling, and improves the quality of the result. Change-Id: I1ee75d19a437ff801192f767fd02a36bcbd1d496
2013-02-25 05:55:14 +01:00
cpi->lst_fb_idx,
cpi->gld_fb_idx,
cpi->alt_fb_idx};
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
int64_t best_rd = INT64_MAX;
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
int64_t best_txfm_rd[NB_TXFM_MODES];
int64_t best_txfm_diff[NB_TXFM_MODES];
int64_t best_pred_diff[NB_PREDICTION_TYPES];
int64_t best_pred_rd[NB_PREDICTION_TYPES];
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
MB_MODE_INFO best_mbmode;
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
int j;
warnings in various experiments Change-Id: Ib5106d4772450f8026f823dd743f162ab833b1d6
2012-11-30 16:29:43 +01:00
int mode_index, best_mode_index = 0;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
unsigned int ref_costs[MAX_REF_FRAMES];
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
int64_t best_overall_rd = INT64_MAX;
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
INTERPOLATIONFILTERTYPE best_filter = SWITCHABLE;
Refactoring of switchable filter search for speed Refactors the switchable filter search in the rd loop to improve encode speed. Uses a piecewise approximation to a closed form expression to estimate rd cost for a Laplacian source with a given variance and quantization step-size. About 40% encode time reduction is achieved. Results (on a feb 12 baseline) show a slight drop: derf: -0.019% yt: +0.010% std-hd: -0.162% hd: -0.050% Change-Id: Ie861badf5bba1e3b1052e29a0ef1b7e256edbcd0
2013-02-12 02:08:52 +01:00
INTERPOLATIONFILTERTYPE tmp_best_filter = SWITCHABLE;
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
int rate_uv_intra[TX_SIZE_MAX_SB], rate_uv_tokenonly[TX_SIZE_MAX_SB];
int dist_uv[TX_SIZE_MAX_SB], skip_uv[TX_SIZE_MAX_SB];
MB_PREDICTION_MODE mode_uv[TX_SIZE_MAX_SB];
Spatial resamping of ZEROMV predictors This patch allows coding frames using references of different resolution, in ZEROMV mode. For compound prediction, either reference may be scaled. To test, I use the resize_test and enable WRITE_RECON_BUFFER in vp9_onyxd_if.c. It's also useful to apply this patch to test/i420_video_source.h: --- a/test/i420_video_source.h +++ b/test/i420_video_source.h @@ -93,6 +93,7 @@ class I420VideoSource : public VideoSource { virtual void FillFrame() { // Read a frame from input_file. + if (frame_ != 3) if (fread(img_->img_data, raw_sz_, 1, input_file_) == 0) { limit_ = frame_; } This forces the frame that the resolution changes on to be coded with no motion, only scaling, and improves the quality of the result. Change-Id: I1ee75d19a437ff801192f767fd02a36bcbd1d496
2013-02-25 05:55:14 +01:00
struct scale_factors scale_factor[4];
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
unsigned int ref_frame_mask = 0;
unsigned int mode_mask = 0;
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
int64_t mode_distortions[MB_MODE_COUNT] = {-1};
int64_t frame_distortions[MAX_REF_FRAMES] = {-1};
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
int intra_cost_penalty = 20 * vp9_dc_quant(cpi->common.base_qindex,
cpi->common.y_dc_delta_q);
int_mv seg_mvs[4][MAX_REF_FRAMES - 1];
Store splitmv modes in context after 8x8 rd loop. Change-Id: I07aa89a67e0ac5f99ef0c448553dbc46b0ed27f2
2013-05-01 23:43:33 +02:00
union b_mode_info best_bmodes[4];
PARTITION_INFO best_partition;
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
for (i = 0; i < 4; i++) {
int j;
for (j = 0; j < MAX_REF_FRAMES - 1; j++)
seg_mvs[i][j].as_int = INVALID_MV;
}
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
// Everywhere the flag is set the error is much higher than its neighbors.
ctx->frames_with_high_error = 0;
ctx->modes_with_high_error = 0;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
xd->mode_info_context->mbmi.segment_id = segment_id;
Remove vp8 in local symbols. For non-static functions, change the prefix to vp9_. For static functions, remove the prefix. Also fix some comments, remove unused code or unused function prototypes. Change-Id: I1f8be05362f66060fe421c3d4c9a906fdf835de5
2012-10-31 22:40:53 +01:00
estimate_ref_frame_costs(cpi, segment_id, ref_costs);
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
vpx_memset(&best_mbmode, 0, sizeof(best_mbmode));
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
vpx_memset(&single_newmv, 0, sizeof(single_newmv));
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
for (i = 0; i < NB_PREDICTION_TYPES; ++i)
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
best_pred_rd[i] = INT64_MAX;
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
for (i = 0; i < NB_TXFM_MODES; i++)
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
best_txfm_rd[i] = INT64_MAX;
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
// Create a mask set to 1 for each frame used by a smaller resolution.
Renaming 'Speed' to 'speed' inside VP9_COMP struct. Change-Id: I4374b5af40ee9082ddf7956a9756a15ad9ad5436
2013-05-08 23:35:42 +02:00
if (cpi->speed > 0) {
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
switch (block_size) {
case BLOCK_64X64:
for (i = 0; i < 4; i++) {
for (j = 0; j < 4; j++) {
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
ref_frame_mask |= x->mb_context[i][j].frames_with_high_error;
mode_mask |= x->mb_context[i][j].modes_with_high_error;
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
}
}
for (i = 0; i < 4; i++) {
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
ref_frame_mask |= x->sb32_context[i].frames_with_high_error;
mode_mask |= x->sb32_context[i].modes_with_high_error;
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
}
break;
case BLOCK_32X32:
for (i = 0; i < 4; i++) {
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
ref_frame_mask |=
x->mb_context[xd->sb_index][i].frames_with_high_error;
mode_mask |= x->mb_context[xd->sb_index][i].modes_with_high_error;
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
}
break;
default:
catch all for new block sizes Just make sure we don't stop them from testing in speed 1. Change-Id: Iec9b3dba0a32616ff7a451207e0f54b81bb72575
2013-04-19 18:48:56 +02:00
// Until we handle all block sizes set it to present;
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
ref_frame_mask = 0;
mode_mask = 0;
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
break;
}
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
ref_frame_mask = ~ref_frame_mask;
mode_mask = ~mode_mask;
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
}
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
for (ref_frame = LAST_FRAME; ref_frame <= ALTREF_FRAME; ref_frame++) {
if (cpi->ref_frame_flags & flag_list[ref_frame]) {
64x64 blocksize support. 3.2% gains on std/hd, 1.0% gains on hd. Change-Id: I481d5df23d8a4fc650a5bcba956554490b2bd200
2013-01-06 03:20:25 +01:00
setup_buffer_inter(cpi, x, idx_list[ref_frame], ref_frame, block_size,
Grow MODE_INFO array to use an 8x8 basis. Change-Id: I087e08e7909a406b71715b8525c104208daa6889
2013-04-26 20:57:17 +02:00
mi_row, mi_col, frame_mv[NEARESTMV], frame_mv[NEARMV],
Remove unused mdcounts mdcounts seems no longer used. Change-Id: Idd8162e8acfa3f5be7a18767156cc79ccbc2bdee
2013-05-09 01:27:35 +02:00
yv12_mb, scale_factor);
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
frame_mv[NEWMV][ref_frame].as_int = INVALID_MV;
frame_mv[ZEROMV][ref_frame].as_int = 0;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
Renaming 'Speed' to 'speed' inside VP9_COMP struct. Change-Id: I4374b5af40ee9082ddf7956a9756a15ad9ad5436
2013-05-08 23:35:42 +02:00
if (cpi->speed == 0
|| (cpi->speed > 0 && (ref_frame_mask & (1 << INTRA_FRAME)))) {
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
mbmi->mode = DC_PRED;
Fix some crashes in sb8x8 experiment. Change-Id: I390bb1cedc835f439fd5dd6cda6572b29cbb139c
2013-05-01 23:45:27 +02:00
for (i = 0; i <= (bsize < BLOCK_SIZE_MB16X16 ? TX_4X4 :
(bsize < BLOCK_SIZE_SB32X32 ? TX_8X8 :
(bsize < BLOCK_SIZE_SB64X64 ? TX_16X16 : TX_32X32)));
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
i++) {
mbmi->txfm_size = i;
rd_pick_intra_sbuv_mode(cpi, x, &rate_uv_intra[i], &rate_uv_tokenonly[i],
Enable recursive partition down to 4x4 This commit allows the rate-distortion optimization recursion at encoder to go down to 4x4 block size. It deprecates the use of I4X4_PRED and SPLITMV syntax elements from bit-stream writing/reading. Will remove the unused probability models in the next patch. The partition type search and bit-stream are now capable of supporting the rectangular partition of 8x8 block, i.e., 8x4 and 4x8. Need to revise the rate-distortion parts to get these two partition tested in the rd loop. Change-Id: I0dfe3b90a1507ad6138db10cc58e6e237a06a9d6
2013-05-11 02:06:37 +02:00
&dist_uv[i], &skip_uv[i],
(bsize < BLOCK_SIZE_SB8X8) ? BLOCK_SIZE_SB8X8 :
bsize);
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
mode_uv[i] = mbmi->uv_mode;
}
32x32 transform for superblocks. This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-07 23:45:05 +01:00
}
Support 32x32 intra modes in non-keyframe superblocks. Change-Id: Icf8ad313c543462e523bff89690e5daa8d49bcc0
2012-11-16 00:50:07 +01:00
Refactoring of switchable filter search for speed Refactors the switchable filter search in the rd loop to improve encode speed. Uses a piecewise approximation to a closed form expression to estimate rd cost for a Laplacian source with a given variance and quantization step-size. About 40% encode time reduction is achieved. Results (on a feb 12 baseline) show a slight drop: derf: -0.019% yt: +0.010% std-hd: -0.162% hd: -0.050% Change-Id: Ie861badf5bba1e3b1052e29a0ef1b7e256edbcd0
2013-02-12 02:08:52 +01:00
for (mode_index = 0; mode_index < MAX_MODES; ++mode_index) {
Support 32x32 intra modes in non-keyframe superblocks. Change-Id: Icf8ad313c543462e523bff89690e5daa8d49bcc0
2012-11-16 00:50:07 +01:00
int mode_excluded = 0;
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
int64_t this_rd = INT64_MAX;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
int disable_skip = 0;
int other_cost = 0;
int compmode_cost = 0;
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
int rate2 = 0, rate_y = 0, rate_uv = 0;
int distortion2 = 0, distortion_y = 0, distortion_uv = 0;
int skippable;
int64_t txfm_cache[NB_TXFM_MODES];
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
int i;
for (i = 0; i < NB_TXFM_MODES; ++i)
txfm_cache[i] = INT64_MAX;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
// Test best rd so far against threshold for trying this mode.
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
if (bsize >= BLOCK_SIZE_SB8X8 &&
(best_rd < cpi->rd_threshes[mode_index] ||
cpi->rd_threshes[mode_index] == INT_MAX))
continue;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Reset skip flag in superblock RD loop. This is the superblock equivalent of commit 290b83a. Change-Id: Ib3945dd9e992fa9ec1fdea5a11e17a3cc0e37637
2013-01-30 06:42:46 +01:00
x->skip = 0;
Change non-function symbol vp8_ prefixes to vp9_. For local symbols, make them static instead. Change-Id: I13d60947a46f711bc8991e16100cea2a13e3a22e
2012-10-31 01:12:12 +01:00
this_mode = vp9_mode_order[mode_index].mode;
ref_frame = vp9_mode_order[mode_index].ref_frame;
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
if (cpi->speed > 0 && bsize >= BLOCK_SIZE_SB8X8) {
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
if (!(ref_frame_mask & (1 << ref_frame))) {
continue;
}
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
if (!(mode_mask & (1 << this_mode))) {
continue;
}
set up a new speed 1 slightly worse results for faster encodes Change-Id: I25ea82a18ce20635dbcd328808c1d05ac1f58fd7
2013-04-19 17:04:57 +02:00
if (vp9_mode_order[mode_index].second_ref_frame != NONE
&& !(ref_frame_mask
& (1 << vp9_mode_order[mode_index].second_ref_frame))) {
continue;
}
}
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
mbmi->ref_frame = ref_frame;
Supports inter-intra prediction with superblocks Adds support for compound inter-intra prediction with superblocks. Also, fixes a bug that disabled intra modes for superblocks. Change-Id: I4d711317e1bc19df8c2f32dc645429f7fff31036
2012-11-30 20:46:20 +01:00
mbmi->second_ref_frame = vp9_mode_order[mode_index].second_ref_frame;
Fix a unit test failure of sb8x8 on scaling ref Disable the use of scaled reference frame for motion search in SPLITMV mode. This fixes the unit test failure issue triggered when merging sb8x8 from experimental list. Change-Id: I02ac25fd8db8d5762f8fee29513b947189875fa0
2013-05-06 19:25:38 +02:00
fix valgrind warning scales for second reference frame vars are unitialized if the second ref frame is one of of those disallowed by refframeflags Change-Id: I4ce42de391178c1699dcaede18c5f12c84993c61
2013-05-29 21:34:10 +02:00
if (!(ref_frame == INTRA_FRAME
|| (cpi->ref_frame_flags & flag_list[ref_frame]))) {
continue;
}
if (!(mbmi->second_ref_frame == NONE
|| (cpi->ref_frame_flags & flag_list[mbmi->second_ref_frame]))) {
continue;
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
// TODO(jingning, jkoleszar): scaling reference frame not supported for
// SPLITMV.
Fix a unit test failure of sb8x8 on scaling ref Disable the use of scaled reference frame for motion search in SPLITMV mode. This fixes the unit test failure issue triggered when merging sb8x8 from experimental list. Change-Id: I02ac25fd8db8d5762f8fee29513b947189875fa0
2013-05-06 19:25:38 +02:00
if (mbmi->ref_frame > 0 &&
Make setup_pred_block subsampling-aware. Code previously set up the pointers by scaling by MI_UV_SIZE, which is 4:2:0 only. Change-Id: Ic13a92895cff018ec1345736746ed84cb31e6e31
2013-05-07 20:31:12 +02:00
(scale_factor[mbmi->ref_frame].x_num !=
scale_factor[mbmi->ref_frame].x_den ||
scale_factor[mbmi->ref_frame].y_num !=
scale_factor[mbmi->ref_frame].y_den) &&
Fix a unit test failure of sb8x8 on scaling ref Disable the use of scaled reference frame for motion search in SPLITMV mode. This fixes the unit test failure issue triggered when merging sb8x8 from experimental list. Change-Id: I02ac25fd8db8d5762f8fee29513b947189875fa0
2013-05-06 19:25:38 +02:00
this_mode == SPLITMV)
continue;
if (mbmi->second_ref_frame > 0 &&
Make setup_pred_block subsampling-aware. Code previously set up the pointers by scaling by MI_UV_SIZE, which is 4:2:0 only. Change-Id: Ic13a92895cff018ec1345736746ed84cb31e6e31
2013-05-07 20:31:12 +02:00
(scale_factor[mbmi->second_ref_frame].x_num !=
scale_factor[mbmi->second_ref_frame].x_den ||
scale_factor[mbmi->second_ref_frame].y_num !=
scale_factor[mbmi->second_ref_frame].y_den) &&
Fix a unit test failure of sb8x8 on scaling ref Disable the use of scaled reference frame for motion search in SPLITMV mode. This fixes the unit test failure issue triggered when merging sb8x8 from experimental list. Change-Id: I02ac25fd8db8d5762f8fee29513b947189875fa0
2013-05-06 19:25:38 +02:00
this_mode == SPLITMV)
continue;
Spatial resamping of ZEROMV predictors This patch allows coding frames using references of different resolution, in ZEROMV mode. For compound prediction, either reference may be scaled. To test, I use the resize_test and enable WRITE_RECON_BUFFER in vp9_onyxd_if.c. It's also useful to apply this patch to test/i420_video_source.h: --- a/test/i420_video_source.h +++ b/test/i420_video_source.h @@ -93,6 +93,7 @@ class I420VideoSource : public VideoSource { virtual void FillFrame() { // Read a frame from input_file. + if (frame_ != 3) if (fread(img_->img_data, raw_sz_, 1, input_file_) == 0) { limit_ = frame_; } This forces the frame that the resolution changes on to be coded with no motion, only scaling, and improves the quality of the result. Change-Id: I1ee75d19a437ff801192f767fd02a36bcbd1d496
2013-02-25 05:55:14 +01:00
set_scale_factors(xd, mbmi->ref_frame, mbmi->second_ref_frame,
scale_factor);
Supports inter-intra prediction with superblocks Adds support for compound inter-intra prediction with superblocks. Also, fixes a bug that disabled intra modes for superblocks. Change-Id: I4d711317e1bc19df8c2f32dc645429f7fff31036
2012-11-30 20:46:20 +01:00
comp_pred = mbmi->second_ref_frame > INTRA_FRAME;
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
mbmi->mode = this_mode;
mbmi->uv_mode = DC_PRED;
Spatial resamping of ZEROMV predictors This patch allows coding frames using references of different resolution, in ZEROMV mode. For compound prediction, either reference may be scaled. To test, I use the resize_test and enable WRITE_RECON_BUFFER in vp9_onyxd_if.c. It's also useful to apply this patch to test/i420_video_source.h: --- a/test/i420_video_source.h +++ b/test/i420_video_source.h @@ -93,6 +93,7 @@ class I420VideoSource : public VideoSource { virtual void FillFrame() { // Read a frame from input_file. + if (frame_ != 3) if (fread(img_->img_data, raw_sz_, 1, input_file_) == 0) { limit_ = frame_; } This forces the frame that the resolution changes on to be coded with no motion, only scaling, and improves the quality of the result. Change-Id: I1ee75d19a437ff801192f767fd02a36bcbd1d496
2013-02-25 05:55:14 +01:00
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
// Evaluate all sub-pel filters irrespective of whether we can use
// them for this frame.
Refactoring of switchable filter search for speed Refactors the switchable filter search in the rd loop to improve encode speed. Uses a piecewise approximation to a closed form expression to estimate rd cost for a Laplacian source with a given variance and quantization step-size. About 40% encode time reduction is achieved. Results (on a feb 12 baseline) show a slight drop: derf: -0.019% yt: +0.010% std-hd: -0.162% hd: -0.050% Change-Id: Ie861badf5bba1e3b1052e29a0ef1b7e256edbcd0
2013-02-12 02:08:52 +01:00
mbmi->interp_filter = cm->mcomp_filter_type;
vp9_setup_interp_filters(xd, mbmi->interp_filter, &cpi->common);
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Enable recursive partition down to 4x4 This commit allows the rate-distortion optimization recursion at encoder to go down to 4x4 block size. It deprecates the use of I4X4_PRED and SPLITMV syntax elements from bit-stream writing/reading. Will remove the unused probability models in the next patch. The partition type search and bit-stream are now capable of supporting the rectangular partition of 8x8 block, i.e., 8x4 and 4x8. Need to revise the rate-distortion parts to get these two partition tested in the rd loop. Change-Id: I0dfe3b90a1507ad6138db10cc58e6e237a06a9d6
2013-05-11 02:06:37 +02:00
if (bsize >= BLOCK_SIZE_SB8X8 &&
(this_mode == I4X4_PRED || this_mode == SPLITMV))
continue;
if (bsize < BLOCK_SIZE_SB8X8 &&
!(this_mode == I4X4_PRED || this_mode == SPLITMV))
continue;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
if (comp_pred) {
if (ref_frame == ALTREF_FRAME) {
second_ref = LAST_FRAME;
} else {
second_ref = ref_frame + 1;
}
if (!(cpi->ref_frame_flags & flag_list[second_ref]))
continue;
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
mbmi->second_ref_frame = second_ref;
Spatial resamping of ZEROMV predictors This patch allows coding frames using references of different resolution, in ZEROMV mode. For compound prediction, either reference may be scaled. To test, I use the resize_test and enable WRITE_RECON_BUFFER in vp9_onyxd_if.c. It's also useful to apply this patch to test/i420_video_source.h: --- a/test/i420_video_source.h +++ b/test/i420_video_source.h @@ -93,6 +93,7 @@ class I420VideoSource : public VideoSource { virtual void FillFrame() { // Read a frame from input_file. + if (frame_ != 3) if (fread(img_->img_data, raw_sz_, 1, input_file_) == 0) { limit_ = frame_; } This forces the frame that the resolution changes on to be coded with no motion, only scaling, and improves the quality of the result. Change-Id: I1ee75d19a437ff801192f767fd02a36bcbd1d496
2013-02-25 05:55:14 +01:00
set_scale_factors(xd, mbmi->ref_frame, mbmi->second_ref_frame,
scale_factor);
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
mode_excluded =
mode_excluded ?
mode_excluded : cm->comp_pred_mode == SINGLE_PREDICTION_ONLY;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
} else {
Supports inter-intra prediction with superblocks Adds support for compound inter-intra prediction with superblocks. Also, fixes a bug that disabled intra modes for superblocks. Change-Id: I4d711317e1bc19df8c2f32dc645429f7fff31036
2012-11-30 20:46:20 +01:00
// mbmi->second_ref_frame = vp9_mode_order[mode_index].second_ref_frame;
if (ref_frame != INTRA_FRAME) {
if (mbmi->second_ref_frame != INTRA_FRAME)
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
mode_excluded =
mode_excluded ?
mode_excluded : cm->comp_pred_mode == COMP_PREDICTION_ONLY;
Supports inter-intra prediction with superblocks Adds support for compound inter-intra prediction with superblocks. Also, fixes a bug that disabled intra modes for superblocks. Change-Id: I4d711317e1bc19df8c2f32dc645429f7fff31036
2012-11-30 20:46:20 +01:00
}
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
Make setup_pred_block subsampling-aware. Code previously set up the pointers by scaling by MI_UV_SIZE, which is 4:2:0 only. Change-Id: Ic13a92895cff018ec1345736746ed84cb31e6e31
2013-05-07 20:31:12 +02:00
// Select predictors
for (i = 0; i < MAX_MB_PLANE; i++) {
xd->plane[i].pre[0] = yv12_mb[ref_frame][i];
if (comp_pred)
xd->plane[i].pre[1] = yv12_mb[second_ref][i];
}
Move pre, second_pre to per-plane MACROBLOCKD data Continue moving framebuffers to per-plane data. Change-Id: I237e5a998b364c4ec20316e7249206c0bff8631a
2013-04-20 04:16:14 +02:00
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
// If the segment reference frame feature is enabled....
// then do nothing if the current ref frame is not allowed..
seg_common: give all exported symbols a vp9_ prefix. Change-Id: Ie8ba934a757acec1c80ac37ab9307c9a2783254e
2012-10-30 06:15:27 +01:00
if (vp9_segfeature_active(xd, segment_id, SEG_LVL_REF_FRAME) &&
!vp9_check_segref(xd, segment_id, ref_frame)) {
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
continue;
Segment Skip Flag First step in simplifying the segment mode and segment EOB flags into a simpler segment skip flag that implies 0,0 mv and EOB at position 0. Change-Id: Ib750cac31a7a02dc21082580498efd9f7d8d72a5
2013-01-28 16:22:53 +01:00
// If the segment skip feature is enabled....
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
// then do nothing if the current mode is not allowed..
Segment Skip Flag First step in simplifying the segment mode and segment EOB flags into a simpler segment skip flag that implies 0,0 mv and EOB at position 0. Change-Id: Ib750cac31a7a02dc21082580498efd9f7d8d72a5
2013-01-28 16:22:53 +01:00
} else if (vp9_segfeature_active(xd, segment_id, SEG_LVL_SKIP) &&
(this_mode != ZEROMV)) {
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
continue;
Segment Skip Flag First step in simplifying the segment mode and segment EOB flags into a simpler segment skip flag that implies 0,0 mv and EOB at position 0. Change-Id: Ib750cac31a7a02dc21082580498efd9f7d8d72a5
2013-01-28 16:22:53 +01:00
// Disable this drop out case if the ref frame
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
// segment level feature is enabled for this segment. This is to
// prevent the possibility that we end up unable to pick any mode.
Segment Skip Flag First step in simplifying the segment mode and segment EOB flags into a simpler segment skip flag that implies 0,0 mv and EOB at position 0. Change-Id: Ib750cac31a7a02dc21082580498efd9f7d8d72a5
2013-01-28 16:22:53 +01:00
} else if (!vp9_segfeature_active(xd, segment_id, SEG_LVL_REF_FRAME)) {
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
// Only consider ZEROMV/ALTREF_FRAME for alt ref frame,
// unless ARNR filtering is enabled in which case we want
// an unfiltered alternative
if (cpi->is_src_frame_alt_ref && (cpi->oxcf.arnr_max_frames == 0)) {
if (this_mode != ZEROMV || ref_frame != ALTREF_FRAME) {
continue;
}
}
}
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
if (this_mode == I4X4_PRED) {
int rate;
mbmi->txfm_size = TX_4X4;
rd_pick_intra4x4mby_modes(cpi, x, &rate, &rate_y,
&distortion_y, INT64_MAX);
rate2 += rate;
rate2 += intra_cost_penalty;
distortion2 += distortion_y;
rate2 += rate_uv_intra[TX_4X4];
rate_uv = rate_uv_intra[TX_4X4];
distortion2 += dist_uv[TX_4X4];
distortion_uv = dist_uv[TX_4X4];
mbmi->uv_mode = mode_uv[TX_4X4];
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
txfm_cache[ONLY_4X4] = RDCOST(x->rdmult, x->rddiv, rate2, distortion2);
for (i = 0; i < NB_TXFM_MODES; ++i)
txfm_cache[i] = txfm_cache[ONLY_4X4];
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
} else if (ref_frame == INTRA_FRAME) {
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
TX_SIZE uv_tx;
Make intra predictor support rectangular blocks The intra predictor supports configurable block sizes. It can handle intra prediction down to 4x4 sizes, when enabled in BLOCK_SIZE_TYPE. Change-Id: I7399ec2512393aa98aadda9813ca0c83e19af854
2013-04-11 21:12:11 +02:00
vp9_build_intra_predictors_sby_s(xd, bsize);
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
super_block_yrd(cpi, x, &rate_y, &distortion_y, &skippable,
bsize, txfm_cache);
uv_tx = mbmi->txfm_size;
Fix some crashes in sb8x8 experiment. Change-Id: I390bb1cedc835f439fd5dd6cda6572b29cbb139c
2013-05-01 23:45:27 +02:00
if (bsize < BLOCK_SIZE_MB16X16 && uv_tx == TX_8X8)
uv_tx = TX_4X4;
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
if (bsize < BLOCK_SIZE_SB32X32 && uv_tx == TX_16X16)
uv_tx = TX_8X8;
else if (bsize < BLOCK_SIZE_SB64X64 && uv_tx == TX_32X32)
uv_tx = TX_16X16;
rate_uv = rate_uv_intra[uv_tx];
distortion_uv = dist_uv[uv_tx];
skippable = skippable && skip_uv[uv_tx];
mbmi->uv_mode = mode_uv[uv_tx];
Support 32x32 intra modes in non-keyframe superblocks. Change-Id: Icf8ad313c543462e523bff89690e5daa8d49bcc0
2012-11-16 00:50:07 +01:00
rate2 = rate_y + x->mbmode_cost[cm->frame_type][mbmi->mode] + rate_uv;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
if (mbmi->mode != DC_PRED && mbmi->mode != TM_PRED)
rate2 += intra_cost_penalty;
Support 32x32 intra modes in non-keyframe superblocks. Change-Id: Icf8ad313c543462e523bff89690e5daa8d49bcc0
2012-11-16 00:50:07 +01:00
distortion2 = distortion_y + distortion_uv;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
} else if (this_mode == SPLITMV) {
const int is_comp_pred = mbmi->second_ref_frame > 0;
int rate, distortion;
int64_t this_rd_thresh;
int64_t tmp_rd, tmp_best_rd = INT64_MAX, tmp_best_rdu = INT64_MAX;
int tmp_best_rate = INT_MAX, tmp_best_ratey = INT_MAX;
int tmp_best_distortion = INT_MAX, tmp_best_skippable = 0;
int switchable_filter_index;
int_mv *second_ref = is_comp_pred ?
&mbmi->ref_mvs[mbmi->second_ref_frame][0] : NULL;
union b_mode_info tmp_best_bmodes[16];
MB_MODE_INFO tmp_best_mbmode;
PARTITION_INFO tmp_best_partition;
int pred_exists = 0;
int uv_skippable;
this_rd_thresh = (mbmi->ref_frame == LAST_FRAME) ?
cpi->rd_threshes[THR_NEWMV] : cpi->rd_threshes[THR_NEWA];
this_rd_thresh = (mbmi->ref_frame == GOLDEN_FRAME) ?
cpi->rd_threshes[THR_NEWG] : this_rd_thresh;
xd->mode_info_context->mbmi.txfm_size = TX_4X4;
for (switchable_filter_index = 0;
switchable_filter_index < VP9_SWITCHABLE_FILTERS;
++switchable_filter_index) {
int newbest;
mbmi->interp_filter =
vp9_switchable_interp[switchable_filter_index];
vp9_setup_interp_filters(xd, mbmi->interp_filter, &cpi->common);
tmp_rd = rd_pick_best_mbsegmentation(cpi, x,
&mbmi->ref_mvs[mbmi->ref_frame][0],
Remove unused mdcounts mdcounts seems no longer used. Change-Id: Idd8162e8acfa3f5be7a18767156cc79ccbc2bdee
2013-05-09 01:27:35 +02:00
second_ref, INT64_MAX,
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
&rate, &rate_y, &distortion,
&skippable,
(int)this_rd_thresh, seg_mvs);
if (cpi->common.mcomp_filter_type == SWITCHABLE) {
Adding get_switchable_rate function. Change-Id: I71311a14f8d7f48508b250f25d1d0914c6a1ac72
2013-05-07 21:23:14 +02:00
const int rs = get_switchable_rate(cm, x);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
tmp_rd += RDCOST(x->rdmult, x->rddiv, rs, 0);
}
newbest = (tmp_rd < tmp_best_rd);
if (newbest) {
tmp_best_filter = mbmi->interp_filter;
tmp_best_rd = tmp_rd;
}
if ((newbest && cm->mcomp_filter_type == SWITCHABLE) ||
(mbmi->interp_filter == cm->mcomp_filter_type &&
cm->mcomp_filter_type != SWITCHABLE)) {
tmp_best_rdu = tmp_rd;
tmp_best_rate = rate;
tmp_best_ratey = rate_y;
tmp_best_distortion = distortion;
tmp_best_skippable = skippable;
vpx_memcpy(&tmp_best_mbmode, mbmi, sizeof(MB_MODE_INFO));
vpx_memcpy(&tmp_best_partition, x->partition_info,
sizeof(PARTITION_INFO));
for (i = 0; i < 4; i++) {
tmp_best_bmodes[i] = xd->mode_info_context->bmi[i];
}
pred_exists = 1;
}
} // switchable_filter_index loop
mbmi->interp_filter = (cm->mcomp_filter_type == SWITCHABLE ?
tmp_best_filter : cm->mcomp_filter_type);
vp9_setup_interp_filters(xd, mbmi->interp_filter, &cpi->common);
if (!pred_exists) {
// Handles the special case when a filter that is not in the
// switchable list (bilinear, 6-tap) is indicated at the frame level
tmp_rd = rd_pick_best_mbsegmentation(cpi, x,
&mbmi->ref_mvs[mbmi->ref_frame][0],
Remove unused mdcounts mdcounts seems no longer used. Change-Id: Idd8162e8acfa3f5be7a18767156cc79ccbc2bdee
2013-05-09 01:27:35 +02:00
second_ref, INT64_MAX,
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
&rate, &rate_y, &distortion,
&skippable,
(int)this_rd_thresh, seg_mvs);
} else {
if (cpi->common.mcomp_filter_type == SWITCHABLE) {
Adding get_switchable_rate function. Change-Id: I71311a14f8d7f48508b250f25d1d0914c6a1ac72
2013-05-07 21:23:14 +02:00
int rs = get_switchable_rate(cm, x);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
tmp_best_rdu -= RDCOST(x->rdmult, x->rddiv, rs, 0);
}
tmp_rd = tmp_best_rdu;
rate = tmp_best_rate;
rate_y = tmp_best_ratey;
distortion = tmp_best_distortion;
skippable = tmp_best_skippable;
vpx_memcpy(mbmi, &tmp_best_mbmode, sizeof(MB_MODE_INFO));
vpx_memcpy(x->partition_info, &tmp_best_partition,
sizeof(PARTITION_INFO));
for (i = 0; i < 4; i++) {
xd->mode_info_context->bmi[i] = tmp_best_bmodes[i];
}
}
rate2 += rate;
distortion2 += distortion;
if (cpi->common.mcomp_filter_type == SWITCHABLE)
Adding get_switchable_rate function. Change-Id: I71311a14f8d7f48508b250f25d1d0914c6a1ac72
2013-05-07 21:23:14 +02:00
rate2 += get_switchable_rate(cm, x);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
// If even the 'Y' rd value of split is higher than best so far
// then dont bother looking at UV
vp9_build_inter_predictors_sbuv(&x->e_mbd, mi_row, mi_col,
Force bsize for UV in I4X4 and SPLITMV Use 4x4 block coding for UV components arbitrarily in I4X4_PRED and SPLITMV coding modes. This is a temporary solution to enable bit-stream support for recursive partition down to 4x4 block size. Will separate the functionalities of 4x4 block coding rate-distortion out from those of superblocks. Change-Id: I03dc15d5897014f175f3f2c91e9b266091d56797
2013-05-11 22:33:13 +02:00
BLOCK_SIZE_SB8X8);
vp9_subtract_sbuv(x, BLOCK_SIZE_SB8X8);
Separate transform and quant from vp9_encode_sb This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 02:05:14 +02:00
super_block_uvrd_for_txfm(cm, x, &rate_uv, &distortion_uv,
Force bsize for UV in I4X4 and SPLITMV Use 4x4 block coding for UV components arbitrarily in I4X4_PRED and SPLITMV coding modes. This is a temporary solution to enable bit-stream support for recursive partition down to 4x4 block size. Will separate the functionalities of 4x4 block coding rate-distortion out from those of superblocks. Change-Id: I03dc15d5897014f175f3f2c91e9b266091d56797
2013-05-11 22:33:13 +02:00
&uv_skippable, BLOCK_SIZE_SB8X8, TX_4X4);
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
rate2 += rate_uv;
distortion2 += distortion_uv;
skippable = skippable && uv_skippable;
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
txfm_cache[ONLY_4X4] = RDCOST(x->rdmult, x->rddiv, rate2, distortion2);
for (i = 0; i < NB_TXFM_MODES; ++i)
txfm_cache[i] = txfm_cache[ONLY_4X4];
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
if (!mode_excluded) {
if (is_comp_pred)
mode_excluded = cpi->common.comp_pred_mode == SINGLE_PREDICTION_ONLY;
else
mode_excluded = cpi->common.comp_pred_mode == COMP_PREDICTION_ONLY;
}
compmode_cost =
vp9_cost_bit(vp9_get_pred_prob(cm, xd, PRED_COMP), is_comp_pred);
mbmi->mode = this_mode;
Support 32x32 intra modes in non-keyframe superblocks. Change-Id: Icf8ad313c543462e523bff89690e5daa8d49bcc0
2012-11-16 00:50:07 +01:00
} else {
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
YV12_BUFFER_CONFIG *scaled_ref_frame[2] = {NULL, NULL};
Adding get_ref_frame_idx function. Change-Id: I4f1a4eca6794cda78d00512196caacd5567e2dcc
2013-05-21 00:59:39 +02:00
int fb = get_ref_frame_idx(cpi, mbmi->ref_frame);
Use ref_frame_map vice active_ref_idx on the encoder This patch makes the encoder's use of ref_frame_map and active_ref_idx consistent with the decoder. ref_frame_map[] maps a reference buffer index to its actual location in the yv12_fb array, since many references may share an underlying buffer. active_ref_idx[] mirrors cpi->{lst,gld,alt}_fb_idx, holding the active references in each slot. This also fixes a bug in setup_buffer_inter() where the incorrect reference was used to populate the scaling factors. Change-Id: Id3728f6d77cffcd27c248903bf51f9c3e594287e
2013-02-22 20:22:03 +01:00
if (cpi->scaled_ref_idx[fb] != cm->ref_frame_map[fb])
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
scaled_ref_frame[0] = &cm->yv12_fb[cpi->scaled_ref_idx[fb]];
if (comp_pred) {
Adding get_ref_frame_idx function. Change-Id: I4f1a4eca6794cda78d00512196caacd5567e2dcc
2013-05-21 00:59:39 +02:00
fb = get_ref_frame_idx(cpi, mbmi->second_ref_frame);
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
if (cpi->scaled_ref_idx[fb] != cm->ref_frame_map[fb])
scaled_ref_frame[1] = &cm->yv12_fb[cpi->scaled_ref_idx[fb]];
}
Combined motion compensation with scaled predictors This patch extends the previous support for using references of a different resolution in ZEROMV mode to all inter prediction modes. Subpixel based best-mv scoring is disabled when the reference frame differs in resolution from the current frame. Change-Id: Id4dc3e5e6692de98d9857fd56bfad3ac57e944ac
2013-02-20 21:34:31 +01:00
Make RD superblock mode search size-agnostic. Merge various super_block_yrd and super_block_uvrd versions into one common function that works for all sizes. Make transform size selection size-agnostic also. This fixes a slight bug in the intra UV superblock code where it used the wrong transform size for txsz > 8x8, and stores the txsz selection for superblocks properly (instead of forgetting it). Lastly, it removes the trellis search that was done for 16x16 intra predictors, since trellis is relatively expensive and should thus only be done after RD mode selection. Gives basically identical results on derf (+0.009%). Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-11 00:55:59 +02:00
this_rd = handle_inter_mode(cpi, x, bsize,
Remove unused mdcounts mdcounts seems no longer used. Change-Id: Idd8162e8acfa3f5be7a18767156cc79ccbc2bdee
2013-05-09 01:27:35 +02:00
txfm_cache,
Support 32x32 intra modes in non-keyframe superblocks. Change-Id: Icf8ad313c543462e523bff89690e5daa8d49bcc0
2012-11-16 00:50:07 +01:00
&rate2, &distortion2, &skippable,
&compmode_cost,
&rate_y, &distortion_y,
&rate_uv, &distortion_uv,
Pass macroblock index to pick inter functions Pass the current mb row and column around rather than the recon_yoffset and recon_uvoffset, since those offsets will change from predictor to predictor, based on the reference frame selection. Change-Id: If3f9df059e00f5048ca729d3d083ff428e1859c1
2013-02-07 19:09:05 +01:00
&mode_excluded, &disable_skip,
Decrease scope of frame_mv argument to handle_inter_mode(). Change-Id: I81c637c61ecc33cb66beb59a2a33166d66b9a0a2
2013-05-27 23:16:45 +02:00
&tmp_best_filter, frame_mv[this_mode],
Add joint motion search in comp_inter_inter mode(experiment) In current code, motion vectors got from single prediction mode are used in compound prediction mode directly. These motion vectors may not give accurate prediction since they are searched independently. In this patch, we took Pascal's suggestion, and did joint motion search in compound prediction mode to find better motion vectors in this situation. Test results: Overall PSNR: 0.570%(derf), 0.918%(stdhd); SSIM: 0.572%(derf), 1.009%(stdhd); The encoder is a little slower. This can be improved since some c code is used in motion search. Change-Id: Ib30c9240f6c56c9b070867b4ca89412a76d9f3c6
2013-05-07 18:45:28 +02:00
scaled_ref_frame, mi_row, mi_col,
single_newmv);
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
if (this_rd == INT64_MAX)
Support 32x32 intra modes in non-keyframe superblocks. Change-Id: Icf8ad313c543462e523bff89690e5daa8d49bcc0
2012-11-16 00:50:07 +01:00
continue;
}
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
if (cpi->common.comp_pred_mode == HYBRID_PREDICTION) {
rate2 += compmode_cost;
}
// Estimate the reference frame signaling cost and add it
// to the rolling cost variable.
rate2 += ref_costs[xd->mode_info_context->mbmi.ref_frame];
if (!disable_skip) {
// Test for the condition where skip block will be activated
// because there are no non zero coefficients and make any
// necessary adjustment for rate. Ignore if skip is coded at
// segment level as the cost wont have been added in.
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
int mb_skip_allowed;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
// Is Mb level skip allowed (i.e. not coded at segment level).
mb_skip_allowed = !vp9_segfeature_active(xd, segment_id, SEG_LVL_SKIP);
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
if (skippable && bsize >= BLOCK_SIZE_SB8X8) {
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
// Back out the coefficient coding costs
rate2 -= (rate_y + rate_uv);
// for best_yrd calculation
rate_uv = 0;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
if (mb_skip_allowed) {
int prob_skip_cost;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
// Cost the skip mb case
vp9_prob skip_prob =
vp9_get_pred_prob(cm, xd, PRED_MBSKIP);
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
if (skip_prob) {
prob_skip_cost = vp9_cost_bit(skip_prob, 1);
rate2 += prob_skip_cost;
other_cost += prob_skip_cost;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
}
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
} else if (mb_skip_allowed) {
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
// Add in the cost of the no skip flag.
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
int prob_skip_cost = vp9_cost_bit(vp9_get_pred_prob(cm, xd,
PRED_MBSKIP), 0);
rate2 += prob_skip_cost;
other_cost += prob_skip_cost;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
// Calculate the final RD estimate for this mode.
this_rd = RDCOST(x->rdmult, x->rddiv, rate2, distortion2);
}
#if 0
// Keep record of best intra distortion
if ((xd->mode_info_context->mbmi.ref_frame == INTRA_FRAME) &&
(this_rd < best_intra_rd)) {
best_intra_rd = this_rd;
*returnintra = distortion2;
}
#endif
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
if (!disable_skip && mbmi->ref_frame == INTRA_FRAME)
for (i = 0; i < NB_PREDICTION_TYPES; ++i)
best_pred_rd[i] = MIN(best_pred_rd[i], this_rd);
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Supports inter-intra prediction with superblocks Adds support for compound inter-intra prediction with superblocks. Also, fixes a bug that disabled intra modes for superblocks. Change-Id: I4d711317e1bc19df8c2f32dc645429f7fff31036
2012-11-30 20:46:20 +01:00
if (this_rd < best_overall_rd) {
best_overall_rd = this_rd;
Refactoring of switchable filter search for speed Refactors the switchable filter search in the rd loop to improve encode speed. Uses a piecewise approximation to a closed form expression to estimate rd cost for a Laplacian source with a given variance and quantization step-size. About 40% encode time reduction is achieved. Results (on a feb 12 baseline) show a slight drop: derf: -0.019% yt: +0.010% std-hd: -0.162% hd: -0.050% Change-Id: Ie861badf5bba1e3b1052e29a0ef1b7e256edbcd0
2013-02-12 02:08:52 +01:00
best_filter = tmp_best_filter;
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
best_mode = this_mode;
Supports inter-intra prediction with superblocks Adds support for compound inter-intra prediction with superblocks. Also, fixes a bug that disabled intra modes for superblocks. Change-Id: I4d711317e1bc19df8c2f32dc645429f7fff31036
2012-11-30 20:46:20 +01:00
}
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
// Store the respective mode distortions for later use.
if (mode_distortions[this_mode] == -1
|| distortion2 < mode_distortions[this_mode]) {
mode_distortions[this_mode] = distortion2;
}
if (frame_distortions[mbmi->ref_frame] == -1
|| distortion2 < frame_distortions[mbmi->ref_frame]) {
frame_distortions[mbmi->ref_frame] = distortion2;
}
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
// Did this mode help.. i.e. is it the new best mode
if (this_rd < best_rd || x->skip) {
if (!mode_excluded) {
// Note index of best mode so far
best_mode_index = mode_index;
Rename B_PRED to I4X4_PRED So it is consistent with I8x8_PRED. Change-Id: Iefa65124b2419690d83e526c611129c0ede29d11
2013-04-11 21:16:35 +02:00
if (this_mode <= I4X4_PRED) {
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
/* required for left and above block mv */
Support 32x32 intra modes in non-keyframe superblocks. Change-Id: Icf8ad313c543462e523bff89690e5daa8d49bcc0
2012-11-16 00:50:07 +01:00
mbmi->mv[0].as_int = 0;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
other_cost += ref_costs[xd->mode_info_context->mbmi.ref_frame];
*returnrate = rate2;
*returndistortion = distortion2;
best_rd = this_rd;
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
vpx_memcpy(&best_mbmode, mbmi, sizeof(MB_MODE_INFO));
Store splitmv modes in context after 8x8 rd loop. Change-Id: I07aa89a67e0ac5f99ef0c448553dbc46b0ed27f2
2013-05-01 23:43:33 +02:00
vpx_memcpy(&best_partition, x->partition_info, sizeof(PARTITION_INFO));
if (this_mode == I4X4_PRED || this_mode == SPLITMV) {
for (i = 0; i < 4; i++) {
best_bmodes[i] = xd->mode_info_context->bmi[i];
}
}
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
#if 0
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
// Testing this mode gave rise to an improvement in best error score.
// Lower threshold a bit for next time
cpi->rd_thresh_mult[mode_index] =
(cpi->rd_thresh_mult[mode_index] >= (MIN_THRESHMULT + 2)) ?
cpi->rd_thresh_mult[mode_index] - 2 : MIN_THRESHMULT;
cpi->rd_threshes[mode_index] =
(cpi->rd_baseline_thresh[mode_index] >> 7)
* cpi->rd_thresh_mult[mode_index];
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
#endif
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
} else {
// If the mode did not help improve the best error case then
// raise the threshold for testing that mode next time around.
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
#if 0
cpi->rd_thresh_mult[mode_index] += 4;
if (cpi->rd_thresh_mult[mode_index] > MAX_THRESHMULT)
cpi->rd_thresh_mult[mode_index] = MAX_THRESHMULT;
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
cpi->rd_threshes[mode_index] =
(cpi->rd_baseline_thresh[mode_index] >> 7)
* cpi->rd_thresh_mult[mode_index];
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
#endif
}
/* keep record of best compound/single-only prediction */
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
if (!disable_skip && mbmi->ref_frame != INTRA_FRAME) {
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
int single_rd, hybrid_rd, single_rate, hybrid_rate;
if (cpi->common.comp_pred_mode == HYBRID_PREDICTION) {
single_rate = rate2 - compmode_cost;
hybrid_rate = rate2;
} else {
single_rate = rate2;
hybrid_rate = rate2 + compmode_cost;
}
single_rd = RDCOST(x->rdmult, x->rddiv, single_rate, distortion2);
hybrid_rd = RDCOST(x->rdmult, x->rddiv, hybrid_rate, distortion2);
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
if (mbmi->second_ref_frame <= INTRA_FRAME &&
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
single_rd < best_pred_rd[SINGLE_PREDICTION_ONLY]) {
best_pred_rd[SINGLE_PREDICTION_ONLY] = single_rd;
Compound inter-intra experiment A patch on compound inter-intra prediction. In compound inter-intra prediction, a new predictor for 16x16 inter coded MBs are obtained by combining a single inter predictor with a 16x16 intra predictor, in a manner that the weight varies with distance from the top/left boundary. The current search strategy is to combine the best inter mode with the best intra mode obtained independently. Results so far: derf +0.31% yt +0.32% std-hd +0.35% hd +0.42% It is conceivable that the results would improve somewhat with a more thorough search strategy where all intra modes are searched given the best mv, or even a joint search for the best mv and the best intra mode. Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-07 15:50:25 +01:00
} else if (mbmi->second_ref_frame > INTRA_FRAME &&
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
single_rd < best_pred_rd[COMP_PREDICTION_ONLY]) {
best_pred_rd[COMP_PREDICTION_ONLY] = single_rd;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
if (hybrid_rd < best_pred_rd[HYBRID_PREDICTION])
best_pred_rd[HYBRID_PREDICTION] = hybrid_rd;
}
/* keep record of best txfm size */
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
if (bsize < BLOCK_SIZE_SB32X32) {
if (bsize < BLOCK_SIZE_MB16X16) {
if (this_mode == SPLITMV || this_mode == I4X4_PRED)
txfm_cache[ALLOW_8X8] = txfm_cache[ONLY_4X4];
txfm_cache[ALLOW_16X16] = txfm_cache[ALLOW_8X8];
}
txfm_cache[ALLOW_32X32] = txfm_cache[ALLOW_16X16];
}
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
if (!mode_excluded && this_rd != INT64_MAX) {
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
for (i = 0; i < NB_TXFM_MODES; i++) {
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
int64_t adj_rd = INT64_MAX;
Rename B_PRED to I4X4_PRED So it is consistent with I8x8_PRED. Change-Id: Iefa65124b2419690d83e526c611129c0ede29d11
2013-04-11 21:16:35 +02:00
if (this_mode != I4X4_PRED) {
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
adj_rd = this_rd + txfm_cache[i] - txfm_cache[cm->txfm_mode];
} else {
adj_rd = this_rd;
}
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
if (adj_rd < best_txfm_rd[i])
best_txfm_rd[i] = adj_rd;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
}
if (x->skip && !mode_excluded)
break;
}
new version of speed 1 This version of speed 1 only disables modes at higher resolution that had distortions >2x the best mode we found... The hope is that this could be a replacement for speed 0 ... Change-Id: I7421f1016b8958314469da84c4dccddf25390720
2013-04-23 00:42:41 +02:00
// Flag all modes that have a distortion thats > 2x the best we found at
// this level.
for (mode_index = 0; mode_index < MB_MODE_COUNT; ++mode_index) {
if (mode_index == NEARESTMV || mode_index == NEARMV || mode_index == NEWMV
|| mode_index == SPLITMV)
continue;
if (mode_distortions[mode_index] > 2 * *returndistortion) {
ctx->modes_with_high_error |= (1 << mode_index);
}
}
// Flag all ref frames that have a distortion thats > 2x the best we found at
// this level.
for (ref_frame = INTRA_FRAME; ref_frame <= ALTREF_FRAME; ref_frame++) {
if (frame_distortions[ref_frame] > 2 * *returndistortion) {
ctx->frames_with_high_error |= (1 << ref_frame);
}
}
Enable recursive partition down to 4x4 This commit allows the rate-distortion optimization recursion at encoder to go down to 4x4 block size. It deprecates the use of I4X4_PRED and SPLITMV syntax elements from bit-stream writing/reading. Will remove the unused probability models in the next patch. The partition type search and bit-stream are now capable of supporting the rectangular partition of 8x8 block, i.e., 8x4 and 4x8. Need to revise the rate-distortion parts to get these two partition tested in the rd loop. Change-Id: I0dfe3b90a1507ad6138db10cc58e6e237a06a9d6
2013-05-11 02:06:37 +02:00
if (best_rd == INT64_MAX && bsize < BLOCK_SIZE_SB8X8) {
*returnrate = INT_MAX;
*returndistortion = INT_MAX;
return best_rd;
}
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
assert((cm->mcomp_filter_type == SWITCHABLE) ||
(cm->mcomp_filter_type == best_mbmode.interp_filter) ||
Rename B_PRED to I4X4_PRED So it is consistent with I8x8_PRED. Change-Id: Iefa65124b2419690d83e526c611129c0ede29d11
2013-04-11 21:16:35 +02:00
(best_mbmode.mode <= I4X4_PRED));
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
// Accumulate filter usage stats
// TODO(agrange): Use RD criteria to select interpolation filter mode.
Adding is_inter_mode function. Change-Id: I2d32d46002cb92c63050c2b8328865c406103621
2013-04-24 21:14:58 +02:00
if (is_inter_mode(best_mode))
New interpolation filter selection algorithm Old Scheme: When SWITCHABLE filter selection is enabled the encoder evaluates the use of each interpolation filter type and selects the best one to use at the MB level. A frame- level flag can be set to force the use of a particular filter type for all MBs in a frame if it is more efficient to encode that way. The logic here involved a Q dependent threshold that assumed that the second 8-tap filter was a high-pass filter. However, this requires a trip around the recode loop. If the frame-level flag indicates use of a particular filter, the other filters are not evaluated in the pick_mode loop. New Scheme: Each filter type is evaluated at the MB level and a record of the best filter is kept, irrespective of what filter is signaled at the frame-level. Once all MBs have been encoded, a decision is made as to what frame-level mode to set for the *next* frame. If one filter is used by 80% or more of the MBs, then this filter is forced since it is assumed that this will be more efficient if the next frame has similar characteristics. i.e. there is a one-frame lag between measuring the filter selection and setting the frame-level mode to use. Change-Id: I6a7e7ced8f27e120fafb99db2dc9c6293f8d20f7
2012-12-20 23:56:19 +01:00
++cpi->best_switchable_interp_count[vp9_switchable_interp_map[best_filter]];
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
// TODO(rbultje) integrate with RD thresholding
#if 0
// Reduce the activation RD thresholds for the best choice mode
if ((cpi->rd_baseline_thresh[best_mode_index] > 0) &&
(cpi->rd_baseline_thresh[best_mode_index] < (INT_MAX >> 2))) {
int best_adjustment = (cpi->rd_thresh_mult[best_mode_index] >> 2);
cpi->rd_thresh_mult[best_mode_index] =
(cpi->rd_thresh_mult[best_mode_index] >= (MIN_THRESHMULT + best_adjustment)) ?
cpi->rd_thresh_mult[best_mode_index] - best_adjustment : MIN_THRESHMULT;
cpi->rd_threshes[best_mode_index] =
(cpi->rd_baseline_thresh[best_mode_index] >> 7) * cpi->rd_thresh_mult[best_mode_index];
}
#endif
// This code forces Altref,0,0 and skip for the frame that overlays a
// an alrtef unless Altref is filtered. However, this is unsafe if
Segment Skip Flag First step in simplifying the segment mode and segment EOB flags into a simpler segment skip flag that implies 0,0 mv and EOB at position 0. Change-Id: Ib750cac31a7a02dc21082580498efd9f7d8d72a5
2013-01-28 16:22:53 +01:00
// segment level coding of ref frame is enabled for this segment.
seg_common: give all exported symbols a vp9_ prefix. Change-Id: Ie8ba934a757acec1c80ac37ab9307c9a2783254e
2012-10-30 06:15:27 +01:00
if (!vp9_segfeature_active(xd, segment_id, SEG_LVL_REF_FRAME) &&
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
cpi->is_src_frame_alt_ref &&
(cpi->oxcf.arnr_max_frames == 0) &&
Enable bit-stream support to 8x4 and 4x8 partition The recursive partition type search is enabled down to 4x4, 4x8 and 8x4, followed by the corresponding rate-distortion optimization for the per-partition encoding mode decisions. The bit-stream writing/reading synchronized in supporting the rectangular partition of 8x8 block. This provides above 1% coding performance gains on derf. To do next: 1. re-design the rate-distortion loop for inter prediction below 8x8. 2. re-design the rate-distortion loop for intra prediction below 4x4. 3. make the loop-filter aware of rectangular partition of 8x8 block. 4. clean the unused probability models. 5. update default probability values. Change-Id: Idd41a315b16879db08f045a322241f46f1d53f20
2013-05-16 07:28:36 +02:00
(best_mbmode.mode != ZEROMV || best_mbmode.ref_frame != ALTREF_FRAME)
Merge 4x4 block level partition into codebase Move 4x4/4x8/8x4 partition coding out of experimental list. This commit fixed the unit test failure issues. It also resolved the merge conflicts between 4x4 block level partition and iterative motion search for comp_inter_inter. Change-Id: I898671f0631f5ddc4f5cc68d4c62ead7de9c5a58
2013-05-22 06:28:42 +02:00
&& bsize >= BLOCK_SIZE_SB8X8) {
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
mbmi->mode = ZEROMV;
mbmi->ref_frame = ALTREF_FRAME;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
mbmi->second_ref_frame = NONE;
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
mbmi->mv[0].as_int = 0;
mbmi->uv_mode = DC_PRED;
Remove the mb_no_coeff_skip flag This flag was added to VP8 to allow a mode where MB-level skipping was not allowed, saving a bit per mb. It was never used in practice, and hasn't been tested in VP9, so remove it. Change-Id: Id450ec6904c6d06c1919508e7efc52d05cde5631
2013-04-09 19:54:19 +02:00
mbmi->mb_skip_coeff = 1;
sb8x8 integration in rd loop. Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-05-01 01:13:20 +02:00
if (cm->txfm_mode == TX_MODE_SELECT) {
if (bsize >= BLOCK_SIZE_SB32X32)
mbmi->txfm_size = TX_32X32;
else if (bsize >= BLOCK_SIZE_MB16X16)
mbmi->txfm_size = TX_16X16;
else
mbmi->txfm_size = TX_8X8;
}
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
vpx_memset(best_txfm_diff, 0, sizeof(best_txfm_diff));
vpx_memset(best_pred_diff, 0, sizeof(best_pred_diff));
goto end;
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
}
// macroblock modes
Fix superblock experiment. Also merge some duplicate code related to the superblock experiment in the RD loop. Change-Id: Ic93f1d4d1ed81220fd7ecf6e65da2821a215b2de
2012-10-30 01:58:18 +01:00
vpx_memcpy(mbmi, &best_mbmode, sizeof(MB_MODE_INFO));
Store splitmv modes in context after 8x8 rd loop. Change-Id: I07aa89a67e0ac5f99ef0c448553dbc46b0ed27f2
2013-05-01 23:43:33 +02:00
if (best_mbmode.mode == I4X4_PRED) {
for (i = 0; i < 4; i++) {
xd->mode_info_context->bmi[i].as_mode = best_bmodes[i].as_mode;
}
}
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
Store splitmv modes in context after 8x8 rd loop. Change-Id: I07aa89a67e0ac5f99ef0c448553dbc46b0ed27f2
2013-05-01 23:43:33 +02:00
if (best_mbmode.mode == SPLITMV) {
for (i = 0; i < 4; i++)
xd->mode_info_context->bmi[i].as_mv[0].as_int =
best_bmodes[i].as_mv[0].as_int;
if (mbmi->second_ref_frame > 0)
for (i = 0; i < 4; i++)
xd->mode_info_context->bmi[i].as_mv[1].as_int =
best_bmodes[i].as_mv[1].as_int;
vpx_memcpy(x->partition_info, &best_partition, sizeof(PARTITION_INFO));
mbmi->mv[0].as_int = x->partition_info->bmi[3].mv.as_int;
mbmi->mv[1].as_int = x->partition_info->bmi[3].second_mv.as_int;
}
Merge SB8X8 into the codebase Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-05 07:09:43 +02:00
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
for (i = 0; i < NB_PREDICTION_TYPES; ++i) {
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
if (best_pred_rd[i] == INT64_MAX)
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
best_pred_diff[i] = INT_MIN;
else
best_pred_diff[i] = best_rd - best_pred_rd[i];
}
if (!x->skip) {
for (i = 0; i < NB_TXFM_MODES; i++) {
Use INT64_MAX instead of LLONG_MAX These variables have the type int64_t, not long long. long long could be a larger type than 64 bits. Emulate INT64_MAX for older versions of MSVC, and remove the unreferenced vpx_ports/vpxtypes.h Change-Id: Ideaca71838fcd3849d816d5ab17aa347c97d03b0
2013-01-14 20:49:30 +01:00
if (best_txfm_rd[i] == INT64_MAX)
Prevent filling transform size cache with uninitialized values. The 32x32 value in case of splitmv was uninitialized. this leads to all kind of erratic behaviour down the line. Also fill in dummy values for superblocks in keyframes (the values are currently unused, but we run into integer overflows anyway, which makes detecting bad cases harder). Lastly, in case we did not find any RD value at all, don't set tx_diff to INT_MIN, but instead set it to zero (since if we couldn't find a mode, it's unlikely that any particular transform would have made that worse or better; rather, it's likely equally bad for all tx_sizes). Change-Id: If236fd3aa2037e5b398d03f3b1978fbbc5ce740e
2013-02-16 00:55:31 +01:00
best_txfm_diff[i] = 0;
Implement tx_select for superblock encoding. Also split superblock handling code out of decode_macroblock() into a new function decode_superblock(), for easier readability. Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains by allowing to select mb_skip_coeff for a subset of the complete SB or something along those lines, because although this change allows coding smaller transforms for bigger predictors, it increases the overhead of coding EOBs to skip the parts where the residual is near-zero, and thus the overall gain is not as high as we'd expect. Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 20:03:00 +01:00
else
best_txfm_diff[i] = best_rd - best_txfm_rd[i];
}
} else {
vpx_memset(best_txfm_diff, 0, sizeof(best_txfm_diff));
}
end:
Spatial resamping of ZEROMV predictors This patch allows coding frames using references of different resolution, in ZEROMV mode. For compound prediction, either reference may be scaled. To test, I use the resize_test and enable WRITE_RECON_BUFFER in vp9_onyxd_if.c. It's also useful to apply this patch to test/i420_video_source.h: --- a/test/i420_video_source.h +++ b/test/i420_video_source.h @@ -93,6 +93,7 @@ class I420VideoSource : public VideoSource { virtual void FillFrame() { // Read a frame from input_file. + if (frame_ != 3) if (fread(img_->img_data, raw_sz_, 1, input_file_) == 0) { limit_ = frame_; } This forces the frame that the resolution changes on to be coded with no motion, only scaling, and improves the quality of the result. Change-Id: I1ee75d19a437ff801192f767fd02a36bcbd1d496
2013-02-25 05:55:14 +01:00
set_scale_factors(xd, mbmi->ref_frame, mbmi->second_ref_frame,
scale_factor);
Store splitmv modes in context after 8x8 rd loop. Change-Id: I07aa89a67e0ac5f99ef0c448553dbc46b0ed27f2
2013-05-01 23:43:33 +02:00
store_coding_context(x, ctx, best_mode_index,
&best_partition,
Merge pick_sb_modes and pick_sb64_modes. Change-Id: Iad69e7a3b7e470acf6094f6a52e7da69066fd552
2013-04-11 18:33:49 +02:00
&mbmi->ref_mvs[mbmi->ref_frame][0],
&mbmi->ref_mvs[mbmi->second_ref_frame < 0 ? 0 :
mbmi->second_ref_frame][0],
best_pred_diff, best_txfm_diff);
Superblock coding. This commit adds a pick_sb_mode() function which selects the best 32x32 superblock coding mode. Then it selects the best per-MB modes, compares the two and encodes that in the bitstream. The bitstream coding is rather simplistic right now. At the SB level, we code a bit to indicate whether this block uses SB-coding (32x32 prediction) or MB-coding (anything else), and then we follow with the actual modes. This could and should be modified in the future, but is omitted from this commit because it will likely involve reorganizing much more code rather than just adding SB coding, so it's better to let that be judged on its own merits. Gains on derf: about even, YT/HD: +0.75%, STD/HD: +1.5%. Change-Id: Iae313a7cbd8f75b3c66d04a68b991cb096eaaba6
2012-08-20 23:43:34 +02:00
return best_rd;
}
Reference in New Issue Copy Permalink