Commit Graph

388 Commits

Author SHA1 Message Date
hui su
7fa61d7d51 Fix rate cost calculation for ext-intra
It was broken by commit 8ee640f979.

Change-Id: I26b9eba810c74849b0805e64da2d269ab0685cb9
2016-06-14 13:46:17 -07:00
Debargha Mukherjee
902ee5060c A crash fix for supertx / ext-inter combination.
Change-Id: I9860376c98aa3b25f5bf86ed13d4a7631fa6b153
2016-06-13 13:57:30 -07:00
Jingning Han
4588676cfb Merge "Trellis based adaptive quantization" into nextgenv2 2016-06-13 17:36:19 +00:00
Debargha Mukherjee
81f8b3f31c Merge "Some refactoring to support warped motion mode" into nextgenv2 2016-06-10 23:18:39 +00:00
Jingning Han
25ca322957 Trellis based adaptive quantization
This commit combines uniform quantizer with trellis based coefficient
level optimization. It improves the codebase compression performance:

lowres 0.8%
midres 1.0%
hdres  1.6%

Note that the current trellis optimization unit is using C code. This
will make the cost of the overall quantization process slower. A number
of optimizations will come up next.

Change-Id: Id441dd238e4844409d0f08f82604be777f3f5282
2016-06-10 12:56:14 -07:00
Debargha Mukherjee
03be30ba3e Some refactoring to support warped motion mode
Change-Id: I15d54a3ae48b2b33082668116792c6595bdb3ddb
2016-06-10 12:04:18 -07:00
Sarah Parker
a21afd421b Move new quant experiment from nextgen
This experiment implements non-uniform quantization where
the width of the bins increases gradually to more closely
match a laplacian distribution of the coeficcients.

Performance Gain:
derflr: 0.15%
hevcmr: 0.675%

Change-Id: I25234244e3bcd94b87c1f77cf682190b61c8ef94
2016-06-10 08:06:22 -07:00
Angie Chiang
95340fccb3 Revert "Optimize wedge partition selection."
This reverts commit efda2831e5.

This commit causes segmentation fault at SSE2/SumSquares2DTest.RandomValues/0

Change-Id: I171937e4daf6f15323e8206418773deb03bd8c53
2016-06-09 19:17:37 -07:00
Jingning Han
cedf90a9d6 Merge "Remove swap buffer speed feature" into nextgenv2 2016-06-08 19:45:54 +00:00
Jingning Han
0d6980d7a1 Remove swap buffer speed feature
The inter prediction residual can undergo different transform types
during the rate-distortion optimization search. The assumption used
in this speed feature no longer holds true. This commit removes the
related code to clean up the codebase and clear out unit test
failure in higher speed setting.

Change-Id: I7f7cd4df2345ed3e607c9fae75b38cd2dbde0cac
2016-06-08 11:27:00 -07:00
Jingning Han
b48eb90023 Merge "Add tx type speed feature to recursive transform block partitioning" into nextgenv2 2016-06-07 23:44:01 +00:00
Jingning Han
0b7f864213 Merge "Rework the tx type speed feature" into nextgenv2 2016-06-07 23:42:43 +00:00
Jingning Han
33dafdb58b Add tx type speed feature to recursive transform block partitioning
Change-Id: I45440a72b4287d98cbe21b72defc67138a8eb953
2016-06-07 11:34:30 -07:00
Jingning Han
9a858e868c Rework the tx type speed feature
This commit re-works the transform type speed feature. It moves
the transform type selection outside of the coding mode loop. This
avoids repeated motion search if the best prediction mode is
chosen as NEWMV. It improves the speed performance for clips that
contain more motion activities.

For mobile_cif at 1000 kbps, this makes the baseline encoding 7%
faster and makes the encoding with dynamic motion vector referencing
scheme enabled 10% faster.

Change-Id: I93e2714b3e461303372c4b66a4134ee212faffd1
2016-06-07 11:32:27 -07:00
Zoe Liu
5414abb4a0 Fix a RD performance bug in bipredictive frames
This patch will make sure the use of the BWDREF_FRAME for the
encoding of both the two types of bipredictive frames, namely
LAST_BIPRED_UPDATE and BIPRED_UPDATE. To realize it, the
updates on the cpi->ref_frame_flags have been moved to before
the encoding of one frame, instread of originally handled after
the encoding of one frame.

RD performance has been improved slightly, approximately by 0.17%
compared to before the applying of this patch:

lowres: Avg -3.474; BDRate -3.324
derflr: Avg -2.097; BDRate -1.353

Change-Id: I0aa19afd752293e345489fbff104c4351ca5498c
2016-06-07 09:45:10 -07:00
Debargha Mukherjee
13155e7725 Merge "Optimize wedge partition selection." into nextgenv2 2016-06-07 09:50:13 +00:00
Jingning Han
3713949b6d Merge "Make ref-mv experiment support ActiveMap" into nextgenv2 2016-06-06 16:06:41 +00:00
Geza Lore
efda2831e5 Optimize wedge partition selection.
We can optimize wedge partition selection by pre-computing the
residuals of the 2 underlying predictors, and then blend these
to compute the sse of the compound predictor, without actually
having to compute and subtract the compound predictor.

Similarly we can pre-compute a proxy array which we can use to
cheaply check which mask sign would have lower sse.

Details are in wedge_utils.c.

Mathematically these are equivalence transformations, but due to the
finite precision the encoder output will be perturbed, though on
average this should make 0% difference.

ext-inter gains about ~4.5% speedup.

Change-Id: Ib2657c3209ae161b4090b58b4b6c392641bf2792
2016-06-06 14:43:10 +01:00
Debargha Mukherjee
b85d0adadf Merge "Always include the cost of tx size in rate for Y." into nextgenv2 2016-06-03 22:57:17 +00:00
Debargha Mukherjee
33c57e6223 Merge "Check if sub8x8 rd stats are valid before reusing them." into nextgenv2 2016-06-03 22:38:56 +00:00
Jingning Han
27d8a948c1 Make ref-mv experiment support ActiveMap
Reset the ref_mv_idx and predicted motion vector when the coding
block belongs to skip segment.

Change-Id: I5746ab315a436b829b64a1a25121989d3c11c995
2016-06-03 15:04:18 -07:00
Geza Lore
b87078d51e Always include the cost of tx size in rate for Y.
The transform can only be skipped if both Y and U/V can be skipped, so
we always include the cost of tx size in the rate for Y. This will
get later subtracted if the transform is actually skipped.

Change-Id: I136a223e5596f18b69bb9f743e7e08438183a215
2016-06-03 11:51:35 -07:00
Geza Lore
d9870c32a9 Check if sub8x8 rd stats are valid before reusing them.
Change-Id: I5d49f15a07de58c226d4003b4691e001abf1f3f8
2016-06-03 11:47:34 -07:00
Geza Lore
8ee640f979 Compute cost of UV mode accurately for intra blocks.
We used to cache the cost of the UV mode from the search with a
different previously tried Y mode, but the UV mode is contexted
on the Y mode, so caching the cost is inaccurate.

Change-Id: Ib003510afb6fc9befb7808b67b0be64f1c0a0804
2016-06-03 11:13:51 -07:00
Geza Lore
73bc3119be Factor out model_rd_from_sse
Change-Id: Ia60ff0ecc8d083870fadbfe07d494d1e2c080489
2016-06-03 09:34:55 +01:00
Geza Lore
ab29978e9f Pre-compute and use contiguous wedge masks.
This is purely a refactoring patch and has no functional effect.

Uses of these masks can be arranged such that all input blocks are
contiguous in memory (stride == block width). In this case 1D versions
of  operations can be used. 1D vector operations have superior performance
over 2D block equivalents as they are more processor cache friendly and
they can do away with a second loop overhead.

Change-Id: I2b76c9888aea2c857cc497e8a4b2841fd3dad54e
2016-06-03 00:16:22 -07:00
hui su
fa933553da ext-intra: speed up keyframe encoding
130% speed increase for keyframe encoding, with 0.4%
compression loss.

When kf-max-dist=150, 1.5% speed increase with 0.03%
compression loss.

Change-Id: I4cf7314ab95b9eb6dd17f314aca8955522c82676
2016-05-31 10:34:44 -07:00
hui su
f523d7b540 Add a speed feature for inter tx type search
Seperate prediction mode and tx type search for inter
modes. Enabled for speed >=1.

baseline:
speed increase     40%
compression drop   0.30%/0.29% on lowres/midres

ext-tx:
speed increase    160%
compression drop  1.08%/0.95% on lowres/midres

Change-Id: Ieb34b1ee80df6980d16e26a5783e08cc0deae55b
2016-05-31 10:34:35 -07:00
hui su
38e6dd71bb Add a speed feature for intra tx type search
Add a speed feature to seperate prediction mode and tx type search
for intra modes: search for best intra prediction mode with fixed
default tx type first, then choose the best tx type for the
selected mode.

Coding performance drop:
baseline
  lowres 0.10% midres 0.08% hdres 0.14%
with ext-tx
  lowres 0.14% midres 0.25% hdres 0.20%

Speed improvement is 20% for baseline and 17% for ext-tx.

It is turned on for speed >= 1.

Change-Id: Ia5e8d39e8a4e2e42c521bfde938f8b6a98ab24f9
2016-05-31 10:33:56 -07:00
Zoe Liu
e89ca180c2 Make the bi-predictive frame group interval adjustable
This is for the bidir-pred experiment. Previously the length of the
bi-predictive frame group interval is fixed at 2, i.e. one
bi-predictive frame may be inserted every other frame. This patch
makes the length adjustable, i.e. any positive number may be
specified, but the use of the backward ref will be turned off if the
bi-predictive frame group interval is larger than the golden frame
group.

Further, an additional rate factor level has been added:
INTER_LOW
, which applies to LAST_BIPRED_UPDATE frames that are not used as
references.

Change-Id: I5514d34a64dd486bbb5756c2d0612946f598a789
2016-05-28 16:46:45 -07:00
Hui Su
88eaf5d6ce Merge "Skip unnecessary calculations in ext-intra" into nextgenv2 2016-05-26 18:03:02 +00:00
Zoe Liu
cf5083d4cd Added an experiment "bidir_pred" for backward prediction
Major parts have been implemented as follows:
(1) Added BRF_UPDATE, LASTNRF_UPDATE, and NRF_UPDATE in firstpass.c;
(2) Added the handling for the scenario of
"cpi->common.show_existing_frame == 1" at the encoder;
(3) Added a new reference frame of BWDREF_FRAME;
(4) Have bwd-ref work with upsampled references.

Note that when the experiment of "ext_refs" turned on, this experiment
will be turned off automatically currently.

RD performance in Overall PSNR has been improved, compared against the
VP10 baseline:

lowres: Avg -3.312; BDRate -3.154
derflr: Avg -1.927; BDRate -1.176
midres: Avg -2.149; BDRate -2.001
hdres : Avg -0.567; BDRate -0.588

Change-Id: I4c06ff51cc20194bffbd4d2346e57ba3dcf6b62c
2016-05-24 13:55:57 -07:00
hui su
4a741a5d5c Skip unnecessary calculations in ext-intra
Around 5% speedup.

Change-Id: I1c552e4e58fbf5637c0b5a97dd2cc4f83a1ca201
2016-05-23 17:24:19 -07:00
Debargha Mukherjee
fa5022978d Merge "Wedge refactoring to handle signs better" into nextgenv2 2016-05-20 23:19:39 +00:00
Debargha Mukherjee
e5de2ad632 Wedge refactoring to handle signs better
Mostly refactoring. Handles signs better though results are
more or less neutral.

Change-Id: If499537c8f8da4f34d104ebfda072eb4c85fb12f
2016-05-20 14:12:52 -07:00
Yaowu Xu
a9fc1cc257 Fix a build issue
When both obmc and dual_filter is enabled.

Change-Id: I56b127573a6cca31469bb357cf7a6a9c3df64071
2016-05-19 14:24:41 -07:00
Yue Chen
a33e3d12cb Merge "Fix obmc + ext-interp interference" into nextgenv2 2016-05-19 18:08:52 +00:00
Geza Lore
009bd1153e Fix obmc + ext-interp interference
With ext-interp, a switchable interpolation filter is coded iff the
motion vector uses fractional pixel movement (ie, true subpixel
movement). With ext-interp and obmc enabled at the same time, the RD
search proceeds as:
1. Do motion search
2. Do interpolation filter search iff subpixel motion, otherwise use
   EIGHTTAP_REGULAR
3. Evaluate obmc=0
4. Evaluete obmc=1 - This involves another motion search

If the motion search in step 4 yields an integer motion vector, while
the search in step 1 did not, then an interp_filter value other than
EIGHTTAP_REGULAR is invalid, and will cause an assertion failure
at output time, or a mismatch if not using --enable-debug.

The fix sets the interp_filter to EIGHTTAP_REGULAR if obmc=1 is picked
with an integer motion vector.

Change-Id: I4685d1ad537f41d833dc9eb64845956b67886cca
2016-05-19 11:30:07 +01:00
Debargha Mukherjee
f1ddf6eb04 Merge "Reducing computation of interintra modes" into nextgenv2 2016-05-18 17:21:15 +00:00
Debargha Mukherjee
049dbe7786 Reducing computation of interintra modes
Use model for interintra mode search.
Speed-up about 5-10% with about 0.04 drop in efficiency.

lowres: -2.60%

Change-Id: I825bf0ba8a46eb7f19fc528c25b8df066fb8ea95
2016-05-17 07:28:06 -07:00
James Zern
a81a75184c Merge "vp10/rdopt,rd_pick_intra4x4block: port tsan fix from vp9" into nextgenv2 2016-05-17 03:04:00 +00:00
James Zern
8eba4ac46e vp10/rdopt,rd_pick_intra4x4block: port tsan fix from vp9
minus the non-existent nonrd portion. original change:

commit d642294b1c
Author: Jingning Han <jingning@google.com>
Date:   Thu Feb 11 12:36:49 2016 -0800

    Fix tsan error in VP9 sub8x8 intra mode search

    This commit fixes issue 1141. The issue was triggered in multi-tile
    encoding. The change properly saves and restores the block context
    information in the real-time mode selection process. It removes
    several redundant memcpy operations in sub8x8 intra block mode
    search.

    Change-Id: I35c9ad197f4bd500ec39b5fc833f052f19eee010

Change-Id: Idfa38c54c9e645479f6870d46f71fb1e91c071da
2016-05-16 17:20:29 -07:00
Jingning Han
4677e1a718 Unify the per directional filter type system for compound modes
For the current stage, we assume a single prediction filter type
per direction in the settings of compound inter prediction modes.

Change-Id: I12a1afdd364b93fcee870bd11ad01fc40ab48cff
2016-05-16 14:41:08 -07:00
Jingning Han
d567e14e81 Enable per motion component filter type selection
Change-Id: I73823fc94f296d225dece7156de71b30bae3fcb7
2016-05-16 14:38:43 -07:00
Debargha Mukherjee
fb8ea1736b Various wedge enhancements
Increases number of wedges for smaller block and removes
wedge coding mode for blocks larger than 32x32.

Also adds various other enhancements for subsequent experimentation,
including adding provision for multiple smoothing functions
(though one is used currently), adds a speed feature that decides
the sign for interinter wedges using a fast mechanism, and refactors
wedge representations.

lowres: -2.651% BDRATE

Most of the gain is due to increase in codebook size for 8x8 - 16x16.

Change-Id: I50669f558c8d0d45e5a6f70aca4385a185b58b5b
2016-05-16 12:41:47 -07:00
Geza Lore
c1b739014f Cost wedge sign/index properly in rdopt.
Lowres improves by about 0.1%

lowres: -2.164 BDRATE

Change-Id: I393bbb92700bfbb8763ace424f4edc2d672a74b4
2016-05-11 11:59:10 -07:00
Yue Chen
372e12b959 Merge "Add single motion search for OBMC predictor" into nextgenv2 2016-05-11 17:20:32 +00:00
Yue Chen
370f203a40 Add single motion search for OBMC predictor
Weighted single motion search is implemented for obmc predictor.
When NEWMV mode is used, to determine the MV for the current block,
we run weighted motion search to compare the weighted prediction
with (source - weighted prediction using neighbors' MVs), in which
the distortion is the actual prediction error of obmc prediction.

Coding gain: 0.404/0.425/0.366 for lowres/midres/hdres
Speed impact: +14% encoding time
              (obmc w/o mv search 13%-> obmc w/ mv search 27%)

Change-Id: Id7ad3fc6ba295b23d9c53c8a16a4ac1677ad835c
2016-05-10 18:27:45 -07:00
Debargha Mukherjee
3fbe6e5e49 Merge "Wedge rd improvements" into nextgenv2 2016-05-10 20:34:00 +00:00
Debargha Mukherjee
447032eb32 Wedge rd improvements
Improves speed by about 10-15% by combining y-only rd with
modeling function in a better way.
Also, coding efficiency improves by about 0.1%

lowres: -1.805% BDRATE with ext-inter

Change-Id: I6ef1f8942ec6806252f3fcf749ae4f30dffe42b1
2016-05-10 11:47:48 -07:00