Commit Graph

374 Commits

Author SHA1 Message Date
Debargha Mukherjee
9a019bce84 Merge "cost_coeff speed improvements" into nextgenv2 2016-02-18 19:31:18 +00:00
Julia Robson
c6eba0b47a cost_coeff speed improvements
Preliminary tests indicated that these changes make cost_coeffs
approximately 20% faster which is a 2% improvement overall

Change-Id: Iaf013ba75884415cd824e98349f654ffb1c3ef33
2016-02-18 13:18:39 +00:00
Jingning Han
95247be0bf Fix enc/dec mismatch in dynamic mv referenceing experiment
This commit fixes an enc/dec mismatch in the dynamic motion vector
referencing experiment introduced in 837ef00.

Change-Id: I9fbe116fce118a80ef0f96bf41ce1f802547c2ee
2016-02-17 09:29:54 -08:00
Yue Chen
907f88c4e6 Fixing a bug in obmc prediction in the rd loop
This bug made the rd loop use one-side obmc (compound of the current
predictor and the predictors of the left mi's, while the above ones
are ignored by mistake) to determine whether to use obmc. This fix
improved the compression performance by ~0.6% on different test sets.

Coding gain (%) of obmc experiment on derflr/derfhd/hevcmr/hevchd:
1.568/TBD/1.628/TBD

Change-Id: I43b239bedf9a8eebfd02315b1b036e140a998140
2016-02-16 14:43:45 -08:00
Hui Su
0107373234 Merge "Add a speed feature to skip transform type selection" into nextgenv2 2016-02-16 18:31:18 +00:00
Debargha Mukherjee
6f49446dfa Merge "Fix double counting of compound reference bit cost." into nextgenv2 2016-02-16 17:55:49 +00:00
Geza Lore
abd00505d1 Add optimized vpx_sum_squares_2d_i16 for vp10.
Using this we can eliminate large numbers of calls to predict intra,
and is also faster than most of the variance functions it replaces.
This is an equivalence transform so coding performance is unaffected.

Encoder speedup is approx 7% when var_tx, super_tx and ext_tx are all
enabled.

Change-Id: I0d4c83afc4a97a1826f3abd864bd68e41bb504fb
2016-02-15 16:54:52 +00:00
Yue Chen
d1cad9c3f5 Overlapped block motion compensation experiment
In this experiment, an obmc inter prediction mode is enabled for
>= 8X8 inter blocks. When the obmc flag is on, the regular block-
based motion compensation will be refined by using predictors of
the above and left blocks.
Fixed some compatibility issues with vp9_highbitdepth, supertx,
ref_mv, and ext_interp.

Coding gain (%) on derflr/hevcmr/hevchd
OBMC:
1.047/1.022/0.708
OBMC + SUPERTX:
1.652/1.616/1.137
SUPERTX:
0.862/0.779/0.630

Change-Id: I5d8d3c4729c6d3ccb03ec7034563107893103b7f
2016-02-12 13:36:25 -08:00
Alex Converse
a45d5d3f94 Merge "Port switch to 9-bit rate cost to vp10." into nextgenv2 2016-02-12 21:15:35 +00:00
Jingning Han
18eaf8e6fc Merge "Refactor vp10_drl_idx concept" into nextgenv2 2016-02-12 19:39:44 +00:00
Jingning Han
a39e83d743 Refactor vp10_drl_idx concept
Remove the implicit assumption on offsetting the index by 1.

Change-Id: I6f1d391e067d57b7e45b9287e866014dbc16da71
2016-02-11 16:38:13 -08:00
Debargha Mukherjee
c1924b9ff0 Merge "Complete high bitdepth VAR_TX implementation." into nextgenv2 2016-02-12 00:16:18 +00:00
Angie Chiang
368e3d9293 Merge "Refactor: add predict_interp_filter() to simplify the flow in handle_inter_mode" into nextgenv2 2016-02-12 00:16:13 +00:00
Jingning Han
57c83b330e Remove redundant parameters from vp10_txfm_rd_in_plane_supertx()
Change-Id: Icb164403239f88f18fd64de75d4881d33d3ab1cc
2016-02-11 11:53:22 -08:00
Jingning Han
5c772f38fa Format clean-ups in transform experiments
Change-Id: Ib2843cb03ae452ce9fec3a94c709431ea0202d8b
2016-02-11 11:07:00 -08:00
Alex Converse
b3ad81288f Port switch to 9-bit rate cost to vp10.
Brings the following commits to vp10:
269428e Tie the bit cost scale to a define.
d13385c Switch to 9-bit rate cost constants built on a 256 probability denominator.
ad43a73 Fix a signed overflow in vp9 motion cost.
1c9b091 Fix some interger overflow errors
fac947d Restore previous motion search bit-error scale.

Change-Id: I598ba7ee7efcde18439c31dfa96b86cbf297a580
2016-02-11 09:54:24 -08:00
Geza Lore
432e875dce Complete high bitdepth VAR_TX implementation.
VAR_TX now works in the high bitdepth configuration.

Change-Id: I4114d7d9ed59c598f1e4d35b8e75876c07074ba7
2016-02-11 10:49:56 +00:00
Hui Su
6779be2487 Merge "Refactor rd_pick_intra_angle_" into nextgenv2 2016-02-11 01:44:14 +00:00
hui su
5a7c8d8c1d Refactor rd_pick_intra_angle_
Change-Id: I6c78188bdedb52655678c63f6a767567b256a880
2016-02-10 15:41:04 -08:00
Angie Chiang
c0035cc480 Refactor: add predict_interp_filter() to
simplify the flow in handle_inter_mode

Change-Id: Ic7934c0a5d0a79bdf546b4d2d106035449b475a6
2016-02-10 15:32:10 -08:00
hui su
329e340dc5 Add a speed feature to skip transform type selection
Setting FIXED_TX_TYPE as 1 makes the encoder skip tx_type search,
about twice as fast.

This speed feature is off by defualt; we can turn it on when we
want to quickly test new ideas.

Change-Id: Ieab5807d17fcd54fce3e8ae2f59a18b42eb79408
2016-02-10 15:11:01 -08:00
Jingning Han
3c8bd0d3de Merge "Resolve conflict between var-tx and super-tx" into nextgenv2 2016-02-10 18:54:21 +00:00
Jingning Han
4c6c82a2e8 Resolve conflict between var-tx and super-tx
This commit aligns the rate-distortion metric for the recursive
transform block partitioning and the super transform. It resolves
the conflicts between these two experiments. The coding performance
gains of the combined experiments (var-tx + super-tx) has been
improved:

derf   0.89%  ->  1.9%
hevcmr 1.06%  ->  1.8%
stdhd  0.29%  ->  1.4%
hevchr 0.80%  ->  2.3%

Change-Id: I7e33994ad70c1b2751435620815f867d82172f41
2016-02-10 09:36:40 -08:00
Jingning Han
260da6ca8d Merge "Entropy coding for dynamic ref mv modes" into nextgenv2 2016-02-10 16:49:46 +00:00
Geza Lore
fd5463571e Fix double counting of compound reference bit cost.
These costs are added in separately just before the computed
ref_costs_* are added in the calling functions, so they were
effectively double counted.

Change-Id: Ic941d0243460cc2e750791cfc508e97d8b90e8fd
2016-02-10 13:37:31 +00:00
Debargha Mukherjee
e2c1ea9422 A variety of fixes for supertx/var-tx rd costing
Change-Id: I8a3d59378abb1dfa4e614b2975c2db05d4224bd5
2016-02-08 21:46:08 -08:00
Jingning Han
4958987b2a Entropy coding for dynamic ref mv modes
This commit enables entropy coding for dynamic reference motion
vector modes. The probability model is contexted on the ranking
categories of the reference motion vector candidates.

Change-Id: I09b58d98a409d63ec1a407331e29f8945b7ef17d
2016-02-08 17:05:24 -08:00
Debargha Mukherjee
d46c1f2349 Explicitly set tx_type for sub8x8 blocks
Fixes an issue where the tx_type was not set correctly for
sub8x8 inter and intra blocks. In the current syntax, for
sub8x8 blocks, there is still a single tx_type that is
transmitted. Ideally, this should be searched for the best
rd performance, albeit at the expense of encode speed.
For now, we just set it to DCT_DCT. Previously it was left
incorrectly as what was used for the previous non sub8x8
block.

derflr: BDRATE -0.277%

Change-Id: If76ba903bfbfd4d374cf1ac7d1daee50e92f0edd
2016-02-08 16:22:25 -08:00
Jingning Han
afd73539bb Merge "Enable dynamic ref motion vector mode for compound inter block" into nextgenv2 2016-02-09 00:04:36 +00:00
Yaowu Xu
78bca69138 Merge "Fix msvc compiler warnings" into nextgenv2 2016-02-08 23:38:48 +00:00
Jingning Han
28e0393f23 Enable dynamic ref motion vector mode for compound inter block
This commit enables the dynamic reference motion vector coding mode
for the compound inter blocks.

Change-Id: Ibe78fd8de6989db392cd67a9d81a69d680345ba1
2016-02-08 11:24:15 -08:00
Yaowu Xu
3c28b4a8ff Fix msvc compiler warnings
There were a number of compiler warnings:
1. int16_t to uint8_t in recon_intra.c;
2. double to float conversions in psnrhvs.c
3. intptr_t to int in quantize.c
4. size_t to int32_t in decoder.c

Change-Id: Id95423b17779dcfa6cf39d9a90fe8cb8b910f5df
2016-02-08 10:14:08 -08:00
Hui Su
e5bd08185a Merge "Add a speed feature for intra filter search" into nextgenv2 2016-02-05 18:49:24 +00:00
Hui Su
9604f69005 Merge "Add 8-tap interpolation filter options for intra prediction" into nextgenv2 2016-02-04 23:06:28 +00:00
hui su
5b618b7cae Add a speed feature for intra filter search
Seperate the prediction angle search and fitler search.
It can reduce the computation overhead of filter search by as much
as 85%, while keeping more than 50% of the coding gain.

Change-Id: Id152f71e20ebcaca8b429bdd4ca1fbeb646fc6bf
2016-02-03 15:12:06 -08:00
hui su
3b1c766802 Add 8-tap interpolation filter options for intra prediction
BD-rate performance improvement (on top of ext-intra):
derflr  0.22%
hevclr  0.36%
hevcmr  0.48%
hevchr  0.37%
stdhd   0.19%

Average speed impact on some derf clips is about 40% slower (on
top of ext-intra). Speed improvment is a to-do.

Change-Id: I8fe3fe8c5e4f60d0462778adbcc15c84dfbe7a25
2016-02-03 14:19:20 -08:00
Jingning Han
4fb8b217e6 Generalize the dynamic reference motion vector coding mode
This commit generalizes dynamic reference motion vector coding mode
to support multiple candidate modes in the rate-distortion
optimization scheme and to support the selection in the bit-stream
syntax. The maximum number of modes allowed is currently limited to
4. The syntax elements for the dynamic reference motion vector
modes are using binary codes. The scheme supports single reference
frame.

It improves the compression performance
derf   0.135%
hevcmr 0.098%

Change-Id: Id053d6ce76e8365e52727bd0d12d28ce3de2e0e8
2016-02-03 11:57:42 -08:00
Jingning Han
590265eaf1 Account for zero-forcing operation in selective ref mv mode
It makes the encoder accounts for the block zero-forcing operation
when optimizing the mode decisions.

Change-Id: I2c8e243756080b446b8a53a9679f75c4c47148cf
2016-02-03 09:26:35 -08:00
Jingning Han
67cf8908bc Enable adaptive motion vector referencing mode
This commit allows an adaptive motion vector referencing mode
approach. It checks the available reference motion vector candidate
list and decides the amount of motion vector referencing modes. The
current implementation assumes simple binary coding for the syntax.

The compression performance is improved by
derf   0.11%
hevcmr 0.38%
stdhd  0.09%
hevchr 0.23%

The coding gains due to the new reference motion vector system are
derf   1.0%
hevcmr 1.7%
stdhd  1.4%
hevchr 1.3%

Change-Id: Idf932fc373546fe59c8741f1b933ff656e8dbc3f
2016-02-02 15:15:40 -08:00
Angie Chiang
10ad97bc55 Pass filter type instead of filter array
Change-Id: I25f2149ddaa332722f7ab82e8f832a253c4b6ab3
2016-02-01 17:03:50 -08:00
Debargha Mukherjee
eef57c1e99 Fixes ext-interp experiment
Fixes integer pel MV usage for the sub8x8 case, which fixes a
rare mismatch issue.

Also adds some other minor missing code related to filter threshes.

Change-Id: I6b07e6cf9b287ba4b5bd6599af4a7412e50b3bdc
2016-01-27 09:24:48 -08:00
Geza Lore
5aa655f79a Reintroduce VAR_TX fix
Reintroduce part of
Iaf2b717e6b8626b2b6a03226127221b776b49884
Which was later reverted in
I4c5b40ec63a6f19521191d3c730af87db3c4bc00

Change-Id: If3e5610ba3985ae7b4d952d8e616982465ac667a
2016-01-26 15:27:47 +00:00
Debargha Mukherjee
9a8a6a1b35 Some supertx fixes
Fixes some of the issues introduced by a merge from master.

derflr: -0.893% BDRATE
hevcmr: -1.667% BDRATE

Change-Id: I4c5b40ec63a6f19521191d3c730af87db3c4bc00
2016-01-25 10:36:41 -08:00
Yue Chen
968bbc7bb2 Adding new compound modes to EXT_INTER experiment
Combinations of different mv modes for two reference frames
are allowed in compound inter modes. 9 options are enabled,
including NEAREST_NEARESTMV, NEAREST_NEARMV, NEAR_NEARESTMV,
NEAREST_NEWMV, NEW_NEARESTMV, NEAR_NEWMV, NEW_NEARMV, ZERO_ZEROMV,
and NEW_NEWMV.
This experiment is mostly deported from the nextgen branch.
It is made compatible with other experiments

Coding gain of EXT_INTER(derflr/hevcmr/hevchd): 0.533%/0.728%/0.639%

Change-Id: Id47e97284e6481b186870afbad33204b7a33dbb0
2016-01-22 13:52:16 -08:00
Debargha Mukherjee
11d3899c60 Merge "A fix for a missing tx_type" into nextgenv2 2016-01-22 18:36:46 +00:00
Debargha Mukherjee
4e406f702c A fix for a missing tx_type
Change-Id: I165cd06256175edb7739020379ba3098251f4a7c
2016-01-22 03:29:39 -08:00
Debargha Mukherjee
ade60d4fe9 Merge "Fixes for var_tx when ext_tx is not enabled" into nextgenv2 2016-01-22 06:09:36 +00:00
Julia Robson
9fe188e4a5 Fixes for var_tx when ext_tx is not enabled
This patch fixes a couple of issues caused by change-id:
I15d20ce5292b70f0c2b4ba55c1f1318181481596
Changes to the code for when the ext_tx experiment is not enabled
were merged from master but as var_tx does not exist on master
the changes had not been applied to the case when var_tx experiment
is enabled

Change-Id: Iaf2b717e6b8626b2b6a03226127221b776b49884
2016-01-21 17:46:03 -08:00
Debargha Mukherjee
8d69a6e816 Some fixes on tx size/type selection
For ext_tx experiment.

Change-Id: Ie37b9b456b09bde8b606fb978fee4cca8d0326b7
2016-01-21 17:45:30 -08:00
Yue Chen
1ac858794a EXT_INTER experiment
NEW2MV is enabled, representing a new motion vector predicted from
NEARMV. It is mostly ported from nextgen, where it was named
NEW_INTER.
A few fixes are done for sub8x8 RDO to correct some misused
mv references in the original patch.
A 'bug-fix' for encoding complexity is done, reducing the additional
encoding time from 50% to 20%. In sub8x8 case, the old patch
did motion search for every interpolation filter (vp9 only
searches once). This fix also slightly improves the coding gain.
This experiment has been made compatible with REF_MV and EXT_REFS.

Coding gain (derflr/hevcmr/hevchd): 0.267%/0.542%/0.257%

Change-Id: I9a94c5f292e7454492a877f65072e8aedba087d4
2016-01-15 14:47:02 -08:00