454 Commits

Author SHA1 Message Date
hui su
329e340dc5 Add a speed feature to skip transform type selection
Setting FIXED_TX_TYPE as 1 makes the encoder skip tx_type search,
about twice as fast.

This speed feature is off by defualt; we can turn it on when we
want to quickly test new ideas.

Change-Id: Ieab5807d17fcd54fce3e8ae2f59a18b42eb79408
2016-02-10 15:11:01 -08:00
Jingning Han
3c8bd0d3de Merge "Resolve conflict between var-tx and super-tx" into nextgenv2 2016-02-10 18:54:21 +00:00
Jingning Han
4c6c82a2e8 Resolve conflict between var-tx and super-tx
This commit aligns the rate-distortion metric for the recursive
transform block partitioning and the super transform. It resolves
the conflicts between these two experiments. The coding performance
gains of the combined experiments (var-tx + super-tx) has been
improved:

derf   0.89%  ->  1.9%
hevcmr 1.06%  ->  1.8%
stdhd  0.29%  ->  1.4%
hevchr 0.80%  ->  2.3%

Change-Id: I7e33994ad70c1b2751435620815f867d82172f41
2016-02-10 09:36:40 -08:00
Jingning Han
260da6ca8d Merge "Entropy coding for dynamic ref mv modes" into nextgenv2 2016-02-10 16:49:46 +00:00
Geza Lore
fd5463571e Fix double counting of compound reference bit cost.
These costs are added in separately just before the computed
ref_costs_* are added in the calling functions, so they were
effectively double counted.

Change-Id: Ic941d0243460cc2e750791cfc508e97d8b90e8fd
2016-02-10 13:37:31 +00:00
Debargha Mukherjee
e2c1ea9422 A variety of fixes for supertx/var-tx rd costing
Change-Id: I8a3d59378abb1dfa4e614b2975c2db05d4224bd5
2016-02-08 21:46:08 -08:00
Jingning Han
4958987b2a Entropy coding for dynamic ref mv modes
This commit enables entropy coding for dynamic reference motion
vector modes. The probability model is contexted on the ranking
categories of the reference motion vector candidates.

Change-Id: I09b58d98a409d63ec1a407331e29f8945b7ef17d
2016-02-08 17:05:24 -08:00
Debargha Mukherjee
d46c1f2349 Explicitly set tx_type for sub8x8 blocks
Fixes an issue where the tx_type was not set correctly for
sub8x8 inter and intra blocks. In the current syntax, for
sub8x8 blocks, there is still a single tx_type that is
transmitted. Ideally, this should be searched for the best
rd performance, albeit at the expense of encode speed.
For now, we just set it to DCT_DCT. Previously it was left
incorrectly as what was used for the previous non sub8x8
block.

derflr: BDRATE -0.277%

Change-Id: If76ba903bfbfd4d374cf1ac7d1daee50e92f0edd
2016-02-08 16:22:25 -08:00
Jingning Han
afd73539bb Merge "Enable dynamic ref motion vector mode for compound inter block" into nextgenv2 2016-02-09 00:04:36 +00:00
Yaowu Xu
78bca69138 Merge "Fix msvc compiler warnings" into nextgenv2 2016-02-08 23:38:48 +00:00
Jingning Han
28e0393f23 Enable dynamic ref motion vector mode for compound inter block
This commit enables the dynamic reference motion vector coding mode
for the compound inter blocks.

Change-Id: Ibe78fd8de6989db392cd67a9d81a69d680345ba1
2016-02-08 11:24:15 -08:00
Yaowu Xu
3c28b4a8ff Fix msvc compiler warnings
There were a number of compiler warnings:
1. int16_t to uint8_t in recon_intra.c;
2. double to float conversions in psnrhvs.c
3. intptr_t to int in quantize.c
4. size_t to int32_t in decoder.c

Change-Id: Id95423b17779dcfa6cf39d9a90fe8cb8b910f5df
2016-02-08 10:14:08 -08:00
Hui Su
e5bd08185a Merge "Add a speed feature for intra filter search" into nextgenv2 2016-02-05 18:49:24 +00:00
Hui Su
9604f69005 Merge "Add 8-tap interpolation filter options for intra prediction" into nextgenv2 2016-02-04 23:06:28 +00:00
hui su
5b618b7cae Add a speed feature for intra filter search
Seperate the prediction angle search and fitler search.
It can reduce the computation overhead of filter search by as much
as 85%, while keeping more than 50% of the coding gain.

Change-Id: Id152f71e20ebcaca8b429bdd4ca1fbeb646fc6bf
2016-02-03 15:12:06 -08:00
hui su
3b1c766802 Add 8-tap interpolation filter options for intra prediction
BD-rate performance improvement (on top of ext-intra):
derflr  0.22%
hevclr  0.36%
hevcmr  0.48%
hevchr  0.37%
stdhd   0.19%

Average speed impact on some derf clips is about 40% slower (on
top of ext-intra). Speed improvment is a to-do.

Change-Id: I8fe3fe8c5e4f60d0462778adbcc15c84dfbe7a25
2016-02-03 14:19:20 -08:00
Jingning Han
4fb8b217e6 Generalize the dynamic reference motion vector coding mode
This commit generalizes dynamic reference motion vector coding mode
to support multiple candidate modes in the rate-distortion
optimization scheme and to support the selection in the bit-stream
syntax. The maximum number of modes allowed is currently limited to
4. The syntax elements for the dynamic reference motion vector
modes are using binary codes. The scheme supports single reference
frame.

It improves the compression performance
derf   0.135%
hevcmr 0.098%

Change-Id: Id053d6ce76e8365e52727bd0d12d28ce3de2e0e8
2016-02-03 11:57:42 -08:00
Jingning Han
590265eaf1 Account for zero-forcing operation in selective ref mv mode
It makes the encoder accounts for the block zero-forcing operation
when optimizing the mode decisions.

Change-Id: I2c8e243756080b446b8a53a9679f75c4c47148cf
2016-02-03 09:26:35 -08:00
Jingning Han
67cf8908bc Enable adaptive motion vector referencing mode
This commit allows an adaptive motion vector referencing mode
approach. It checks the available reference motion vector candidate
list and decides the amount of motion vector referencing modes. The
current implementation assumes simple binary coding for the syntax.

The compression performance is improved by
derf   0.11%
hevcmr 0.38%
stdhd  0.09%
hevchr 0.23%

The coding gains due to the new reference motion vector system are
derf   1.0%
hevcmr 1.7%
stdhd  1.4%
hevchr 1.3%

Change-Id: Idf932fc373546fe59c8741f1b933ff656e8dbc3f
2016-02-02 15:15:40 -08:00
Angie Chiang
10ad97bc55 Pass filter type instead of filter array
Change-Id: I25f2149ddaa332722f7ab82e8f832a253c4b6ab3
2016-02-01 17:03:50 -08:00
Debargha Mukherjee
eef57c1e99 Fixes ext-interp experiment
Fixes integer pel MV usage for the sub8x8 case, which fixes a
rare mismatch issue.

Also adds some other minor missing code related to filter threshes.

Change-Id: I6b07e6cf9b287ba4b5bd6599af4a7412e50b3bdc
2016-01-27 09:24:48 -08:00
Geza Lore
5aa655f79a Reintroduce VAR_TX fix
Reintroduce part of
Iaf2b717e6b8626b2b6a03226127221b776b49884
Which was later reverted in
I4c5b40ec63a6f19521191d3c730af87db3c4bc00

Change-Id: If3e5610ba3985ae7b4d952d8e616982465ac667a
2016-01-26 15:27:47 +00:00
Debargha Mukherjee
9a8a6a1b35 Some supertx fixes
Fixes some of the issues introduced by a merge from master.

derflr: -0.893% BDRATE
hevcmr: -1.667% BDRATE

Change-Id: I4c5b40ec63a6f19521191d3c730af87db3c4bc00
2016-01-25 10:36:41 -08:00
Yue Chen
968bbc7bb2 Adding new compound modes to EXT_INTER experiment
Combinations of different mv modes for two reference frames
are allowed in compound inter modes. 9 options are enabled,
including NEAREST_NEARESTMV, NEAREST_NEARMV, NEAR_NEARESTMV,
NEAREST_NEWMV, NEW_NEARESTMV, NEAR_NEWMV, NEW_NEARMV, ZERO_ZEROMV,
and NEW_NEWMV.
This experiment is mostly deported from the nextgen branch.
It is made compatible with other experiments

Coding gain of EXT_INTER(derflr/hevcmr/hevchd): 0.533%/0.728%/0.639%

Change-Id: Id47e97284e6481b186870afbad33204b7a33dbb0
2016-01-22 13:52:16 -08:00
Debargha Mukherjee
11d3899c60 Merge "A fix for a missing tx_type" into nextgenv2 2016-01-22 18:36:46 +00:00
Debargha Mukherjee
4e406f702c A fix for a missing tx_type
Change-Id: I165cd06256175edb7739020379ba3098251f4a7c
2016-01-22 03:29:39 -08:00
Debargha Mukherjee
ade60d4fe9 Merge "Fixes for var_tx when ext_tx is not enabled" into nextgenv2 2016-01-22 06:09:36 +00:00
Julia Robson
9fe188e4a5 Fixes for var_tx when ext_tx is not enabled
This patch fixes a couple of issues caused by change-id:
I15d20ce5292b70f0c2b4ba55c1f1318181481596
Changes to the code for when the ext_tx experiment is not enabled
were merged from master but as var_tx does not exist on master
the changes had not been applied to the case when var_tx experiment
is enabled

Change-Id: Iaf2b717e6b8626b2b6a03226127221b776b49884
2016-01-21 17:46:03 -08:00
Debargha Mukherjee
8d69a6e816 Some fixes on tx size/type selection
For ext_tx experiment.

Change-Id: Ie37b9b456b09bde8b606fb978fee4cca8d0326b7
2016-01-21 17:45:30 -08:00
Yue Chen
1ac858794a EXT_INTER experiment
NEW2MV is enabled, representing a new motion vector predicted from
NEARMV. It is mostly ported from nextgen, where it was named
NEW_INTER.
A few fixes are done for sub8x8 RDO to correct some misused
mv references in the original patch.
A 'bug-fix' for encoding complexity is done, reducing the additional
encoding time from 50% to 20%. In sub8x8 case, the old patch
did motion search for every interpolation filter (vp9 only
searches once). This fix also slightly improves the coding gain.
This experiment has been made compatible with REF_MV and EXT_REFS.

Coding gain (derflr/hevcmr/hevchd): 0.267%/0.542%/0.257%

Change-Id: I9a94c5f292e7454492a877f65072e8aedba087d4
2016-01-15 14:47:02 -08:00
Jingning Han
c38fc52df4 Merge "Handle single ref mv pair in the candidate list for compound mode" into nextgenv2 2016-01-14 02:55:36 +00:00
Yaowu Xu
727ca802bf Merge "Merge branch 'master' into nextgenv2" into nextgenv2 2016-01-14 00:26:45 +00:00
Jingning Han
3944cfb14d Handle single ref mv pair in the candidate list for compound mode
This commit considers the case where a single reference motion
vector pair is found in the candidate list. It treats this pair
as the effective motion vector for nearestmv mode. This improves
the coding performance by 0.06% for stdhd sets.

Change-Id: I9ce12f456b52912933e05c18c3841a78c26155d2
2016-01-13 16:19:27 -08:00
Yaowu Xu
0367f32ea8 Merge branch 'master' into nextgenv2
Manually resovled the following conflicts:
	vp10/common/blockd.h
	vp10/common/entropy.h
	vp10/common/entropymode.c
	vp10/common/entropymode.h
	vp10/common/enums.h
	vp10/common/thread_common.c
	vp10/decoder/decodeframe.c
	vp10/decoder/decodemv.c
	vp10/encoder/bitstream.c
	vp10/encoder/encodeframe.c
	vp10/encoder/rd.c
	vp10/encoder/rdopt.c

Change-Id: I15d20ce5292b70f0c2b4ba55c1f1318181481596
2016-01-13 13:18:06 -08:00
Jingning Han
33cc1bd21d Generate compound reference motion vector
This commit allows the codec to add motion vector pairs into
the candidate list. It further improves the compression performance
by 0.1% across derf, hevcmr, stdhd, and hevchr sets without adding
encode/decode time.

Change-Id: I88d36da25a2a89bb506d411844af667081eba98b
2016-01-12 15:28:47 -08:00
Debargha Mukherjee
a0900fd0db Remove experimental flag for ext_tx
Also includes a bug fix.

Change-Id: Ia49ed00f8ffd1531c10bcf89b1f497310ee7cb82
2016-01-08 13:48:24 -08:00
Debargha Mukherjee
f7dfa4ece7 Modifies inter/intra coding to allow all tx types
The nominal tx_type for a given mode is used as a context
to encode the actual tx_type for intra.

Results:
derflr: -0.241% BDRATE
hevcmr: -0.366% BDRATE

Change-Id: Icfe7b0a58d79bc6497a06e3441779afec6e01e21
2016-01-08 11:13:46 -08:00
Jingning Han
387a10e3dc Enable context analyzer for inter mode entropy coding
It allows the codec to account for certain corner cases when
processing inter prediction mode entropy coding.

Change-Id: Ied451f4fff26ba579f6556554b8381ff2ccd0003
2016-01-08 10:27:27 -08:00
Peter de Rivaz
2f943131df Change to rd costing for CONFIG_VAR_TX
In select_tx_block I believe the rd cost was ignoring the bits
needed to encode the split bit.

Change-Id: Iacbf705b880db9a68967a994406ba90ecf693ab8
2016-01-05 15:35:43 +00:00
Debargha Mukherjee
3787b17439 Super transform - ported from nextgen branch
Various additional changes were made to make the experiment
compatible with misc_fixes.

derflr: +0.979%
hevcmr: +0.865%

Speed-wise with --enable-supertx the encoder is only about 10%
slower than without. Decoding impact is about 30% slowdown.

Note this does not work with ext-tx or var-tx yet. That is
a TODO.

Change-Id: If25af4241a7a9efbd28f58eda3c4f044c7a7ef4b
2016-01-04 22:12:57 -08:00
hui su
1559afda85 Use precise rate cost for intra modes in inter frames
derflr +0.021%
hevclr +0.207%
hevcmr +0.035%
stdhd  +0.042%

Change-Id: Ic750df93bcc0a261a66a9b19d939a5cd61a6b516
2016-01-04 08:35:29 -08:00
Zoe Liu
ec36a2b061 Restore the flexibility for the new 3 references
For the experiment of EXT_REFS, removed the previous special handling
on the new last 3 references, i.e. LAST2_FRAME, LAST3_FRAME, and
LAST4_FRAME, at the decoder, so that these new last references are
treated the same way as the other 3 references (LAST_FRAME,
GOLDEN_FRAME, and ALTREF_FRAME). Encoder changes have been made
accordingly to realize this flexibility.

Change-Id: Ic6546f9443b4377bb7e7b101bfa3e70a8b8d1c65
2015-12-17 16:34:02 -08:00
Yaowu Xu
dab7515aa4 Merge branch 'master' into nextgenv2
With a few manual fixes of merge conflicts.

Change-Id: I0dd65ff90f9fa8606e5563f528659e2607b12376
2015-12-16 09:00:57 -08:00
Yaowu Xu
9232f69b26 Merge "Fix a enc/dec mismatch under CONFIG_MISC_FIXES" 2015-12-15 16:02:39 +00:00
Yaowu Xu
c7101830a6 Fix a enc/dec mismatch under CONFIG_MISC_FIXES
The culprit is on the decode side xd->lossless[i] setup was in wrong
location where segment features are not yet decoded.

Also on the encoder side, transform mode was not set consistently
between when tx_mode is selected and how tx_mode is enforced in
tx size selection.

Change-Id: I4c4c32188fda7530cadab9b46d4201f33f7ceca3
2015-12-14 20:56:37 -08:00
Jingning Han
6f1f0d896a Merge "Enable adaptive prediction mode coding" into nextgenv2 2015-12-15 04:38:15 +00:00
Angie Chiang
30ee689da3 Merge "Refactor vp10_xform_quant" into nextgenv2 2015-12-11 20:29:04 +00:00
Yaowu Xu
f07d73b9bf Merge branch 'master' into nextgenv2
Change-Id: Id0b784b115602e2502b42fa972a5ae210435a3be
2015-12-11 08:58:40 -08:00
Jingning Han
aa5d53eb17 Enable adaptive prediction mode coding
This commit allows the codec to analyze the reference motion vector
candidate list and adaptively reduce the size of inter prediction
mode set.

Change-Id: Ied6a403843b860d66f26ed485c1825c05c71bdfc
2015-12-10 09:02:32 -08:00
Jingning Han
1dc18077b8 Re-design motion compensated prediction mode entropy coding system
This commit re-works the entropy coding scheme of the motion
compensated prediction modes. It allows more flexible hyperplane
partition for precise classification.

Change-Id: Iba5035c76691946cf1386b6c495e399c3d9c8fc5
2015-12-09 18:02:20 -08:00