Commit Graph

117 Commits

Author SHA1 Message Date
Jingning Han
d1d11fc6dd Unify frame border extension operation
This commit unifies the encoder and decoder border extension and
motion compensated prediction process. Remove the decoder specific
flow to simplify the development flow.

Change-Id: I9c43bbe6d7c017e6da2db6a62c5bf3d0af7ccfce
2016-02-26 13:58:53 -08:00
Yaowu Xu
aa6c754635 Merge remote-tracking branch 'webm/master' into nextgenv2 2016-02-24 10:53:17 -08:00
hui su
875aa1c58c Fix palette mode
It was broken by commit 3787b17439

Change-Id: I2be0f6243e8716b9ca4c0321de381419108d1abe
2016-02-23 12:01:23 -08:00
Angie Chiang
a90f8b8c8f Merge "sync dec/enc mv clamp behavior in prediction" into nextgenv2 2016-02-22 23:30:10 +00:00
Angie Chiang
e9336e4dfc sync dec/enc mv clamp behavior in prediction
Change-Id: I12ce1da18b3db7bd2f36e0424a264b3c36fbed61
2016-02-22 11:36:03 -08:00
Jingning Han
df59bb8986 Vectorize motion vector probability models
This commit converts the scalar motion vector probability model
into vector format for later precise estimate.

Change-Id: I7008d047ecc1b9577aa8442b4db2df312be869dc
2016-02-19 16:20:41 -08:00
James Zern
429d61252d vp10/decoder,resize_mv_buffer: add missing alloc check
Change-Id: I8eef989847fcf3c4b9a16b02db477e1007625145
2016-02-17 14:36:05 -08:00
Yue Chen
d1cad9c3f5 Overlapped block motion compensation experiment
In this experiment, an obmc inter prediction mode is enabled for
>= 8X8 inter blocks. When the obmc flag is on, the regular block-
based motion compensation will be refined by using predictors of
the above and left blocks.
Fixed some compatibility issues with vp9_highbitdepth, supertx,
ref_mv, and ext_interp.

Coding gain (%) on derflr/hevcmr/hevchd
OBMC:
1.047/1.022/0.708
OBMC + SUPERTX:
1.652/1.616/1.137
SUPERTX:
0.862/0.779/0.630

Change-Id: I5d8d3c4729c6d3ccb03ec7034563107893103b7f
2016-02-12 13:36:25 -08:00
Debargha Mukherjee
8b0a5b8718 Adding loop wiener restoration
Adds a wiener filter based restoration scheme in loop which can
be optionally selected instead of the bilateral filter.

The LMMSE filter generated per frame is a separable symmetric 7
tap filter. Three parameters for each of horizontal and vertical
filters are transmitted in the bitstream. The fourth parameter
is obtained assuming the sum is normalized to 1.
Also integerizes the bilateral filters, along with other
refactoring necessary in order to support the new switchable
restoration type framework.

derflr: -0.75% BDRATE

[A lot of videos still prefer bilateral, however since many frames
now use the simpler separable filter, the decoding speed is
much better].

Further experiments to follow, related to replacing the bilateral.

Change-Id: I6b1879983d50aab7ec5647340b6aef6b22299636
2016-02-12 09:56:24 -08:00
Jingning Han
4958987b2a Entropy coding for dynamic ref mv modes
This commit enables entropy coding for dynamic reference motion
vector modes. The probability model is contexted on the ranking
categories of the reference motion vector candidates.

Change-Id: I09b58d98a409d63ec1a407331e29f8945b7ef17d
2016-02-08 17:05:24 -08:00
Angie Chiang
eb71ef9235 Merge "add convolution function with adjustable length" into nextgenv2 2016-02-08 18:32:38 +00:00
Angie Chiang
d5349112e8 add convolution function with adjustable length
Change-Id: I1a5b1e15a188ef11594d0c6ac0dbd42aac59cfca
2016-02-05 17:33:19 -08:00
Yaowu Xu
105da4128d Fix bad merge artifacts
Temporaly disable warning for unused function for vp10, needs clean
out the warnings before re-enable the flag for vp10.

Change-Id: I5636f8cd607423f6ea6963db9c2cbd688e30b495
2016-02-05 09:04:41 -08:00
Hui Su
9604f69005 Merge "Add 8-tap interpolation filter options for intra prediction" into nextgenv2 2016-02-04 23:06:28 +00:00
Yaowu Xu
027cc8c873 merge per-segment lossless feature in misc_fixes
Change-Id: I56d56781d371c99aa5cdd2db1cbc0a17437723e9
2016-02-04 08:31:16 -08:00
hui su
3b1c766802 Add 8-tap interpolation filter options for intra prediction
BD-rate performance improvement (on top of ext-intra):
derflr  0.22%
hevclr  0.36%
hevcmr  0.48%
hevchr  0.37%
stdhd   0.19%

Average speed impact on some derf clips is about 40% slower (on
top of ext-intra). Speed improvment is a to-do.

Change-Id: I8fe3fe8c5e4f60d0462778adbcc15c84dfbe7a25
2016-02-03 14:19:20 -08:00
Angie Chiang
392d577c49 Merge "Pass filter type instead of filter array" into nextgenv2 2016-02-02 02:32:16 +00:00
Angie Chiang
10ad97bc55 Pass filter type instead of filter array
Change-Id: I25f2149ddaa332722f7ab82e8f832a253c4b6ab3
2016-02-01 17:03:50 -08:00
Debargha Mukherjee
f0a4485e54 Refactor to separate restoration from loop filter
Change-Id: Iab517862d957f3aa2a664e9349d57bbf424febb3
2016-01-29 15:39:23 -08:00
Debargha Mukherjee
3eb10fcf21 Cosmetic changes to loop restoration
Also adds a normalized filtering function to be used later.

Change-Id: I30e2140e664db635602f26a73b81ce8e008dff5e
2016-01-27 17:33:36 -08:00
Geza Lore
e7c0e157d2 Set inter_tx_size for supertx coded blocks.
The loop filter relies on inter_tx_size in MB_MODE_INFO being set
properly when VAR_TX is enabled. Supertx coded blocks did not set this
previously at all, and the differing garbage values eventually resulted
in in a YUV mismatch between encoder and decoder after loop filtering.

This patch fixes this by setting inter_tx_size to the proper supertx
size in both the encoder and the decoder. This should also mean that
loop filtering is done at the proper transform boundaries, even when
supertx or vartx is being used.

Change-Id: I41a564cd6d34ce4a8313ad4efa89d905f5ead731
2016-01-26 01:18:56 +00:00
Debargha Mukherjee
9a8a6a1b35 Some supertx fixes
Fixes some of the issues introduced by a merge from master.

derflr: -0.893% BDRATE
hevcmr: -1.667% BDRATE

Change-Id: I4c5b40ec63a6f19521191d3c730af87db3c4bc00
2016-01-25 10:36:41 -08:00
Yue Chen
968bbc7bb2 Adding new compound modes to EXT_INTER experiment
Combinations of different mv modes for two reference frames
are allowed in compound inter modes. 9 options are enabled,
including NEAREST_NEARESTMV, NEAREST_NEARMV, NEAR_NEARESTMV,
NEAREST_NEWMV, NEW_NEARESTMV, NEAR_NEWMV, NEW_NEARMV, ZERO_ZEROMV,
and NEW_NEWMV.
This experiment is mostly deported from the nextgen branch.
It is made compatible with other experiments

Coding gain of EXT_INTER(derflr/hevcmr/hevchd): 0.533%/0.728%/0.639%

Change-Id: Id47e97284e6481b186870afbad33204b7a33dbb0
2016-01-22 13:52:16 -08:00
Debargha Mukherjee
84ca7a9f0f Loop restoration filter
Current implementation is a bilateral filter whose
parameters are transmitted in the bitstream.

derflr: -0.647% BDRATE
hevcmr: -0.794% BDRATE

This is a prelimary patch. Various other variations are to
be investigated next, that will hopefully be less expensive
on the decoder side.

Change-Id: I50634ae8f5014ad0bf7432306348908a349d81e1
2016-01-20 17:59:46 -08:00
Yue Chen
1ac858794a EXT_INTER experiment
NEW2MV is enabled, representing a new motion vector predicted from
NEARMV. It is mostly ported from nextgen, where it was named
NEW_INTER.
A few fixes are done for sub8x8 RDO to correct some misused
mv references in the original patch.
A 'bug-fix' for encoding complexity is done, reducing the additional
encoding time from 50% to 20%. In sub8x8 case, the old patch
did motion search for every interpolation filter (vp9 only
searches once). This fix also slightly improves the coding gain.
This experiment has been made compatible with REF_MV and EXT_REFS.

Coding gain (derflr/hevcmr/hevchd): 0.267%/0.542%/0.257%

Change-Id: I9a94c5f292e7454492a877f65072e8aedba087d4
2016-01-15 14:47:02 -08:00
Debargha Mukherjee
eee6afe0b9 Fixing some compile issues
Fixes a breakage introduced with the latest merge from master and
cleans up a couple of compiler warnings.

Change-Id: Ia55b39ba78e43f6fe52c54d7f34faa4dd6bbbf26
2016-01-15 11:02:30 -08:00
Yaowu Xu
727ca802bf Merge "Merge branch 'master' into nextgenv2" into nextgenv2 2016-01-14 00:26:45 +00:00
Yaowu Xu
0367f32ea8 Merge branch 'master' into nextgenv2
Manually resovled the following conflicts:
	vp10/common/blockd.h
	vp10/common/entropy.h
	vp10/common/entropymode.c
	vp10/common/entropymode.h
	vp10/common/enums.h
	vp10/common/thread_common.c
	vp10/decoder/decodeframe.c
	vp10/decoder/decodemv.c
	vp10/encoder/bitstream.c
	vp10/encoder/encodeframe.c
	vp10/encoder/rd.c
	vp10/encoder/rdopt.c

Change-Id: I15d20ce5292b70f0c2b4ba55c1f1318181481596
2016-01-13 13:18:06 -08:00
Alex Converse
5311ad9e83 Merge "Code DCT tokens with ANS" into nextgenv2 2016-01-12 23:08:36 +00:00
Alex Converse
d1893f64e0 Code DCT tokens with ANS
Change-Id: I452f9675325a5f45bfbbe3e7e135009a125539f1
2016-01-12 09:08:19 +00:00
Debargha Mukherjee
d738ce2b0a Merge "Fixing issue with txfm context when var_tx and supertx are enabled" into nextgenv2 2016-01-11 18:40:25 +00:00
Debargha Mukherjee
a0900fd0db Remove experimental flag for ext_tx
Also includes a bug fix.

Change-Id: Ia49ed00f8ffd1531c10bcf89b1f497310ee7cb82
2016-01-08 13:48:24 -08:00
Debargha Mukherjee
f7dfa4ece7 Modifies inter/intra coding to allow all tx types
The nominal tx_type for a given mode is used as a context
to encode the actual tx_type for intra.

Results:
derflr: -0.241% BDRATE
hevcmr: -0.366% BDRATE

Change-Id: Icfe7b0a58d79bc6497a06e3441779afec6e01e21
2016-01-08 11:13:46 -08:00
Geza Lore
a167244346 Fix 2 bugs when using both SUPERTX and EXT_TX
Change-Id: Ibcbe470a97880c294600345337054ed9af84de2b
2016-01-07 12:13:35 -08:00
Julia Robson
54fbf7e55f Fixing issue with txfm context when var_tx and supertx are enabled
In the decoder, the txfm context was not being set for supertx
blocks.

Change-Id: Ifa0882bba36bc54bbd9dba3e370317b5531e33d3
2016-01-07 15:23:08 +00:00
Yaowu Xu
55118ad061 Merge branch 'master' into nextgenv2
Conflicts:
	vp10/decoder/decodeframe.c
	vp10/encoder/bitstream.c

Change-Id: I743449f49b723d2ce357832619a28b53369d0547
2016-01-06 08:36:18 -08:00
Debargha Mukherjee
3787b17439 Super transform - ported from nextgen branch
Various additional changes were made to make the experiment
compatible with misc_fixes.

derflr: +0.979%
hevcmr: +0.865%

Speed-wise with --enable-supertx the encoder is only about 10%
slower than without. Decoding impact is about 30% slowdown.

Note this does not work with ext-tx or var-tx yet. That is
a TODO.

Change-Id: If25af4241a7a9efbd28f58eda3c4f044c7a7ef4b
2016-01-04 22:12:57 -08:00
Ronald S. Bultje
53a11656cd vp10: only assume ONLY_4X4 if segmentation is disabled.
Otherwise, per-segment lossless might mean that some segments are not
lossless and they could still want to use another mode. The per-block
tx points remain uncoded on blocks where (per the segment id) the Q
value implies lossless.

Change-Id: If210206ab1fe3dd11976797370c77f961f13dfa0
2016-01-04 15:21:02 -05:00
Yaowu Xu
7330108009 Merge branch 'master' into nextgenv2 2015-12-18 05:00:05 -08:00
Zoe Liu
ec36a2b061 Restore the flexibility for the new 3 references
For the experiment of EXT_REFS, removed the previous special handling
on the new last 3 references, i.e. LAST2_FRAME, LAST3_FRAME, and
LAST4_FRAME, at the decoder, so that these new last references are
treated the same way as the other 3 references (LAST_FRAME,
GOLDEN_FRAME, and ALTREF_FRAME). Encoder changes have been made
accordingly to realize this flexibility.

Change-Id: Ic6546f9443b4377bb7e7b101bfa3e70a8b8d1c65
2015-12-17 16:34:02 -08:00
Yaowu Xu
e650129683 Move bit_depth init out of setup_quantization
This also fixes a compiling error under --enable-vp9_highbitdepth.

Change-Id: I9d1dcb95d3336d797eb3c23a4702c30b04355357
2015-12-16 11:43:11 -08:00
Yaowu Xu
dab7515aa4 Merge branch 'master' into nextgenv2
With a few manual fixes of merge conflicts.

Change-Id: I0dd65ff90f9fa8606e5563f528659e2607b12376
2015-12-16 09:00:57 -08:00
Yaowu Xu
c7101830a6 Fix a enc/dec mismatch under CONFIG_MISC_FIXES
The culprit is on the decode side xd->lossless[i] setup was in wrong
location where segment features are not yet decoded.

Also on the encoder side, transform mode was not set consistently
between when tx_mode is selected and how tx_mode is enforced in
tx size selection.

Change-Id: I4c4c32188fda7530cadab9b46d4201f33f7ceca3
2015-12-14 20:56:37 -08:00
Yaowu Xu
f07d73b9bf Merge branch 'master' into nextgenv2
Change-Id: Id0b784b115602e2502b42fa972a5ae210435a3be
2015-12-11 08:58:40 -08:00
Yaowu Xu
221ed5e47b Fix enc/dec mismatches for aq-mode 1 and 2
Change-Id: Iefad9d8d96a08dcc788a5efdca2df6a815d1205f
2015-12-10 11:45:26 -08:00
Jingning Han
1dc18077b8 Re-design motion compensated prediction mode entropy coding system
This commit re-works the entropy coding scheme of the motion
compensated prediction modes. It allows more flexible hyperplane
partition for precise classification.

Change-Id: Iba5035c76691946cf1386b6c495e399c3d9c8fc5
2015-12-09 18:02:20 -08:00
hui su
c93e5cc3e9 Bring palette back to nextgenv2
It was removed by the master branch merge.

Change-Id: I4b2a524c9e052e41063359afcb4ba22bf78344cf
2015-12-07 18:24:15 -08:00
Yaowu Xu
69f4930041 Merge branch 'master' into nextgenv2
Conflicts:
	vp10/common/blockd.h
	vp10/common/entropymode.h
	vp10/common/reconintra.c
	vp10/decoder/decodemv.c
	vp10/encoder/bitstream.c
	vp10/encoder/encoder.h
	vp10/encoder/rd.c
	vp10/encoder/rdopt.c
	vp10/encoder/tokenize.h

Change-Id: Ic4891839b6f0474026d6d69821e38edec9632df1
2015-12-07 11:37:14 -08:00
hui su
5d3327e891 Remove palette from VP10
Store it in nextgenv2 for now.

Change-Id: Iab0af0e15246758e3b6e8bde4a74b13c410576fc
2015-12-03 12:30:47 -08:00
Debargha Mukherjee
6ef5d8c4ed Merge "Reduce transform options for ext-tx experiment" into nextgenv2 2015-11-24 21:30:10 +00:00