1015 Commits

Author SHA1 Message Date
Yue Chen
11af20dbeb Fixing a issue of calculating tx cost for SUPERTX+VAR_TX
Update blk_skip in update_state_supertx() and rd_supertx_sb().
Performance (SUPERTX+VAR_TX): TBD
(Eventually will merge update_state() and update_state_supertx())

Change-Id: I34ef982b80151ba2dfba745859cb2ca7b90dc888
2016-01-29 09:31:04 +00:00
Zoe Liu
b6f1f369af Cleaned code in firstpass related to altref frame
Change-Id: I6dbddf500fd4f64216774685700efea972165470
2016-01-28 16:37:17 -08:00
James Zern
4f780e94a1 vp9/10: fix encoder crash on flush
the lookahead buffer allocation is deferred to receipt of the first
frame to allow profile changes. if the encoder was flushed before
supplying any frames the encoder would crash trying to dereference the
NULL buffer. vp8 is unaffected.

fixes mozilla bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1237848

Change-Id: Icee4b64de760476eee0d33b568f0a1010335ff13
2016-01-28 12:53:41 -08:00
Debargha Mukherjee
3eb10fcf21 Cosmetic changes to loop restoration
Also adds a normalized filtering function to be used later.

Change-Id: I30e2140e664db635602f26a73b81ce8e008dff5e
2016-01-27 17:33:36 -08:00
Debargha Mukherjee
eef57c1e99 Fixes ext-interp experiment
Fixes integer pel MV usage for the sub8x8 case, which fixes a
rare mismatch issue.

Also adds some other minor missing code related to filter threshes.

Change-Id: I6b07e6cf9b287ba4b5bd6599af4a7412e50b3bdc
2016-01-27 09:24:48 -08:00
Geza Lore
5aa655f79a Reintroduce VAR_TX fix
Reintroduce part of
Iaf2b717e6b8626b2b6a03226127221b776b49884
Which was later reverted in
I4c5b40ec63a6f19521191d3c730af87db3c4bc00

Change-Id: If3e5610ba3985ae7b4d952d8e616982465ac667a
2016-01-26 15:27:47 +00:00
Geza Lore
f33755ef1f Always recode SUPERTX blocks.
There is still an assertion failure when tokenizing transform
coefficients for a supertx block, due to the eob not being set
consistently with the coefficients, so we always recode supertx blocks
for now. Also added further PICK_MODE_CONTEXT instances to avoid
potential clash between horizontal/vertical/split partition SUPERTX
trials.

Change-Id: I5f3da1fa0d8d20fc21face170487e1a285fd1cc6
2016-01-26 15:21:07 +00:00
Geza Lore
753dcbe5f9 Merge "Update VAR_TX related context when using SUPERTX." into nextgenv2 2016-01-26 15:20:32 +00:00
Yaowu Xu
f512a311f2 Merge branch 'master' into nextgenv2 2016-01-26 05:00:05 -08:00
Geza Lore
22cfc841d5 Update VAR_TX related context when using SUPERTX.
The encoder did not update left_txfm_context and above_txfm_context in
MACROBLOCKD (used for choosing the probability context for the vartx
split bits) when the supertx bit was set for a block. The deoder on the
other hand did update these for supertx blocks. The encoder used these
to compute the context counts, which the packer then uses to adapt it's
probabilities. This results in the packer and the decoder using
different probabilities.

This patch harmonizes the encoder and the decoder by making the encoder
update the mentioned context for supertx coded blocks.

Change-Id: I3a22132124b1bce2ee501d640ceab374b19e3ca1
2016-01-26 01:18:56 +00:00
Geza Lore
56686b4514 Initalize mbmi->tx_size during rdopt.
This is necessary when using SUPERTX, as the bitstream packer relies on
tx_size being set correctly to decide whether to output the block using
supertx or not.

Change-Id: I79e776b3b810f4a15b9dbc6afdd6fc90c73c8934
2016-01-26 01:18:56 +00:00
Geza Lore
e7c0e157d2 Set inter_tx_size for supertx coded blocks.
The loop filter relies on inter_tx_size in MB_MODE_INFO being set
properly when VAR_TX is enabled. Supertx coded blocks did not set this
previously at all, and the differing garbage values eventually resulted
in in a YUV mismatch between encoder and decoder after loop filtering.

This patch fixes this by setting inter_tx_size to the proper supertx
size in both the encoder and the decoder. This should also mean that
loop filtering is done at the proper transform boundaries, even when
supertx or vartx is being used.

Change-Id: I41a564cd6d34ce4a8313ad4efa89d905f5ead731
2016-01-26 01:18:56 +00:00
Debargha Mukherjee
a8122bb957 A minor assert fix for supertx
Change-Id: I532fff64ccaa1f38240ba7ca5ce2f7e1eb531771
2016-01-25 17:15:51 -08:00
Debargha Mukherjee
9a8a6a1b35 Some supertx fixes
Fixes some of the issues introduced by a merge from master.

derflr: -0.893% BDRATE
hevcmr: -1.667% BDRATE

Change-Id: I4c5b40ec63a6f19521191d3c730af87db3c4bc00
2016-01-25 10:36:41 -08:00
Zoe Liu
9e612763f0 Cleaned a code in define_gf_group() for firstpass
Change-Id: I0b8eff9e74e3446024482d011ec4fb28b19136d3
2016-01-25 09:38:45 -08:00
Yue Chen
968bbc7bb2 Adding new compound modes to EXT_INTER experiment
Combinations of different mv modes for two reference frames
are allowed in compound inter modes. 9 options are enabled,
including NEAREST_NEARESTMV, NEAREST_NEARMV, NEAR_NEARESTMV,
NEAREST_NEWMV, NEW_NEARESTMV, NEAR_NEWMV, NEW_NEARMV, ZERO_ZEROMV,
and NEW_NEWMV.
This experiment is mostly deported from the nextgen branch.
It is made compatible with other experiments

Coding gain of EXT_INTER(derflr/hevcmr/hevchd): 0.533%/0.728%/0.639%

Change-Id: Id47e97284e6481b186870afbad33204b7a33dbb0
2016-01-22 13:52:16 -08:00
Debargha Mukherjee
11d3899c60 Merge "A fix for a missing tx_type" into nextgenv2 2016-01-22 18:36:46 +00:00
Debargha Mukherjee
4e406f702c A fix for a missing tx_type
Change-Id: I165cd06256175edb7739020379ba3098251f4a7c
2016-01-22 03:29:39 -08:00
Debargha Mukherjee
ade60d4fe9 Merge "Fixes for var_tx when ext_tx is not enabled" into nextgenv2 2016-01-22 06:09:36 +00:00
Julia Robson
9fe188e4a5 Fixes for var_tx when ext_tx is not enabled
This patch fixes a couple of issues caused by change-id:
I15d20ce5292b70f0c2b4ba55c1f1318181481596
Changes to the code for when the ext_tx experiment is not enabled
were merged from master but as var_tx does not exist on master
the changes had not been applied to the case when var_tx experiment
is enabled

Change-Id: Iaf2b717e6b8626b2b6a03226127221b776b49884
2016-01-21 17:46:03 -08:00
Debargha Mukherjee
8d69a6e816 Some fixes on tx size/type selection
For ext_tx experiment.

Change-Id: Ie37b9b456b09bde8b606fb978fee4cca8d0326b7
2016-01-21 17:45:30 -08:00
Debargha Mukherjee
09f3615606 Merge "Loop restoration filter" into nextgenv2 2016-01-21 17:43:39 +00:00
Debargha Mukherjee
0880b42129 Merge "Making the forward transform consistent with high bit depth" into nextgenv2 2016-01-21 15:22:48 +00:00
Debargha Mukherjee
84ca7a9f0f Loop restoration filter
Current implementation is a bilateral filter whose
parameters are transmitted in the bitstream.

derflr: -0.647% BDRATE
hevcmr: -0.794% BDRATE

This is a prelimary patch. Various other variations are to
be investigated next, that will hopefully be less expensive
on the decoder side.

Change-Id: I50634ae8f5014ad0bf7432306348908a349d81e1
2016-01-20 17:59:46 -08:00
Yaowu Xu
a40d486215 Merge branch 'master' into nextgenv2
Change-Id: I4e5dd38caa9608252235265c7f2342b183f99815
2016-01-20 09:24:41 -08:00
Julia Robson
c178b2d192 Making the forward transform consistent with high bit depth
This patch changes the code for 16bit buffers to use the same
optimisation as is used for 8bit buffers. (See change-Id:
I0452da1786d59bc8bcbe0a35fdae9f623d1d44e1 for more information
about the optimisation)

Change-Id: I5f327a13a7b01fc356114a2aa9d1261bf76d8d69
2016-01-20 12:03:16 +00:00
hui su
2778b1cbb9 Fix a bug with ext-intra when skip_recode is enabled
Change-Id: I906945d61254149b315a6de81ac6373ed31791e6
2016-01-19 14:54:31 -08:00
Yue Chen
1ac858794a EXT_INTER experiment
NEW2MV is enabled, representing a new motion vector predicted from
NEARMV. It is mostly ported from nextgen, where it was named
NEW_INTER.
A few fixes are done for sub8x8 RDO to correct some misused
mv references in the original patch.
A 'bug-fix' for encoding complexity is done, reducing the additional
encoding time from 50% to 20%. In sub8x8 case, the old patch
did motion search for every interpolation filter (vp9 only
searches once). This fix also slightly improves the coding gain.
This experiment has been made compatible with REF_MV and EXT_REFS.

Coding gain (derflr/hevcmr/hevchd): 0.267%/0.542%/0.257%

Change-Id: I9a94c5f292e7454492a877f65072e8aedba087d4
2016-01-15 14:47:02 -08:00
Alex Converse
dea6f5217e Deduplicate sign cost for ONE/TWO/THREE/FOUR tokens
Change-Id: I9c383d604dfd240e38b198ac5bf86ddd3456f442
2016-01-15 13:56:34 -08:00
Yaowu Xu
44da65fb44 Merge branch 'master' into nextgenv2 2016-01-14 13:57:27 -08:00
Jingning Han
c38fc52df4 Merge "Handle single ref mv pair in the candidate list for compound mode" into nextgenv2 2016-01-14 02:55:36 +00:00
Yaowu Xu
727ca802bf Merge "Merge branch 'master' into nextgenv2" into nextgenv2 2016-01-14 00:26:45 +00:00
Jingning Han
3944cfb14d Handle single ref mv pair in the candidate list for compound mode
This commit considers the case where a single reference motion
vector pair is found in the candidate list. It treats this pair
as the effective motion vector for nearestmv mode. This improves
the coding performance by 0.06% for stdhd sets.

Change-Id: I9ce12f456b52912933e05c18c3841a78c26155d2
2016-01-13 16:19:27 -08:00
Yaowu Xu
0367f32ea8 Merge branch 'master' into nextgenv2
Manually resovled the following conflicts:
	vp10/common/blockd.h
	vp10/common/entropy.h
	vp10/common/entropymode.c
	vp10/common/entropymode.h
	vp10/common/enums.h
	vp10/common/thread_common.c
	vp10/decoder/decodeframe.c
	vp10/decoder/decodemv.c
	vp10/encoder/bitstream.c
	vp10/encoder/encodeframe.c
	vp10/encoder/rd.c
	vp10/encoder/rdopt.c

Change-Id: I15d20ce5292b70f0c2b4ba55c1f1318181481596
2016-01-13 13:18:06 -08:00
Debargha Mukherjee
e9db70fb63 Merge "Use specific PICK_MODE_CONTEXT for supertx." into nextgenv2 2016-01-13 19:41:39 +00:00
Jingning Han
177601b561 Merge "Generate compound reference motion vector" into nextgenv2 2016-01-13 19:37:38 +00:00
Yaowu Xu
d803e8a20e Fix encoder crashes and enc/dec mismatches
This commit adds the logic for segmentation map initialization and
disable temporal update of segmentation map when error-resilient
mode is on. It fixes the enc/dec mistmates (release build) and
assertions(debug) when both aq-mode and error-resilient are on.

Change-Id: Id2155e8b28962cf1f64494f4df0c8d79499b6890
2016-01-13 09:51:05 -08:00
Jingning Han
33cc1bd21d Generate compound reference motion vector
This commit allows the codec to add motion vector pairs into
the candidate list. It further improves the compression performance
by 0.1% across derf, hevcmr, stdhd, and hevchr sets without adding
encode/decode time.

Change-Id: I88d36da25a2a89bb506d411844af667081eba98b
2016-01-12 15:28:47 -08:00
Geza Lore
e4663e6da1 Use specific PICK_MODE_CONTEXT for supertx.
Change-Id: I402d10dd666ebc4a06ce4472810a7e22f2e056ff
2016-01-12 13:06:42 +00:00
Alex Converse
d1893f64e0 Code DCT tokens with ANS
Change-Id: I452f9675325a5f45bfbbe3e7e135009a125539f1
2016-01-12 09:08:19 +00:00
Debargha Mukherjee
a0900fd0db Remove experimental flag for ext_tx
Also includes a bug fix.

Change-Id: Ia49ed00f8ffd1531c10bcf89b1f497310ee7cb82
2016-01-08 13:48:24 -08:00
Debargha Mukherjee
f7dfa4ece7 Modifies inter/intra coding to allow all tx types
The nominal tx_type for a given mode is used as a context
to encode the actual tx_type for intra.

Results:
derflr: -0.241% BDRATE
hevcmr: -0.366% BDRATE

Change-Id: Icfe7b0a58d79bc6497a06e3441779afec6e01e21
2016-01-08 11:13:46 -08:00
Jingning Han
387a10e3dc Enable context analyzer for inter mode entropy coding
It allows the codec to account for certain corner cases when
processing inter prediction mode entropy coding.

Change-Id: Ied451f4fff26ba579f6556554b8381ff2ccd0003
2016-01-08 10:27:27 -08:00
Geza Lore
a167244346 Fix 2 bugs when using both SUPERTX and EXT_TX
Change-Id: Ibcbe470a97880c294600345337054ed9af84de2b
2016-01-07 12:13:35 -08:00
Debargha Mukherjee
b5480b3594 Merge "Change to rd costing for CONFIG_VAR_TX" into nextgenv2 2016-01-06 23:49:01 +00:00
Yaowu Xu
55118ad061 Merge branch 'master' into nextgenv2
Conflicts:
	vp10/decoder/decodeframe.c
	vp10/encoder/bitstream.c

Change-Id: I743449f49b723d2ce357832619a28b53369d0547
2016-01-06 08:36:18 -08:00
Peter de Rivaz
2f943131df Change to rd costing for CONFIG_VAR_TX
In select_tx_block I believe the rd cost was ignoring the bits
needed to encode the split bit.

Change-Id: Iacbf705b880db9a68967a994406ba90ecf693ab8
2016-01-05 15:35:43 +00:00
Debargha Mukherjee
3787b17439 Super transform - ported from nextgen branch
Various additional changes were made to make the experiment
compatible with misc_fixes.

derflr: +0.979%
hevcmr: +0.865%

Speed-wise with --enable-supertx the encoder is only about 10%
slower than without. Decoding impact is about 30% slowdown.

Note this does not work with ext-tx or var-tx yet. That is
a TODO.

Change-Id: If25af4241a7a9efbd28f58eda3c4f044c7a7ef4b
2016-01-04 22:12:57 -08:00
Ronald S. Bultje
53a11656cd vp10: only assume ONLY_4X4 if segmentation is disabled.
Otherwise, per-segment lossless might mean that some segments are not
lossless and they could still want to use another mode. The per-block
tx points remain uncoded on blocks where (per the segment id) the Q
value implies lossless.

Change-Id: If210206ab1fe3dd11976797370c77f961f13dfa0
2016-01-04 15:21:02 -05:00
Ronald S. Bultje
d9439fdc36 vp10: skip coding of txsz for lossless-segment blocks.
Change-Id: Ic23c10b6d2a9fed3abe69c6bf10e910832444f2c
2016-01-04 15:21:02 -05:00