Commit Graph

221 Commits

Author SHA1 Message Date
Jingning Han
47c7fd984e Fix a switch condition in select_tx_block
Change-Id: I3d90a0286c5ef559b91ad298db97e8990becf85f
2015-10-30 13:01:52 -07:00
Jingning Han
bfeac5e19c Support per transform block skip coding
Allow the encoder to drop individual transform block coding.

Change-Id: I2c2b2985254cb92baf891f03daa33f067279373b
2015-10-30 08:55:17 -07:00
Jingning Han
3edad6e887 Enable entropy coding of recursive transform block partition
This commit enables the entropy coding of the recursive transform
block partition syntax.

Change-Id: I0c2509fb7b9822d12a721f9ebf9327fac83c777e
2015-10-29 11:06:46 -07:00
Debargha Mukherjee
8a4292441f Refactoring tx-types to add more flexibility
Allows inter and intra tx_types to have different sets of
transforms for different tx_size/sb_type combinations.

Change-Id: Ic0ac1daef7a9fb15c4210271e4d04cd36e5cec8e
2015-10-28 23:31:32 -07:00
Jingning Han
71c156070c Use precise distortion metric
Rework the rate distortion optimization pipeline. Use precise
distortion metric that accounts for the forward and inverse
transform rounding effect.

Change-Id: Ibe19ce9791ec3547739294cc3012dd9e11f4ea49
2015-10-28 11:47:14 -07:00
Jingning Han
4bfed0b32e Account for variable txfm sizes in coeff token packing
This commit makes the coefficient token packtization process account
for variable transform block sizes supported in a single processing
block. It fixes an enc/dec mismatch issue when var-tx, ext-tx, and
misc-fixes experiments are all turned on.

Change-Id: I2e8946e6f72de567603a568debbadad11196430c
2015-10-28 11:45:31 -07:00
Yaowu Xu
b6da40ad82 Merge branch 'master' into nextgenv2
Change-Id: I0e4030a37354bb23b3aa8be5cc1473770b9e7b06
2015-10-27 08:28:09 -07:00
Jingning Han
236623cf2c Fix early termination flag in recursive transform block search
Properly reset the early termination flag in the recursive transform
block partitioning rate-distortion optimization scheme.

Change-Id: Ibfe918f21f11dcb1ec267c09f954c635305cc95a
2015-10-26 19:41:14 -07:00
Jingning Han
f0dee7765a Fix lossless coding
Use inter_block_yrd as rate-distortion optimization for lossless
coding. This fixes transform coefficient buffer swap use case and
resolves the unit test failure related to lossless coding.

Change-Id: I1512dab5ed5760c31f7de21a06e8d9ed1eb081fa
2015-10-26 22:03:39 +00:00
Jingning Han
79fe7246c1 Properly handle non-420 color format in recursive transform scheme
This commit makes the recursive transform block partitioning properly
handle the non-420 color format. It resolves an enc/dec mismatch
issue in that setting when var-tx experiment is turned on.

Change-Id: I48a91de02c11b3153f897d1cca0ae948eec15605
2015-10-23 14:42:01 -07:00
Jingning Han
caeb10bf06 Use explicit block position in foreach_transformed_block
Add the row and column index to the argument list of unit functions
called by foreach_transformed_block wrapper. This avoids the
repeated internal parsing according to the block index.

Change-Id: Ie7508acdac0b498487564639bc5cc6378a8a0df7
2015-10-23 09:19:17 -07:00
Yaowu Xu
df33be4b22 Merge "Fix merge defects" into nextgenv2 2015-10-23 01:28:32 +00:00
Yaowu Xu
5a27b3bb85 Fix merge defects
This commit fixes the merge conflicts between master and nextgenv2 and
disable early termination in choose_tx_size() to avoid failure in test.

The test failures are pre-existing, some of the issue were fixed in
masterbase already, so will have another merge to introduce the fixes.

Change-Id: Ib71889661955e73aedbb4db49d8be70425281dcb
2015-10-22 18:25:41 -07:00
Jingning Han
18aea429af Merge "Reset tx_type in recursive transform block partitioning" into nextgenv2 2015-10-22 21:34:32 +00:00
Jingning Han
d3e5545fa6 Reset tx_type in recursive transform block partitioning
Temporarily reset the transform type in the inter modes when
recursive transform block partitioning is used. This resolves an
enc/dec mismatch issue in nextgenv2 codebase when both var-tx and
ext-tx experiments are turned on.

Change-Id: I2543f0a567243da95b237752d46964b07b669ad9
2015-10-22 14:28:56 -07:00
Yaowu Xu
4ac2ae3a4d Merge branch 'masterbase' into nextgenv2
Conflicts:
	configure
	test/vp9_encoder_parms_get_to_decoder.cc
	vp10/common/blockd.h
	vp10/common/entropymode.c
	vp10/common/entropymode.h
	vp10/common/idct.c
	vp10/decoder/decodeframe.c
	vp10/decoder/decodemv.c
	vp10/encoder/bitstream.c
	vp10/encoder/encodeframe.c
	vp10/encoder/encodemb.c
	vp10/encoder/encoder.c
	vp10/encoder/encoder.h
	vp10/encoder/rd.c
	vp10/encoder/rdopt.c
	vp10/encoder/tokenize.c
	vp10/encoder/tokenize.h
	vp9/decoder/vp9_decodeframe.c
	vp9/decoder/vp9_decoder.h
	vp9/encoder/vp9_aq_cyclicrefresh.c
	vp9/encoder/vp9_encoder.h
	vp9/vp9_cx_iface.c
	vpx/vp8cx.h
	vpx_dsp/x86/vpx_subpixel_8t_intrin_ssse3.c
	vpx_scale/yv12config.h

Change-Id: I604a329d38badec7a11e8ede16ca1404476e9b93
2015-10-22 11:40:44 -07:00
Jingning Han
20484048a9 Fix compiler error with high bit-depth and var-tx
Clear the compiler errors when both high bit-depth and recursive
transform block partition experiments are enabled.

Change-Id: If0b6396851f10c28b4f26350322ccd1ba2fc9aff
2015-10-21 17:38:00 -07:00
Hui Su
96b69deca5 Merge "VP10: some changes to palette mode" 2015-10-20 16:37:31 +00:00
Ronald S. Bultje
60c58b5284 vp10: per-segment lossless coding.
Some more testing of this patch would probably be useful, but I
think the basics of it should work fine now.

See issue 1035.

Change-Id: I4a36d58f671c5391cb09d564581784a00ed26245
2015-10-16 19:30:39 -04:00
Ronald S. Bultje
c7dc1d78bf vp10: add extended-intra prediction edges experiment.
This experiment allows using full above/right edges for all transform
sizes whenever available (for d45/d63), and adds bottom/left edges for
d207.

See issue 1043.

Change-Id: I5cf7f345e783e8539bb6b6d2c9972fb1d6d0a78b
2015-10-16 19:30:39 -04:00
Ronald S. Bultje
d8f3bb1837 vp10: merge keyframe/interframe uvintramode/partition probabilities.
This has various benefits:
- simplify implementations because we don't have to switch between
  multiple probability tables depending on frametype
- allows fw subexp and bw adaptivity for partitions/uvmode in keyframes

See issue 1040 point 5.

Change-Id: Ia566aa2863252d130cee9deedcf123bb2a0d3765
2015-10-16 19:30:38 -04:00
hui su
17c817adfc VP10: some changes to palette mode
Account for rounding in distortion calculation in k-means;
carry out rounding before duplicates removal of base colors;
replace numbers with macros;
use prefix increment.

Slight coding gain (<0.1%) on screen_content testset.

Change-Id: Ie8bd241266da6b82c7b2874befc3a0c72b4fcd8c
2015-10-16 11:41:26 -07:00
hui su
aaf6f6215f Fix palette mode in multi-thread encoding setting
Fix a couple of memory related errors. Also fix thread test failures.

Change-Id: I0103995f832cecf1dd2380000321ac7204f0cfc0
2015-10-15 15:00:57 -07:00
Yaowu Xu
15cc8bc72f Merge "fix a msvc compiler warning" 2015-10-15 14:39:01 +00:00
Yaowu Xu
8ced62f250 fix a msvc compiler warning
Change-Id: Ifd6581c1bdb8d8f4b2ecf676c1a3d385dc129abf
2015-10-15 01:05:13 +00:00
Yaowu Xu
4727fa2a75 Fix two asan failures
Change-Id: I57865e9604ac162ef0d97deb16e81ca436a98428
2015-10-14 18:03:31 -07:00
Jingning Han
1e48f74d9a Enable early termination in the recursive transform block search
It makes the encoder 5% faster for CIF clips and 12% faster for
1080p clips.

Change-Id: I073408dbb4d50675a79db8794fe73975ac957b91
2015-10-13 21:14:58 +00:00
Jingning Han
3a27961cf3 Refactor recursive transform block scheme
This commit re-designs the recursive transform block partition
rate-distortion optimization framework. It allows the encoder to
improve speed by 10%.

Change-Id: I6dd3a7dd428a530d8012e5c6ddc40e650c8b392b
2015-10-13 21:13:29 +00:00
Jingning Han
2cdc12742d Rate-distortion optimization for recursive transform block coding
This commit enables the rate-distortion optimization for recursive
transform block coding scheme.

Change-Id: Id6a8336ca847bb3af1e94cbfb51db1f4da12d38f
2015-10-13 12:49:03 -07:00
Hui Su
b9e31b5163 Merge "VP10: Add palette mode part 1" 2015-10-13 17:34:27 +00:00
Ronald S. Bultje
5b4805d6e9 vp10: remove clamp_mv2() call from vp10_find_best_ref_mvs().
This actually has no effect whatsoever, since the input MVs themselves
are clamped by clamp_mv_ref() already, which is significantly more
restrictive in its bounds.

Change-Id: I4a3a7b2b121ee422c56428c2a12d930c3813c06e
2015-10-12 14:45:18 -04:00
hui su
5d011cb278 VP10: Add palette mode part 1
Add palette mode for keyframe luma channel. Palette mode is enabled
when using "--tune-content=screen" in encoding config parameters.

on screen_content testset:  +6.89%
on derlr                 :  +0.00%

Design doc (WIP):
https://goo.gl/lD4yJw

Change-Id: Ib368b216bfd3ea21c6c27436934ad87afdaa6f88
2015-10-12 10:02:17 -07:00
Jingning Han
a8dad55c82 Make chroma component RD estimate support transform partition
This commit makes the rate-distortion optimization for chroma
component support the recursive transform block coding scheme.

Change-Id: I1bfed6d05b0ebb3905cb625222401e2ccbae10f3
2015-10-08 18:04:03 -07:00
Jingning Han
704985e65a Add decoder support to recursive transform block partition
This commit allows the decoder to recursively parse and rebuild
the pixel blocks.

Change-Id: I510f3a30ae7cdad5b70725c66882b00a0594e96f
2015-10-08 16:16:41 -07:00
Jingning Han
ebc48efe37 Use explicit block position in foreach_transformed_block
Add the row and column index to the argument list of unit functions
called by foreach_transformed_block wrapper. This avoids the
repeated internal parsing according to the block index.

Change-Id: I42b3578eac258ebaba7a7c74f684de9abab521a6
2015-10-07 16:32:19 -07:00
Jingning Han
00ca5c1c98 Simplify vp10_xform_quant index parsing
Change-Id: Id7f7a9b2e53fc0074b55d58143f296afad6b844e
2015-10-06 17:19:23 -07:00
hui su
4f16f11993 Refactor ext-tx experiment
Remove unnecessary transform type lookups and unused codes.

Change-Id: Ib52d26690468996b1501b419d919643c8ea5ecaa
2015-10-02 11:00:51 -07:00
hui su
3fa0129caf Extend ext_tx experiment to intra blocks
ext-tx on derflr +2.30% (was +1.84%)

Change-Id: Ic91565cacc38e7a8e1200d054ed7bf99295fe19e
2015-10-02 09:39:38 -07:00
hui su
2afe7320c8 Add identity transform to ext-tx experiment
ext-tx on derflr: +1.756% (was +1.648)

Change-Id: I8a87970fa589e8f5f96db7aa68ec9b6c98e20188
2015-09-30 18:47:46 -07:00
Hui Su
4b7043f804 Merge "ext-tx experiment support in choose_largest_tx_size" into nextgenv2 2015-09-30 15:14:17 +00:00
Debargha Mukherjee
3e8cceb3fc Speed up of DST and the search in ext_tx
Adds an early termination to the ext_tx search, and also
implements the DST transforms more efficiently.

About 4 times faster with the ext-tx experiment.

There is a 0.09% drop in performance on derflr from 1.735% to
1.648%, but worth it with the speedup achieved.

Change-Id: I2ede9d69c557f25e0a76cd5d701cc0e36e825c7c
2015-09-29 19:11:43 -07:00
Yaowu Xu
7c514e2dfd Merged branch 'master' into nextgenv2
Resolved Conflicts in the following files:
        configure
        vp10/common/idct.c
        vp10/encoder/dct.c
        vp10/encoder/encodemb.c
        vp10/encoder/rdopt.c

Change-Id: I4cb3986b0b80de65c722ca29d53a0a57f5a94316
2015-09-29 16:17:32 -07:00
hui su
6c81e37916 ext-tx experiment support in choose_largest_tx_size
Change-Id: Ic161b8b257a02c1c43e515d830c1051d0de074de
2015-09-29 12:09:15 -07:00
Ronald S. Bultje
bab8d38f7f vp10: remove MACROBLOCK.{highbd_,}itxfm_add function pointer.
This is preparatory work for allowing per-segment lossless coding.

See issue 1035.

Change-Id: I9487d02717ee3e766aee61a487780056bb35d2d3
2015-09-25 19:30:46 -04:00
Ronald S. Bultje
c74b33a413 vp10: remove MACROBLOCK.fwd_txm4x4 function pointer.
This is preparatory work for allowing per-segment lossless coding.

See issue 1035.

Change-Id: Idd72e2a42d90fa7319c10122032d1a7c7a54dc05
2015-09-25 19:30:46 -04:00
hui su
38cc168822 Adjust rd calculation in choose_tx_size_from_rd
Coding gain:
derflr 0.142%
hevclr 0.153%
hevcmr 0.124%

Change-Id: I63b56ae3a9002c3a266e10e2964135ed43b0ba53
2015-09-23 10:54:28 -07:00
hui su
07154b0216 Refactor ext-tx experiment
At this point, ext-tx compapred to baseline +1.735%.

Change-Id: Ia16ac293e2cc87e06a0d898c1d52a8f3495ff814
2015-09-23 09:14:49 -07:00
hui su
8e273b23ad Adjust rd calculation in choose_tx_size_from_rd
Consider tha case in which skipping transform coefficients is more
efficient.

derflr +0.13%
hevclr +0.11%
hevcmr +0.14%
hevchr +0.22%

with ext-tx, the impact is -0.02%.

Change-Id: I0aa2965cf9e152396623c2fee62545bd3a3a7f07
2015-09-23 09:13:55 -07:00
hui su
38debe512e Simplify choose_tx_size_from_rd
No impact on performance.

Change-Id: Ib0420b190c9d83ef47b14ea78d5918a6a5078e3a
2015-09-23 09:08:47 -07:00
Jingning Han
d6be2671ed Create sub8x8 block inter prediction function
Change-Id: Ib161e6fb3eb081f7176a1d969fed16a7d1ffc320
2015-09-18 16:31:36 -07:00