1015 Commits

Author SHA1 Message Date
Jingning Han
93748c3e4f Enable dynamic motion vector referencing for newmv mode
This commit enables the dynamic motion vector predictor for NEWMV
mode. It allows the codec to select the best motion vector predictor
in a rate-distortion optimization framework for motion vector
residual coding. The compression performance is improved:
lowres  0.14%
midres  0.27%
hdres   0.24%

Change-Id: I6a601c74eb6cb0b71a613336d40363359f2edecd
2016-03-18 09:51:37 -07:00
hui su
30d2d9783e Small speed up for super_block_uvrd
Eearly termination if U plane RD cost is large enough.

No notable compression performance changes.

Change-Id: Ieeefc5859cb55d94391b502b4bd840bc8bcb2578
2016-03-18 09:28:10 -07:00
Geza Lore
f8cfb72a32 Refactor bsse and skip_txfm in MACROBLOCK.
Simple refactoring to 2 dimensional arrays, in preparation for 128
wide superblocks.

Change-Id: I40d447bd9fbd4f755534ea3cc82fc8f4676cea07
2016-03-18 15:30:10 +00:00
Geza Lore
efe7d4e5a2 Refactor mbmi->inter_tx_size to 2D array.
This is in preparation of increasing the superblock size.

Change-Id: I9197e397399fbe8aec1178a45ea0337dd90412d7
2016-03-18 15:30:09 +00:00
Hui Su
cb61bfa695 Merge "Minor bug fix in ex-intra experiment" into nextgenv2 2016-03-18 05:01:56 +00:00
hui su
507c55b227 Minor bug fix in ex-intra experiment
No performance change observed from borg tests.

Change-Id: I20c232c5dde8cfc84452a4c7185389bd8b812ffd
2016-03-17 19:37:21 -07:00
Sarah Parker
99235fd908 Merge "Fix hbd segfault for prune_one and prune_two" into nextgenv2 2016-03-17 22:40:47 +00:00
Angie Chiang
cb3072ca4a Refactor: call inv_txfm_add
Change-Id: I52c209a5db1b4a6525c04b70291a08ab5a68c6fe
2016-03-17 11:19:00 -07:00
Sarah Parker
79dd1ee907 Fix hbd segfault for prune_one and prune_two
Change-Id: I71da102550aa7d81961e5f10c71058d5fa8dc6c4
2016-03-17 09:58:29 -07:00
Geza Lore
24659dbca3 Refactor save_context restore_context in rd_pick_partition.
This is a cosmetic patch that removes a great deal of conditional
compilation around CONFIG_VAR_TX from the partition search function.

Change-Id: I9dcef9d4fe6847b793c77bdf565a5cacbdfacd59
2016-03-17 15:53:59 +00:00
Yaowu Xu
51f951292a Merge "Properly save and restore skip related variables" into nextgenv2 2016-03-17 14:41:03 +00:00
Yunqing Wang
a129905c70 Merge "Optimize HBD up-sampled prediction functions" into nextgenv2 2016-03-16 23:52:08 +00:00
Yaowu Xu
4fcc4f816e Properly save and restore skip related variables
Change-Id: Id52744e140585c08d047fb395b75ac9318a0b4c3
2016-03-16 16:28:52 -07:00
hui su
83b47af18d Add "entropy" experiment
This patch added two features to improve entropy coding efficiency
for coefficient tokens.

1. Choose 1 of 4 default probability tables based on q-index for
key-frames.
It is ported from nextgen branch:
https://chromium-review.googlesource.com/#/c/280586/

2. Do backward update after each superblock (64X64) row using
subframe token counts.

Coding gain: 0.1% on lowres; 0.42% on midres; 0.36% on hdres.
Much larger gain for key-frames: 2.6%, 2.3%, 1.7%.

Design doc: go/huisu-entropy

Change-Id: Ia3b6a615636be09247d70e4c520405637561532b
2016-03-16 11:55:50 -07:00
Angie Chiang
f330444a09 Merge "remove filter_cache" into nextgenv2 2016-03-16 17:21:53 +00:00
Geza Lore
c2005c578b Factor out zeroing above and left context.
Change-Id: I6e5d8cff869c7415a924f845c9e6ccaabe2b7a9b
2016-03-16 13:08:29 +00:00
Geza Lore
8735c17ceb Fix build without supertx.
Change-Id: Ib60821487710f6cf06aaa8ddcbdd5487ba1cbe4f
2016-03-16 13:08:13 +00:00
Yaowu Xu
2becffaef5 Properly set the estimate of rate_nocoef
This commit fixes the computation of rate_nocoef for situation when
rate_y is uninitialized at INT_MAX for  x->skip is true.

Change-Id: If3dde4e4ee16667f4408067d3bb3084f916272f1
2016-03-15 15:00:53 -07:00
Angie Chiang
b6fef12481 remove filter_cache
PSNR test
        lowres  hdres
lowbd   -0.013  0.067
highbd  -0.044  0.039

Change-Id: Iefdb1e966bd004b2027456778185b675e8fb9b81
2016-03-15 14:40:59 -07:00
Debargha Mukherjee
dcbbb81605 Merge "Refactor 1D transforms" into nextgenv2 2016-03-15 19:08:07 +00:00
Debargha Mukherjee
9b88762b17 Refactor 1D transforms
In preparation for adding more 1D variants with ADST/FlipADST/etc.

BDRATE actually improves by 0.21% on lowres.

Change-Id: I2fa4720c69fe001fa666119a284dfc6b17fffab2
2016-03-14 22:30:09 -07:00
Yunqing Wang
5f5552d846 Optimize HBD up-sampled prediction functions
Optimized 2 up-sampled reference prediction functions in high-bit
depth case. This reduced the HBD encoding time by 3%.

Change-Id: I8663ffb5234f5e70168c0fc9ca676309fe8e98f2
2016-03-14 19:04:33 -07:00
Yue Chen
66e6fb84de Merge "Speed up rd selection in OBMC experiment" into nextgenv2 2016-03-15 00:14:06 +00:00
Yue Chen
b5f8b70ce5 Speed up rd selection in OBMC experiment
Instead of testing all interpfilter-BMC/OBMC combinations, we choose
the best interpolation filter based on regular inter prediction.

Reduction in encoding time: ~10%
Drop in performance gain: 0.08% lowres, 0.04% midres

Change-Id: Ifc19097a918ac76b529db9af4c60e2c70e93f7ad
2016-03-14 15:36:44 -07:00
Yunqing Wang
91b8236cdd Merge "Add high-precision sub-pixel search as a speed feature" into nextgenv2 2016-03-12 02:26:36 +00:00
Angie Chiang
46cd6ee9bd Merge "Fix sub8x8 interpolation full pixel bug" into nextgenv2 2016-03-12 01:45:27 +00:00
Yunqing Wang
e6e2d886d3 Add high-precision sub-pixel search as a speed feature
Using the up-sampled reference frames in sub-pixel motion search is
enabled as a speed feature for good-quality mode speed 0 and speed 1.

Change-Id: Ieb454bf8c646ddb99e87bd64c8e74dbd78d84a50
2016-03-11 16:32:11 -08:00
Debargha Mukherjee
e38e2ad86e Merge "Fix an overflow in highbitdepth loop restoration" into nextgenv2 2016-03-11 21:48:37 +00:00
Hui Su
f0e0a7e7e9 Merge "Complete (mostly) migration of palette mode" into nextgenv2 2016-03-11 19:52:41 +00:00
Hui Su
571072b84b Merge "Fix a bug in ext-intra experiment" into nextgenv2 2016-03-11 19:52:34 +00:00
Debargha Mukherjee
7ea59de69c Fix an overflow in highbitdepth loop restoration
Change-Id: Ie20cd35a4c96443c0de234d2cf097187a70ec8dd
2016-03-11 11:48:24 -08:00
hui su
8102aeb368 Fix a bug in ext-intra experiment
Change-Id: I6fab352eb1f7d9c5dc783a4d4d878b6b42838ca2
2016-03-11 10:23:51 -08:00
Jingning Han
68d9a14e9f Merge "Enable hybrid 1-D/2-D transform coding for highbd setting" into nextgenv2 2016-03-11 18:09:11 +00:00
hui su
78b0bd0a0d Complete (mostly) migration of palette mode
Coding gain on screen_content is 12.2% (was 6.6%).

Some features such as frame-level color buffer, adaptive
entropy coding, are coming in future patches.

Change-Id: I2658cf5ec0cbb02cff685475759f3b68c9807697
2016-03-11 09:56:21 -08:00
Sarah Parker
09368fcf99 Filling in speed feature functions for ext tx search
Filled in prune one and prune two. Prune three is still
being experimented with.

Change-Id: Ic07f828c448e86cacb0369aa3a9a0feb2edae054
2016-03-10 14:08:13 -08:00
Debargha Mukherjee
ce4b35d510 Merge "Adds compound wedge prediction modes" into nextgenv2 2016-03-10 17:44:45 +00:00
Jingning Han
c453ae53d0 Enable hybrid 1-D/2-D transform coding for highbd setting
This commit enables the hybrid 1-D/2-D transform coding scheme for
high bit-depth setting. It improves the compression performance of
ext-tx experiment by 0.98% for lowres_all set.

Change-Id: Ic27f5037f2c36b095a93b9f15dbae34bdcdf00aa
2016-03-10 08:58:07 -08:00
Debargha Mukherjee
f34deab243 Adds compound wedge prediction modes
Incorporates wedge compound prediction modes.

Change-Id: Ie73b54b629105b9dcc5f3763be87f35b09ad2ec7
2016-03-10 07:19:54 -08:00
Jingning Han
ccc809f30c Merge "Fix an assertion condition in transform type search" into nextgenv2 2016-03-10 00:20:30 +00:00
Yi Luo
431e35913e Merge "Implemented DST 16x16 SSE2 intrinsics optimization" into nextgenv2 2016-03-09 22:27:44 +00:00
Angie Chiang
836e83c49f Fix sub8x8 interpolation full pixel bug
Change-Id: I5df744dc6b21ed9dbbf6ddf38004f2a9e88b7d00
2016-03-09 11:15:19 -08:00
Jingning Han
02734b6457 Fix an assertion condition in transform type search
Change-Id: I442475e559be2acdc1c2a3e5ca021b3de77adda5
2016-03-09 19:07:23 +00:00
hui su
954e560f9e Refactor entropy coding of transform size
No performance change.

Change-Id: If35125fed909d89235b303514f77a33183bb36b3
2016-03-08 16:46:00 -08:00
Yi Luo
50a164a1f6 Implemented DST 16x16 SSE2 intrinsics optimization
- Implemented fdst16_sse2(), fdst16_8col() against C version: fdst16().
- Turned on 7 DST related hybrid txfm types in vp10_fht16x16_sse2().
- Replaced vp10_fht10x10_c() with vp10_fht16x16_sse2() in
  fwd_txfm_16x16().
- Added vp10_fht16x16_sse2() unit test against C version:
  vp10_fht16x16_c() (--gtest_filter=*VP10Trans16x16*).
- Unit test passed.
- Speed improvement: 2.4%, 3.2%, 3.2%, for city_cif.y4m, garden_sif.y4m,
  and mobile_cif.y4m.

Change-Id: Ib30a67ce5d5964bef143d588d0f8fa438be8901f
2016-03-08 14:56:38 -08:00
Yaowu Xu
de661cdbc5 Merge "Fix several MSVC compiler warning/errors" into nextgenv2 2016-03-08 16:44:17 +00:00
Yaowu Xu
28eb784e46 Fix several MSVC compiler warning/errors
Change-Id: Iccaacee9b7a66b016b5747a3902c236888ad4ba1
2016-03-07 17:00:03 -08:00
Yue Chen
043b698a87 Merge "Calculate the distortion in pixel domain for sub8x8 rd selection" into nextgenv2 2016-03-08 00:13:46 +00:00
Yue Chen
ef8f7c1211 Calculate the distortion in pixel domain for sub8x8 rd selection
Pixel domain distortion calculation is enabled for the rd loop of
inter sub8x8 and intra 4x4 cases.

Coding gain: 0.124% derflr, 0.122% derfhd

Change-Id: I43b47fe81b4f5ccc1c66bc626bd310c413a1ed87
2016-03-07 14:49:22 -08:00
Alex Converse
76d4fdd391 Merge "ANS: Switch from PDFs to CDFs." into nextgenv2 2016-03-07 20:51:45 +00:00
Jingning Han
79c5a533cd Merge "Hybrid 1-D/2-D transform coding" into nextgenv2 2016-03-07 19:15:44 +00:00