454 Commits

Author SHA1 Message Date
Angie Chiang
836e83c49f Fix sub8x8 interpolation full pixel bug
Change-Id: I5df744dc6b21ed9dbbf6ddf38004f2a9e88b7d00
2016-03-09 11:15:19 -08:00
Jingning Han
02734b6457 Fix an assertion condition in transform type search
Change-Id: I442475e559be2acdc1c2a3e5ca021b3de77adda5
2016-03-09 19:07:23 +00:00
hui su
954e560f9e Refactor entropy coding of transform size
No performance change.

Change-Id: If35125fed909d89235b303514f77a33183bb36b3
2016-03-08 16:46:00 -08:00
Yaowu Xu
de661cdbc5 Merge "Fix several MSVC compiler warning/errors" into nextgenv2 2016-03-08 16:44:17 +00:00
Yaowu Xu
28eb784e46 Fix several MSVC compiler warning/errors
Change-Id: Iccaacee9b7a66b016b5747a3902c236888ad4ba1
2016-03-07 17:00:03 -08:00
Yue Chen
043b698a87 Merge "Calculate the distortion in pixel domain for sub8x8 rd selection" into nextgenv2 2016-03-08 00:13:46 +00:00
Yue Chen
ef8f7c1211 Calculate the distortion in pixel domain for sub8x8 rd selection
Pixel domain distortion calculation is enabled for the rd loop of
inter sub8x8 and intra 4x4 cases.

Coding gain: 0.124% derflr, 0.122% derfhd

Change-Id: I43b47fe81b4f5ccc1c66bc626bd310c413a1ed87
2016-03-07 14:49:22 -08:00
Sarah Parker
df3849370a Merge "Adding speed feature interface for ext tx search" into nextgenv2 2016-03-07 16:32:55 +00:00
Sarah Parker
2ca7d42e7e Adding speed feature interface for ext tx search
This sets up the interface for 3 speed features that progressively
eliminate a greater number of transforms in ext tx using
pre-trained support vector machines.
Each speed feature still needs to be implemented.

Change-Id: Ia508aeadc0cffdc080fb227f357a5d1dfbca08e2
2016-03-04 10:27:21 -08:00
Jingning Han
7174d637e8 Properly restore transform block skip flag in RD search
This commit fixes an encoding issue related to var-tx and ref-mv
experiments that causes the codec to use random values for transform
block skip flag.

Change-Id: I8daa6d6b88ea45b5bbeb81b43dd0eeff545c8e5a
2016-03-03 13:52:49 -08:00
Yaowu Xu
5c613ea881 Fix an overflow issue for HBD
The sum of squared value of a block can overflow 32bit, this commit
changes to use int64_t to avoid the overflow issue.

Change-Id: I78fcd6999634f186f86d649cfce85d97a993d040
2016-03-01 09:44:04 -08:00
Yunqing Wang
342a368fd4 Do sub-pixel motion search in up-sampled reference frames
Up-sampled the reference frames to 8 times in each dimension using
the 8-tap interpolation filter. In sub-pixel motion search, use the
up-sampled reference frames to find the best matching blocks. This
largely improved the motion search precision, and thus, improved
the compression quality. There was no change in decoder side.

Borg test and speed test results:
1. On derflr set,
Overall PSNR gain: 1.306%, and SSIM gain: 1.512%.
Average speed loss on derf set was 6.0%.
2. On stdhd set,
Overall PSNR gain: 0.754%, and SSIM gain: 0.814%.
On hevchd set,
Overall PSNR gain: 0.465%, and SSIM gain: 0.527%.
Speed loss on HD clips was 3.5%.

Change-Id: I300ebaafff57e88914f3dedc8784cb21d316b04f
2016-02-29 12:14:47 -08:00
Debargha Mukherjee
48589e8d07 Merge "Some refactoring and cleanups of interp filter" into nextgenv2 2016-02-29 15:55:48 +00:00
Debargha Mukherjee
bab2912b5e Some refactoring and cleanups of interp filter
Includes various cosmetic changes and refactoring including
naming the sharp filters differently (since they are no longer
8-tap).

Change-Id: Ida5a19ca0daa9f6a64a6734394c685b2a4a2564a
2016-02-26 15:42:49 -08:00
Geza Lore
7ded038af5 Port interintra experiment from nextgen.
The interintra experiment, which combines an inter prediction and an
inter prediction have been ported from the nextgen branch. The
experiment is merged into ext_inter, so there is no separate configure
option to enable it.

Change-Id: I0cc20cefd29e9b77ab7bbbb709abc11512320325
2016-02-26 13:01:51 -08:00
Debargha Mukherjee
3287f5519e Merge "Hooks to use 32x32 masked transforms for ext-tx" into nextgenv2 2016-02-26 20:54:37 +00:00
Debargha Mukherjee
da2d4a7afc Hooks to use 32x32 masked transforms for ext-tx
Adds hooks to use 32x32 ext-tx. Also adds scan orders for the masked
transforms for 32x32.
Make macro USE_MSKTX_FOR_32X32 1 in blockd.h to support 32x32 masked
transforms for ext-tx.

Change-Id: Ie6564830266651fcafae2d536c274dafd664ce17
2016-02-24 13:08:37 -08:00
Debargha Mukherjee
389efb289e Adds an utility macro ROUNDZ_POWER_OF_TWO
This macro works for the shift parameter being 0.
The ROUND_POWER_OF_TWO macro does not.

Change-Id: I8434d2933892e09bbc0d2dafc934d0c3637df347
2016-02-24 12:35:29 -08:00
Debargha Mukherjee
c1e51beba6 Merge "Experiment to use image domain dist in baseline." into nextgenv2 2016-02-24 18:30:50 +00:00
Yue Chen
02e734168c Merge "Optimizing obmc rd decision by checking the real rd cost" into nextgenv2 2016-02-23 23:05:06 +00:00
Yue Chen
a614262edb Optimizing obmc rd decision by checking the real rd cost
Instead of using model_rd_for_sb() to estimate the cost and make the
decision on bmc/obmc, we use super_block_yrd/uvrd() to calculate and
compare the real rd costs of bmc and obmc.

Average bit-rate reduction(%) of obmc experiment:
derflr/derfhd/hevcmr/hevchd
2.353/TBD/TBD/TBD
Before the optimization, the coding gain was:
1.582/1.109/1.600/1.164

Note: there is still some mysterious bug because that compared to
the previous version, the performance at low bit rate drops a lot.

Change-Id: I8dbee04a272190f10516a3953c1ae690f8136766
2016-02-23 14:16:12 -08:00
Geza Lore
3c4b56c4dd Experiment to use image domain dist in baseline.
Change-Id: Ib29f510289716b5ab5c7d74d32a450c190308a83
2016-02-23 09:35:40 -08:00
Jingning Han
a10814e11e Merge "Account context based prob model for motion vector cost estimate" into nextgenv2 2016-02-22 17:37:42 +00:00
Jingning Han
1f984a5a63 Merge "Vectorize motion vector probability models" into nextgenv2 2016-02-22 17:37:29 +00:00
Jingning Han
682dad0ec7 Merge "Store predicted motion vectors" into nextgenv2 2016-02-22 17:14:05 +00:00
Jingning Han
03c01bc3c0 Account context based prob model for motion vector cost estimate
This commit accounts for the context based probability model for
motion vector cost estimate in rate-distortion optimization.

Change-Id: Ia068a9395dcb4ecc348f128b17b8d24734660b83
2016-02-19 16:32:51 -08:00
Jingning Han
df59bb8986 Vectorize motion vector probability models
This commit converts the scalar motion vector probability model
into vector format for later precise estimate.

Change-Id: I7008d047ecc1b9577aa8442b4db2df312be869dc
2016-02-19 16:20:41 -08:00
Jingning Han
876c8b03e6 Store predicted motion vectors
Change-Id: I51307a217eeba14dbdaa2522be474530316a4faa
2016-02-19 14:25:34 -08:00
Yaowu Xu
5712456bd9 Merge "Properly normalize HBD sse computation" into nextgenv2 2016-02-19 02:26:47 +00:00
Yaowu Xu
0c0f3efdeb Properly normalize HBD sse computation
This fixes a bug in HBD sum of squared error computation introduced
in  #abd00505d1c658cc106bad51369197270a299f92.

Change-Id: I9d4e8627eb8ea491bac44794c40c7f1e6ba135dc
2016-02-18 15:42:19 -08:00
Debargha Mukherjee
9a019bce84 Merge "cost_coeff speed improvements" into nextgenv2 2016-02-18 19:31:18 +00:00
Julia Robson
c6eba0b47a cost_coeff speed improvements
Preliminary tests indicated that these changes make cost_coeffs
approximately 20% faster which is a 2% improvement overall

Change-Id: Iaf013ba75884415cd824e98349f654ffb1c3ef33
2016-02-18 13:18:39 +00:00
Jingning Han
95247be0bf Fix enc/dec mismatch in dynamic mv referenceing experiment
This commit fixes an enc/dec mismatch in the dynamic motion vector
referencing experiment introduced in 837ef00.

Change-Id: I9fbe116fce118a80ef0f96bf41ce1f802547c2ee
2016-02-17 09:29:54 -08:00
Yue Chen
907f88c4e6 Fixing a bug in obmc prediction in the rd loop
This bug made the rd loop use one-side obmc (compound of the current
predictor and the predictors of the left mi's, while the above ones
are ignored by mistake) to determine whether to use obmc. This fix
improved the compression performance by ~0.6% on different test sets.

Coding gain (%) of obmc experiment on derflr/derfhd/hevcmr/hevchd:
1.568/TBD/1.628/TBD

Change-Id: I43b239bedf9a8eebfd02315b1b036e140a998140
2016-02-16 14:43:45 -08:00
Hui Su
0107373234 Merge "Add a speed feature to skip transform type selection" into nextgenv2 2016-02-16 18:31:18 +00:00
Debargha Mukherjee
6f49446dfa Merge "Fix double counting of compound reference bit cost." into nextgenv2 2016-02-16 17:55:49 +00:00
Geza Lore
abd00505d1 Add optimized vpx_sum_squares_2d_i16 for vp10.
Using this we can eliminate large numbers of calls to predict intra,
and is also faster than most of the variance functions it replaces.
This is an equivalence transform so coding performance is unaffected.

Encoder speedup is approx 7% when var_tx, super_tx and ext_tx are all
enabled.

Change-Id: I0d4c83afc4a97a1826f3abd864bd68e41bb504fb
2016-02-15 16:54:52 +00:00
Yue Chen
d1cad9c3f5 Overlapped block motion compensation experiment
In this experiment, an obmc inter prediction mode is enabled for
>= 8X8 inter blocks. When the obmc flag is on, the regular block-
based motion compensation will be refined by using predictors of
the above and left blocks.
Fixed some compatibility issues with vp9_highbitdepth, supertx,
ref_mv, and ext_interp.

Coding gain (%) on derflr/hevcmr/hevchd
OBMC:
1.047/1.022/0.708
OBMC + SUPERTX:
1.652/1.616/1.137
SUPERTX:
0.862/0.779/0.630

Change-Id: I5d8d3c4729c6d3ccb03ec7034563107893103b7f
2016-02-12 13:36:25 -08:00
Alex Converse
a45d5d3f94 Merge "Port switch to 9-bit rate cost to vp10." into nextgenv2 2016-02-12 21:15:35 +00:00
Jingning Han
18eaf8e6fc Merge "Refactor vp10_drl_idx concept" into nextgenv2 2016-02-12 19:39:44 +00:00
Jingning Han
a39e83d743 Refactor vp10_drl_idx concept
Remove the implicit assumption on offsetting the index by 1.

Change-Id: I6f1d391e067d57b7e45b9287e866014dbc16da71
2016-02-11 16:38:13 -08:00
Debargha Mukherjee
c1924b9ff0 Merge "Complete high bitdepth VAR_TX implementation." into nextgenv2 2016-02-12 00:16:18 +00:00
Angie Chiang
368e3d9293 Merge "Refactor: add predict_interp_filter() to simplify the flow in handle_inter_mode" into nextgenv2 2016-02-12 00:16:13 +00:00
Jingning Han
57c83b330e Remove redundant parameters from vp10_txfm_rd_in_plane_supertx()
Change-Id: Icb164403239f88f18fd64de75d4881d33d3ab1cc
2016-02-11 11:53:22 -08:00
Jingning Han
5c772f38fa Format clean-ups in transform experiments
Change-Id: Ib2843cb03ae452ce9fec3a94c709431ea0202d8b
2016-02-11 11:07:00 -08:00
Alex Converse
b3ad81288f Port switch to 9-bit rate cost to vp10.
Brings the following commits to vp10:
269428e Tie the bit cost scale to a define.
d13385c Switch to 9-bit rate cost constants built on a 256 probability denominator.
ad43a73 Fix a signed overflow in vp9 motion cost.
1c9b091 Fix some interger overflow errors
fac947d Restore previous motion search bit-error scale.

Change-Id: I598ba7ee7efcde18439c31dfa96b86cbf297a580
2016-02-11 09:54:24 -08:00
Geza Lore
432e875dce Complete high bitdepth VAR_TX implementation.
VAR_TX now works in the high bitdepth configuration.

Change-Id: I4114d7d9ed59c598f1e4d35b8e75876c07074ba7
2016-02-11 10:49:56 +00:00
Hui Su
6779be2487 Merge "Refactor rd_pick_intra_angle_" into nextgenv2 2016-02-11 01:44:14 +00:00
hui su
5a7c8d8c1d Refactor rd_pick_intra_angle_
Change-Id: I6c78188bdedb52655678c63f6a767567b256a880
2016-02-10 15:41:04 -08:00
Angie Chiang
c0035cc480 Refactor: add predict_interp_filter() to
simplify the flow in handle_inter_mode

Change-Id: Ic7934c0a5d0a79bdf546b4d2d106035449b475a6
2016-02-10 15:32:10 -08:00