4774 Commits

Author SHA1 Message Date
Debargha Mukherjee
b6d5b0838b ext-tx extension to intra blocks
derflr: improves to 1.692%

Change-Id: Idf583216b40ed02526b9b39031eaf2fb72fed11d
2015-09-17 14:21:24 -07:00
Debargha Mukherjee
c104998b61 Merge "Redo DST1 in the ext-tx experiment" into nextgen 2015-09-16 18:38:03 +00:00
Debargha Mukherjee
4dbaf9a5ab Redo DST1 in the ext-tx experiment
Moved from nextgenv2 branch to test with other experiments.

derflr: +1.629%

Change-Id: Ie7c720053ed8b628177679c4351bb31b54716a71
2015-09-16 09:46:13 -07:00
Zoe Liu
9b0635fc75 Fixed a bug on the number of MAX_MODES in baseline
All the numbers of MAX_MODES have been changed assuming
CONFIG_MULTI_REF. Now correct numbers have been put in for both with and
without the enabling of the experiment MULTI_REF.

Change-Id: I70ffe2f1a89fa572d612dd3d311d3af19fe3a632
2015-09-15 14:12:12 -07:00
Zoe Liu
f48a159430 Added more LAST2 modes for INTERINTRA
Turning on all the other experiments, compared the RD performance
between with and without the use of LAST2_FRAME, on derflr testset,
on Average PSNR:

8-bit: +0.653% (All positive except one,
max: mobile_cif: 2.019%; min: paris_cif: -0.081%)
12-bit HBD: +0.735% (All positive,
max: bridge_far_cif: 2.416%; min: bowing_cif: 0.132%)

Change-Id: Ia0a375667e228c8ba3d2e223abff608206f2f545
2015-09-15 11:57:18 -07:00
Zoe Liu
ec8864a8bf Added MACRO for reference frame encoding
This CL introduces a few macros plus code cleaning on the encoding of
the reference frames. Coding performance remains unchanged.

For the encoding of either the compound reference or the single reference
case, since each bit has different contexts, the tree structure may not
be applied to treat the combined bits as one symbol. It is possible we may
explore the sharing of the same context for all the bits to introduce
the use of tree structure for the next step.

Change-Id: I6916ae53c66be1a0b23e6273811c0139515484df
2015-09-11 14:57:31 -07:00
Zoe Liu
897192be43 Added one more reference frame LAST2_FRAME
Under the experiment CONFIG_MULTI_REF. Current version shows
LAST2 vs base in nextgen on the testset of derflr:

(1) 8-bit: Average PSNR +0.53%
(worst: students_cif: -0.247%; best: mobile_cif: 1.902%)
(2) 12-bit HBD: Average PSNR +0.63%
(worst: pamphlet_cif: -0.213%, best: mobile_cif: 2.101%)

More tuning on the reference frame context design and default
probs are being conducted. This version does not guarantee to
work with other experiments in nextgen. A separate CL will address
the working with all other experiments.

Change-Id: I7f40d2522517afc26ca389c995bad56989587f65
2015-09-09 14:27:05 -07:00
Shunyao Li
2de18d1fd2 Super resolution mode (+CONFIG_SR_MODE)
CONFIG_SR_MODE=1, enable SR mode
USE_POST_F=1, enable SR post filter
SR_USE_MULTI_F=1, enable SR post filter family
Not compatible with other experiments yet

Change-Id: I116f1d898cc2ff7dd114d7379664304907afe0ec
2015-08-31 15:29:39 -07:00
Shunyao Li
c7d886d96a Add transform size rate for intra skip mode in rdopt
Change-Id: I81fedd99cd39c12b66b93b786cb43234c867b84b
2015-08-31 11:28:27 -07:00
hui su
d5eaca7fee code cleanup in encode_block_intra
Change-Id: I376b7e9b243178d79141a96e0aeafcbc15758e97
2015-07-30 15:28:51 -07:00
Cheng Chen
cc4d523d9f Resolve bug of DST1 in ext_tx experiment.
Change-Id: I828569e3596f9b9e8487aec7c4056e66cf1fc1f2
2015-07-28 10:58:14 -07:00
Debargha Mukherjee
23690fc5d1 Adds support for DST1 transforms for inter blocks
Adds an additional transform in the ext_tx experiment that
is a 2d DST1-DST1 combination.

To enable use --enable-ext-tx --enable-dst1.

This needs to be later extended to combine DST1 with DCT
or ADST.

Change-Id: I6d29f1b778ef8294bcfb6a512a78fc5eda20723b
2015-07-24 16:23:09 -07:00
Shunyao Li
188087202c Speed up of supertx
Limited the prediction extension to 8 pixels at each edge
Fixed a bug in the combination of wedge prediction and supertx

~10% speed up in decoder
derflr:     -0.004
derflr+hbd: +0.002
hevcmr:     +0.015

Change-Id: I777518896894a612c9704d3de0e7902bf498b0ea
2015-07-24 11:19:19 -07:00
Debargha Mukherjee
4b57a8b356 Add extended transforms for 32x32 and 64x64
Framework for alternate transforms for inter 32x32 and larger based
on dwt-dct hybrid is implemented.
Further experiments are to be condcuted with different
variations of hybrid dct/dwt or plain dwt, as well as super-resolution
mode.

Change-Id: I9a2bf49ba317e7668002cf1499211d7da6fa14ad
2015-07-23 18:01:22 -07:00
Yunqing Wang
7959dd012c Nextgen branch cleanup: remove elevate_newmv_thresh feature
Code cleanup.

Change-Id: Idf00d5359e5f3d943ee39b4a00f8d40325c0f8b3
2015-07-14 14:28:56 -07:00
Yunqing Wang
527f88316c Nextgen branch cleanup: remove real-time speed features
Removed inter_mode_mask and max_intra_bsize.

Change-Id: I835d31f95af64a524b2fdb685d1d09a6b0ad5c9f
2015-07-14 14:26:17 -07:00
Zoe Liu
08effeaad5 A small code cleanup for the encoder workflow.
Change-Id: I543a69ad5a34cd9a0671b57131cafc541d595d9a
2015-07-08 12:12:47 -07:00
Alex Converse
7932eb014e Merge "intrabc: Allow odd pel displacement for non-444" into nextgen 2015-07-08 04:06:24 +00:00
Yunqing Wang
ca42973ea2 Nextgen branch cleanup: remove svc code
Spatial/temporal svc code was removed.

Change-Id: Ie25c7a58ee5feb662d4de526406d8cd834d19977
2015-07-06 21:19:05 -07:00
Yunqing Wang
020293588d Nextgen branch cleanup: remove unused functions
Removed unused real-time functions.

Change-Id: I0cbcee67fb8f495f87c6330e04ed6d56ed8e2625
2015-07-06 12:07:33 -07:00
Yunqing Wang
cc6dc0b7d9 Nextgen branch cleanup: remove nonrd code
Code cleanup.

Change-Id: I4d8d7fa2fc61a58b819c9a18bf25cda53e3fd88c
2015-07-06 10:03:23 -07:00
Alex Converse
5241acf6e2 intrabc: Allow odd pel displacement for non-444
intrabc:
screen_content: -0.618 derflr: +0.015

intrabc+tx_skip+palette:
screen_content: -0.124 derflr: -0.048

Change-Id: Iabea4be19dce2f6fdab8f639e585a424c90c81b4
2015-06-30 10:11:46 -07:00
Alex Converse
a4f4ed60cf Merge "intrabc: displacement vector prediction" into nextgen 2015-06-25 20:55:21 +00:00
Alex Converse
93cd5cf3f4 intrabc: displacement vector prediction
Predict displacement vector with the same logic as NEARESTMV. If no
neighbors are available fall back to the current one block left or up
prediction.

vp9+intrabc+tx_skip+palette: -0.489
vp9+intrabc: -0.771

Change-Id: If67d08b54f1a3b847cf7ab8c7b800c55baa1a86b
2015-06-24 10:02:50 -07:00
Peter de Rivaz
fbfeb03f02 Fix for copy-mode with segmentation
Change-Id: Icfc97826e494b856aa80fbefe4811dff99f3a0b4
2015-06-24 09:25:23 +01:00
Debargha Mukherjee
567e4c754f Merge "Removing some compiler warnings" into nextgen 2015-06-23 19:13:09 +00:00
Peter de Rivaz
269f6e2cb1 Added support for extended partitions with copy_mode
Change-Id: I815374bd89e3faf2bac1b4155dbbe7366f0a1d29
2015-06-23 11:09:34 -07:00
hui su
2f1325049a Merge "Bug fix in tx-skip" into nextgen 2015-06-23 17:42:41 +00:00
Debargha Mukherjee
3a18fc6784 Removing some compiler warnings
Change-Id: I572594a9c00ba5a2b8000d3b4c86ec5757aa586c
2015-06-22 15:26:25 -07:00
Peter de Rivaz
bad321d9e4 Added palette support to extended partitions
Change-Id: Ide11bd3fcc9fb26b27ae39cdf216a2c32b853348
2015-06-22 20:34:26 +01:00
Julia Robson
84d0da63d0 Palette high bit depth functionality
Changes to allow high bit depth and palette to be enabled at the
same time by using a 16 bit (instead of 8bit) palette when high
bit depth is enabled and modifying related functions accordingly.

Change-Id: I97d30b4d9338d3a51db02c94bc568eba60a8905d
2015-06-22 18:53:34 +01:00
hui su
c96afa256f Merge "Add q-index as context for initial token probs" into nextgen 2015-06-22 17:44:26 +00:00
Debargha Mukherjee
adc9ed5a87 Merge "Fix supertx HORZ and VERT partitioning." into nextgen 2015-06-22 17:11:39 +00:00
hui su
315351e9de Bug fix in tx-skip
This patch avoids using tx_size larger than 16x16 in lossless mode.
Big block quantization (32x32 or larger) is not lossless.

Change-Id: I69cd84d4f3fd06d641048d6096da1bfde18ad24e
2015-06-22 09:54:42 -07:00
hui su
5963fd35dd Add q-index as context for initial token probs
There are 4 entropy tables to select for initial entropy table,
depending on the frame base q-index. The entropy tables are
trained with derf, yt, and stdhd sets. About 0.2% gain on
the following test sets:

derflr       0.227%
yt           0.277%
stdhd        0.233%
hevclr       0.221%
hevcmr       0.155%
hevchr       0.182%

Change-Id: I3fde846c47fc020e80c814897690b4cda1da569c

Change-Id: I460408372586c823974f945ed9fd8dcb0360fbaf
2015-06-21 13:09:57 -07:00
Peter de Rivaz
bdbae2520a Fix supertx HORZ and VERT partitioning.
Change-Id: I9a44d70dc2514baf679025335badc2744eeb4b88
2015-06-17 22:02:58 -07:00
Peter de Rivaz
100fff09b6 Added ext_partitions to supertx
Change-Id: I0e2a0253e5a8dcdb5b765836a3301299409e9d0a
2015-06-17 20:28:32 +00:00
Peter de Rivaz
3a3fb8d100 Fix seg fault in count_segs_sb with ext_partitions
Change-Id: I609f429006e7d2b849fe0a509f5d9f5123ecf0ec
2015-06-17 08:25:58 +00:00
Pieter Kapsenberg
d0e9499f94 Fixing skip simplifcation with SuperTX experiment
Combination of MISC_ENTROPY and SUPERTX resulted in compilation error

Change-Id: Ibcf28216712d93f00f38f3786fce8fcdca3f835e
2015-06-15 16:50:10 -07:00
Alex Converse
a046bece1e Merge changes I9d80152b,I4e13da4b into nextgen
* changes:
  intrabc: Add a frame level flag
  intrabc: Remove NEWDV from the intra mode tree.
2015-06-15 16:57:05 +00:00
Debargha Mukherjee
dcb3d7f398 Bug fix in highbd psnr computation with postproc
Change-Id: I6944a5fc38d6a89e1bb739938c6ecae7ec731b8d
2015-06-12 11:57:40 -07:00
Peter de Rivaz
3bf31c4c98 Allow extended partition support to encode
Change-Id: I75246e2ee35a1b7c1ad46669c464e582e3a9961c
2015-06-11 11:52:45 -07:00
Peter de Rivaz
31d17053f3 Add extended partition support to decode
Change-Id: I65f7aed37d65dbba76999d2b9585129c48e70b11
2015-06-11 09:37:00 -07:00
Debargha Mukherjee
db90b74c1a Merge "Fix copy mode bug: searches made outside of the current tile" into nextgen 2015-06-11 16:30:23 +00:00
Jack Haughton
286983b8c6 Fix copy mode bug: searches made outside of the current tile
Change-Id: Ib3cf4bf4da0a575053493609956e133d1ce028a1
2015-06-11 09:23:32 +01:00
Debargha Mukherjee
278de7b73a Merge "Fix cost array overrun bug in search_bilateral_level()" into nextgen 2015-06-11 05:28:43 +00:00
Alex Converse
c3e884f5be intrabc: Add a frame level flag
For the combination of this and removing NEWDV from the tree:
derflr: -0.101 screen_content: +0.053

The bulk of the decline in screen content effecincy is from the liquify
clip. These should be recoverable by further entropy tweaks.

Change-Id: I9d80152b8492e60a0367c31797fb6932fb09bba9
2015-06-09 11:23:23 -07:00
Debargha Mukherjee
36eec6a9b1 Merge "Fix masked_variance overflow for non-highbitdepth" into nextgen 2015-06-09 17:07:13 +00:00
Jack Haughton
fa4b1ca714 Fix cost array overrun bug in search_bilateral_level()
Change-Id: I9b7b727cb45328040a82477a2ce8eb3d15feec77
2015-06-09 14:21:55 +01:00
Debargha Mukherjee
b433dd4443 Adds wavelet transforms + hybrid dct/dwt variants
The wavelets implemented are 2/6, 5/3 and 9/7 each with
a lifting based scheme for even block sizes. The 9/7
one is a double implementation currently.

This is to start experiments with:
1. Replacing large transforms (32x32 and 64x64) with wavelets
or wavelet-dct hybrids that can hopefully localize errors better
spatially. (Will also need alternate entropy coder)
2. Super-resolution modes where the higher sub-bands may be
selectively skipped from being conveyed, while a smart
reconstruction recovers the lost frequencies.

The current patch includes two types of 32x32 and 64x64
transforms: one where only wavelets are used, and another
where a single level wavelet decomposition is followed
by a lower resolution dct on the low-low band.

Change-Id: I2d6755c4e6c8ec9386a04633dacbe0de3b0043ec
2015-06-08 23:30:38 -07:00