43 Commits

Author SHA1 Message Date
Debargha Mukherjee
f28ea3e830 Explicitly set tx_type for sub8x8 blocks
Fixes an issue where the tx_type was not set correctly for
sub8x8 inter and intra blocks. In the current syntax, for
sub8x8 blocks, there is still a single tx_type that is
transmitted. Ideally, this should be searched for the best
rd performance, albeit at the expense of encode speed.
For now, we just set it to DCT_DCT. Previously it was left
incorrectly as what was used for the previous non sub8x8
block.

derflr: BDRATE -0.277%

Change-Id: If76ba903bfbfd4d374cf1ac7d1daee50e92f0edd
2016-02-08 16:11:53 -08:00
James Zern
ebf258688e vp10: remove unused (read|write)_uniform
dead code since:
5d3327e Remove palette from VP10

Change-Id: I4a36575706ea6fffefe5bc778595112ef6ff37d8
2016-02-01 23:47:35 -08:00
Debargha Mukherjee
a0900fd0db Remove experimental flag for ext_tx
Also includes a bug fix.

Change-Id: Ia49ed00f8ffd1531c10bcf89b1f497310ee7cb82
2016-01-08 13:48:24 -08:00
Debargha Mukherjee
f7dfa4ece7 Modifies inter/intra coding to allow all tx types
The nominal tx_type for a given mode is used as a context
to encode the actual tx_type for intra.

Results:
derflr: -0.241% BDRATE
hevcmr: -0.366% BDRATE

Change-Id: Icfe7b0a58d79bc6497a06e3441779afec6e01e21
2016-01-08 11:13:46 -08:00
Yaowu Xu
9232f69b26 Merge "Fix a enc/dec mismatch under CONFIG_MISC_FIXES" 2015-12-15 16:02:39 +00:00
Yaowu Xu
c7101830a6 Fix a enc/dec mismatch under CONFIG_MISC_FIXES
The culprit is on the decode side xd->lossless[i] setup was in wrong
location where segment features are not yet decoded.

Also on the encoder side, transform mode was not set consistently
between when tx_mode is selected and how tx_mode is enforced in
tx size selection.

Change-Id: I4c4c32188fda7530cadab9b46d4201f33f7ceca3
2015-12-14 20:56:37 -08:00
paulwilkins
4e692bbee2 Changes to exhaustive motion search.
This change has been imported from VP9 and
alters the nature and use of exhaustive motion search.

Firstly any exhaustive search is preceded by a normal step search.
The exhaustive search is only carried out if the distortion resulting
from the step search is above a threshold value.

Secondly the simple +/- 64 exhaustive search is replaced by a
multi stage mesh based search where each stage has a range
and step/interval size. Subsequent stages use the best position from
the previous stage as the center of the search but use a reduced range
and interval size.

For example:
  stage 1: Range +/- 64 interval 4
  stage 2: Range +/- 32 interval 2
  stage 3: Range +/- 15 interval 1

This process, especially when it follows on from a normal step
search, has shown itself to be almost as effective as a full range
exhaustive search with step 1 but greatly lowers the computational
complexity such that it can be used in some cases for speeds 0-2.

This patch also removes a double exhaustive search for sub 8x8 blocks
which also contained  a bug (the two searches used different distortion
metrics).

For best quality in my test animation sequence this patch has almost
no impact on quality but improves encode speed by more than 5X.

Restricted use in good quality speeds 0-2 yields significant quality gains
on the animation test of 0.2 - 0.5 db with only a small impact on encode
speed. On most natural video clips, however, where the step search
is performing well, the quality gain and speed impact are small.

Change-Id: Iac24152ae239f42a246f39ee5f00fe62d193cb98
2015-12-08 16:54:42 +00:00
hui su
5d3327e891 Remove palette from VP10
Store it in nextgenv2 for now.

Change-Id: Iab0af0e15246758e3b6e8bde4a74b13c410576fc
2015-12-03 12:30:47 -08:00
Alex Converse
b1fcd1751e Fix unsigned overflow in rd_variance_adjustment.
Found with clang -fsanitize=integer

Change-Id: I2538e7483cb2d5f06bceecbd3326bdd88bfecfa1
2015-11-19 15:00:59 -08:00
hui su
6ab6ac450b Use accurate bit cost for uv_mode in UV intra mode RD selection
On derflr, +0.1% for VP10; however, -0.03% on VP9.

Change-Id: I09c724232ede74254043d61d3cadc506256af0af
2015-11-06 14:45:43 -08:00
Jingning Han
caeb10bf06 Use explicit block position in foreach_transformed_block
Add the row and column index to the argument list of unit functions
called by foreach_transformed_block wrapper. This avoids the
repeated internal parsing according to the block index.

Change-Id: Ie7508acdac0b498487564639bc5cc6378a8a0df7
2015-10-23 09:19:17 -07:00
Hui Su
96b69deca5 Merge "VP10: some changes to palette mode" 2015-10-20 16:37:31 +00:00
Ronald S. Bultje
60c58b5284 vp10: per-segment lossless coding.
Some more testing of this patch would probably be useful, but I
think the basics of it should work fine now.

See issue 1035.

Change-Id: I4a36d58f671c5391cb09d564581784a00ed26245
2015-10-16 19:30:39 -04:00
Ronald S. Bultje
c7dc1d78bf vp10: add extended-intra prediction edges experiment.
This experiment allows using full above/right edges for all transform
sizes whenever available (for d45/d63), and adds bottom/left edges for
d207.

See issue 1043.

Change-Id: I5cf7f345e783e8539bb6b6d2c9972fb1d6d0a78b
2015-10-16 19:30:39 -04:00
Ronald S. Bultje
d8f3bb1837 vp10: merge keyframe/interframe uvintramode/partition probabilities.
This has various benefits:
- simplify implementations because we don't have to switch between
  multiple probability tables depending on frametype
- allows fw subexp and bw adaptivity for partitions/uvmode in keyframes

See issue 1040 point 5.

Change-Id: Ia566aa2863252d130cee9deedcf123bb2a0d3765
2015-10-16 19:30:38 -04:00
hui su
17c817adfc VP10: some changes to palette mode
Account for rounding in distortion calculation in k-means;
carry out rounding before duplicates removal of base colors;
replace numbers with macros;
use prefix increment.

Slight coding gain (<0.1%) on screen_content testset.

Change-Id: Ie8bd241266da6b82c7b2874befc3a0c72b4fcd8c
2015-10-16 11:41:26 -07:00
hui su
aaf6f6215f Fix palette mode in multi-thread encoding setting
Fix a couple of memory related errors. Also fix thread test failures.

Change-Id: I0103995f832cecf1dd2380000321ac7204f0cfc0
2015-10-15 15:00:57 -07:00
Yaowu Xu
15cc8bc72f Merge "fix a msvc compiler warning" 2015-10-15 14:39:01 +00:00
Yaowu Xu
8ced62f250 fix a msvc compiler warning
Change-Id: Ifd6581c1bdb8d8f4b2ecf676c1a3d385dc129abf
2015-10-15 01:05:13 +00:00
Yaowu Xu
4727fa2a75 Fix two asan failures
Change-Id: I57865e9604ac162ef0d97deb16e81ca436a98428
2015-10-14 18:03:31 -07:00
Hui Su
b9e31b5163 Merge "VP10: Add palette mode part 1" 2015-10-13 17:34:27 +00:00
Ronald S. Bultje
5b4805d6e9 vp10: remove clamp_mv2() call from vp10_find_best_ref_mvs().
This actually has no effect whatsoever, since the input MVs themselves
are clamped by clamp_mv_ref() already, which is significantly more
restrictive in its bounds.

Change-Id: I4a3a7b2b121ee422c56428c2a12d930c3813c06e
2015-10-12 14:45:18 -04:00
hui su
5d011cb278 VP10: Add palette mode part 1
Add palette mode for keyframe luma channel. Palette mode is enabled
when using "--tune-content=screen" in encoding config parameters.

on screen_content testset:  +6.89%
on derlr                 :  +0.00%

Design doc (WIP):
https://goo.gl/lD4yJw

Change-Id: Ib368b216bfd3ea21c6c27436934ad87afdaa6f88
2015-10-12 10:02:17 -07:00
Ronald S. Bultje
bab8d38f7f vp10: remove MACROBLOCK.{highbd_,}itxfm_add function pointer.
This is preparatory work for allowing per-segment lossless coding.

See issue 1035.

Change-Id: I9487d02717ee3e766aee61a487780056bb35d2d3
2015-09-25 19:30:46 -04:00
Ronald S. Bultje
c74b33a413 vp10: remove MACROBLOCK.fwd_txm4x4 function pointer.
This is preparatory work for allowing per-segment lossless coding.

See issue 1035.

Change-Id: Idd72e2a42d90fa7319c10122032d1a7c7a54dc05
2015-09-25 19:30:46 -04:00
hui su
38cc168822 Adjust rd calculation in choose_tx_size_from_rd
Coding gain:
derflr 0.142%
hevclr 0.153%
hevcmr 0.124%

Change-Id: I63b56ae3a9002c3a266e10e2964135ed43b0ba53
2015-09-23 10:54:28 -07:00
Jingning Han
d6be2671ed Create sub8x8 block inter prediction function
Change-Id: Ib161e6fb3eb081f7176a1d969fed16a7d1ffc320
2015-09-18 16:31:36 -07:00
Jingning Han
481b834842 Fix vp10 high bit-depth build
Change-Id: Ie3daed0b282b43ef81d2f8797ac1f6e8bde7d65e
2015-09-11 08:56:29 -07:00
Jingning Han
1eb760e55d Take out reference_masking speed feature
This condition is not effectively in use. The actual reference
frame masking is done in other route.

Change-Id: Ia59c843bcac7243dada92f0f67658d7ce43df5e8
2015-09-10 12:57:48 -07:00
Jingning Han
f137697c32 Take out skip_encode speed feature in vp10
Change-Id: Ic39d4523e78863c816b0fc85f56ea5ae5e0b3310
2015-09-10 12:45:39 -07:00
Jingning Han
4fa8e73249 Remove speed features in vp10
Take out speed features that affect the compression performance
to simplify the coding route. This commit removes the motion field
mode search used in speed 3.

Change-Id: Ifdf6862cb1ece8261125a56d9d89bcef60758c00
2015-09-10 12:25:33 -07:00
Johann
c5f11912ae Include vpx_dsp_common.h when using VPXMIN/MAX
Change-Id: I2e387a06484a06301f3cd6600c4ba2f4335b61ee
2015-08-31 14:36:35 -07:00
James Zern
5e16d397bd vpx_dsp_common: add VPX prefix to MIN/MAX
prevents redeclaration warnings;
vp8 has its own define which will be resolved in a future commit

Change-Id: Ic941fef3dd4262fcdce48b73075fe6b375f11c9c
2015-08-26 20:11:32 -07:00
Yunqing Wang
1c0769107b vp10 cleanup: remove nonrd and real-time code
Code cleanup.

Change-Id: I668bd5a4e5fbe96969d51d9ee777fc5f1d8abfe6
2015-08-26 13:36:54 -07:00
Shunyao Li
aa006d7149 Add transform size rate for intra skip mode in rdopt
stdhd +0.226 hevchr +0.091 hevcmr +0.052 derflr +0.033

Change-Id: I84034209c5760609a99bd6e0ce55e02534b72cac
2015-08-24 18:15:09 -07:00
hui su
d76e5b3652 Refactoring on transform types
Prepare for adding more transform varieties (EXT_TX and TX_SKIP in nextgen).

Change-Id: I2dfe024f6be7a92078775917092ed62abc2e7d1e
2015-08-24 10:47:25 -07:00
hui su
5eed74e1d3 Refactor get_tx_type and get_scan
This makes it easier to add new transform types and scan orders
to VP10 in the future.

Change-Id: I94874ddc9b19928d7820d57e94e2af04adf51efe
2015-08-21 09:53:37 -07:00
Yaowu Xu
26a9afc32c VP9_COMP -> VP10_COMP
Change-Id: I83b5c69621f9f28b742e5b13517d4e5b99c6cd26
2015-08-13 11:37:17 -07:00
Yaowu Xu
fc7cbd1f60 VP9_COMMON -> VP10->COMMON
Change-Id: I651b7bee90f33581368853da81f9622805ccc0ea
2015-08-13 09:58:26 -07:00
Hui Su
54fb24aac3 Merge "Use sizeof(variable) instead of sizeof(type)" 2015-08-12 19:49:22 +00:00
hui su
088b05fd99 Use sizeof(variable) instead of sizeof(type)
Change-Id: Ia069da11eebb271063e9eb837bdb3e7175ecce13
2015-08-12 11:25:38 -07:00
Jingning Han
3acfe46e8d Sync vp10 with vpx_ports/system_state.h
Change-Id: Ic5004f8bdc1c2b025b598e80374ee1f286ea95ee
2015-08-12 09:21:25 -07:00
Jingning Han
54d66ef165 Remove vp9_ prefix from vp10 files
Remove the vp9_ prefix from vp10 file names.

Change-Id: I513a211b286a57d6126fc1b0fbfd6405120014f1
2015-08-11 21:24:08 -07:00