Commit Graph

411 Commits

Author SHA1 Message Date
Yaowu Xu
5adb43b8be Fix non-highbitdepth coding path for HBD build
Change-Id: I38eb42b8d051924a7cd1ccc3421a4057cf6e170f
2016-07-08 11:26:34 -07:00
Yaowu Xu
dc008cc17d Merge "Enable HBD support in real time encoding path" 2016-07-07 22:32:48 +00:00
Marco
f451b404ea vp9: Adjustment to mv bias for non-rd pickmode.
Replace the existing mv bias with a bias only for
NEWMV, and based on the motion vector difference of
its top/left neighbors.

For cbr non-screen-content mode.

Change-Id: I8a8cf56347cfa23e9ffd8ead69eec8746c8f9e09
2016-07-07 10:33:06 -07:00
Yaowu Xu
884c2ddc48 Enable HBD support in real time encoding path
BUG=webm:1223

Change-Id: If83a613784e3b2a33c9c93f9ad0ba39dd4d23056
2016-07-06 14:18:37 -07:00
JackyChen
2678aefc48 vp9: Choose the scheme for modeling rd for 32x32 based on skin color.
For real time CBR mode, use model_rd_for_sb_y for 32x32 if the sb is
a skin sb to avoid visual regression on the slowly moving face.

Refer to the cl: https://chromium-review.googlesource.com/#/c/356020/

Change-Id: I42c36666b2b474ce5ee274239d52ae8ab400fd46
2016-07-06 11:12:03 -07:00
Jingning Han
51aad61c8c Merge "Remove txfrm_block_to_raster_xy() from vp9 encoder" 2016-07-06 16:00:18 +00:00
Jingning Han
14011f037d Remove txfrm_block_to_raster_xy() from vp9 encoder
The transform block row and column positions are always available
outside the callees. There is no need to re-compute these values
again. This approach has been used by the decoder. This commit
removes txfrm_block_to_raster_xy() function.

Change-Id: I5b90f91a0d8b7c35cfa7d171da9edf8202630108
2016-07-04 18:41:47 -07:00
JackyChen
5fc2d6cb9f vp9: Change the scheme for modeling rd for 32x32 on newmv_last mode.
For real time CBR mode, use model_rd_for_sb_y for 32x32 if the mode is
newmv last, which is less aggressive in skipping transform and
quantization, to avoid quality regression in some conditions.

Change-Id: Ifa30be587f2a8a4a7f182a172de6ce277c0f8556
2016-06-29 16:28:15 -07:00
Yaowu Xu
b9ec759bc2 Fix ubsan warnings: vp9/encoder/vp9_pickmode.c
This commit fixes a number of integer out of range issue in HBD build.

BUG=webm:1219

Change-Id: Ib4192dc74a500e1b86c37a399114c7f6d4ed5185
2016-06-27 05:53:46 +00:00
James Zern
1c0a9f36f1 vp9_pickmode: revert rd modeling change for hbd
Avoids a segfault in high-bitdepth builds.
This restores the condition to its state prior to:
7991241 vp9: Change the scheme for modeling rd for bsize 32x32.

BUG=webm:1250

Change-Id: I6183d5b34cb89dfbf27b7bb589812148a72cd7de
2016-06-25 11:40:26 -07:00
jackychen
7991241a50 vp9: Change the scheme for modeling rd for bsize 32x32.
For real-time CBR mode, use model_rd_for_sb_y_large instead of
model_rd_for_sb_y for 32x32 block. In the former model, transform
might be skipped more aggressively in some condtions, which speeds
up encoding time with only a little PSNR/SSIM drop on rtc test set.
No obvious visual quality regression.

PSNR effect on different speed settings:
speed 8 rtc:  0.129% overall PSNR drop, 0.137% SSIM drop
speed 7 rtc:  0.135% overall PSNR drop, 0.062% SSIM drop
speed 5 rtc_derf: 0.105% overall PSNR drop, 0.095% SSIM drop

Speed up:
gips_motion_WHD, 1mbps: 3.29% faster on speed 7, 2.56% faster on speed8
gips_stat_WHD, 1mbps: 2.17% faster on speed 7, 1.62% faster on speed8

BUG=webm:1250

Change-Id: I818babce5b8549b4b1a7c3978df8591bffde7173
2016-06-24 12:09:13 -07:00
James Zern
d4596485be Revert "vp9: Change the scheme for modeling rd for bsize 32x32."
This reverts commit 5c29ee726e.

Causes segfaults in VP9/EndToEndTestLarge.EndtoEndPSNRTest.

BUG=webm:1250

Change-Id: I8a30e97be30589abdb76820b5c3c37c46cd6cafb
2016-06-23 15:59:25 -07:00
Angie Chiang
424982bc41 Merge "set interp_filter to SWITCHABLE_FILTER for intra block" 2016-06-23 18:56:27 +00:00
Angie Chiang
d9c417cb49 set interp_filter to SWITCHABLE_FILTER for intra block
In vp9_pick_inter_mode(), instead of using
vp9_get_pred_context_switchable_interp(xd) to assign filter_ref,
we use a less strict condition on assigning filter_ref.
This is to reduce the probabily of entering the flow of not
assigning filter_ref and then skipping filter search.

Overall PSNR gain 0.074% for rtc dataset

Details:
Low    Mid     High
0.185% -0.008% -0.082%

Change-Id: Id5c5ab38d3766c213d5681e17b4d1afd1529e676
2016-06-22 17:19:43 -07:00
Jacky Chen
8496390e73 Merge "vp9: Change the scheme for modeling rd for bsize 32x32." 2016-06-22 23:50:46 +00:00
jackychen
5c29ee726e vp9: Change the scheme for modeling rd for bsize 32x32.
For real-time CBR mode, use model_rd_for_sb_y_large instead of
model_rd_for_sb_y for 32x32 block. In the former model, transform
might be skipped more aggressively in some condtions, which speeds
up encoding time with only a little PSNR/SSIM drop on rtc test set.
No obvious visual quality regression.

PSNR effect on different speed setting:
speed 8 rtc:  0.129% overall PSNR drop, 0.137% SSIM drop
speed 7 rtc:  0.135% overall PSNR drop, 0.062% SSIM drop
speed 5 rtc_derf: 0.105% overall PSNR drop, 0.095% SSIM drop

Speed up:
gips_motion_WHD, 1mbps: 3.29% faster on speed 7, 2.56% faster on speed8
gips_stat_WHD, 1mbps: 2.17% faster on speed 7, 1.62% faster on speed8

Change-Id: I902f62def225ea01c145d7e5a93497398b8f5edf
2016-06-22 11:17:56 -07:00
Yaowu Xu
87bf1a149c Fix ubsan warnings: vp9/encoder/vp9_mcomp.c
This commit fixes a number of ubsan warnings in HBD build.

BUG=webm:1219

Change-Id: I05f0fd0ef50e93db4ba34205005c54af1ed32acc
2016-06-21 15:37:59 -07:00
James Zern
cf2bfcd9d8 Merge "vp9_pickmode: quiet int64->int conversion warning" 2016-06-17 03:58:46 +00:00
Marco Paniconi
8870756ac6 Merge "vp9: Adjustments to nonrd-pickmode for vbr" 2016-06-16 16:07:06 +00:00
James Zern
ff8edd3509 vp9_pickmode: quiet int64->int conversion warning
since:
7901a05 VP9: block_yrd(): Use RD_COST * instead

Change-Id: I808be7a7bc96b81eb7d6255ae96cea7e9b1ecf1f
2016-06-15 23:34:09 -07:00
Marco
8e070558b0 vp9: Adjustments to nonrd-pickmode for vbr
For VBR: (1) allow newmv mode for golden ref to
select interpolation filter (as in last ref case), and
(2) don't use the more aggressive tx-skip testing logic for large blocks.

Only affects 1 pass real-time vbr mode (speed >= 5).

PSNR/SSIM metrics on ytlive set are all positive, ~0.5-2% gain.

Change-Id: I0ffbb0a9755563a5acd6230c58236e4f19a47266
2016-06-15 14:31:28 -07:00
Scott LaVarnway
55d0ffeb54 Merge "VP9: block_yrd(): Use RD_COST * instead" 2016-06-15 21:02:45 +00:00
Jacky Chen
9c90830165 Merge "vp9: Code clean up for short circuit feature in low temp variance." 2016-06-15 16:49:32 +00:00
Scott LaVarnway
7901a051c9 VP9: block_yrd(): Use RD_COST * instead
of int *rate, int *dist.

Change-Id: I7cbaea5252615b6b45b33e896f637cf7d076588a
2016-06-15 05:39:00 -07:00
Scott LaVarnway
221fcdac85 Merge "VP9: minor block_yrd() refactoring" 2016-06-15 11:46:54 +00:00
JackyChen
17962ab2c0 vp9: Code clean up for short circuit feature in low temp variance.
Change-Id: I7573a5cf0ab79abed8d124019c0ed6d9531277f9
2016-06-14 16:53:13 -07:00
JackyChen
f9c0587200 vp9: Encoding cycle reduction for speed 8.
1. Skip golden non-zeromv and newmv-last for bsize >= 16x16 if the
temporal variance obtained from choose_partitioning is very low.
2. Skip horz and vert INTRA mode for speed 8.

This change works best on the clips with little noise and with some
motion (e.g. gips_motion which has > 5% speed up). PSNR drop is 1.78%
on rtc test set, no obvious visual quality regression found.

Change-Id: Ib43b5b20e67809d03c5a6890818ddff59e1fc94a
2016-06-13 09:33:22 -07:00
Scott LaVarnway
580317160c VP9: minor block_yrd() refactoring
Change-Id: Ic0a81fb5816c8252ca4876753320fde5c28d893f
2016-06-13 07:35:45 -07:00
jackychen
bacc67f4a8 vp9: Skip some modes when variance is low for big blocks, for 1 pass real-time.
Skip intra-mode and some inter-modes (newmv, nearmv, nearestmv) for
golden frame if the variance got from choose_partitioning is very low.
Only for 1 pass real-time CBR mode and bsize >= 32x32, it has ~2.5%
speed up with less than 0.1% PSNR drop for rtc test set. Don't see
visual regression.

Change-Id: I70efbc95a1007231ae36f02c5b2fbf6cd35077ad
2016-06-01 13:54:18 -07:00
Marco
bedf1c3af6 vp9: Skip computation of best_sad for newmv, unless needed.
For non-rd pickmode:
best_pred_sad, computed for NEWMV-last, is only used for
skipping golden non-zero modes. Add condition to avoid this
computation if not used (i.e, if golden nonzero modes are not used).

And remove code for computing best_pred_sad for NEWMV-golden,
since that sad is not used.

No change in behavior; small speed gain (~1%) for svc encodes.

Change-Id: Ic2cbdef6c4e9a233a57c0db0eeac8ad5fcead366
2016-05-31 10:29:00 -07:00
jackychen
6f397b8a5b vp9: Remove a redundent condition in sub-pixel filter choosing.
Change-Id: I5cbb0f452ec9622437482b3a9496ead1253acfe0
2016-05-20 14:38:45 -07:00
JackyChen
af0a9fcf70 vp9: Refactor some denoiser logic in vp9_pick_inter_mode.
Move the logic for rechecking zeromv on denoised block out to simplify
the function. To simplify the param passing, add a new structure
VP9_PICKMODE_CTX_DEN which is only used when denoiser is enabled.

Change-Id: Iaa9b4396dfcb8147236c02d4a1868a09103a4476
2016-05-19 15:28:25 -07:00
JackyChen
0dfa6b38dc Move non-zero mv bias on large block out of vp9_pick_inter_mode.
This is to simplify pick inter mode funtion and is bitexact.

Change-Id: Ib1a115bac5fbd212c626be194bcd4a6ce4af9bf2
2016-05-13 15:29:05 -07:00
Alex Converse
d8a18186ba pickmode: Fix a pair of unsigned overflows.
block_variance: This operates on 8x8s and would be safe with a int32 *
int32 to uint32 multiply, but this is potentially unsafe for 12-bit
input. Unfortunately the code already segfaults on 12-bit input:
https://bugs.chromium.org/p/webm/issues/detail?id=1223

calculate_variance: This operates on up to a 32x32 of 8x8s and can
overflow even with 8-bit input (log2((256*32*32)**2) == 36).

BUG=https://bugs.chromium.org/p/webm/issues/detail?id=1220

Change-Id: I1ca4ff6092db9a7580da371ee9a21f403fdadc40
2016-05-09 11:21:20 -07:00
James Zern
9bfa12764c vp9_pickmode.c: make function static
set_intra_cost_penalty() is local to this module

Change-Id: Ie277bfc5e8a26f05a04fd261bdb888388ad0e2a1
2016-05-04 15:10:00 -07:00
JackyChen
c69a141a9e vp9: Don't force block size to 8x8 in denoiser.
First, we only set use_4x4_partition for key frame where we don't
denoise; second, envision we have small partitions, we should pass the
actual block size to denoiser and make an early termination if needed.

Change-Id: I331f42046d792b17360723d17ff817d601394658
2016-04-27 22:18:11 -07:00
Alex Converse
a3a10a323b pickmode: only cost the skip flag once per prediction block
RTC speed 6:
File    Match   Avg     BDRate  Low     Mid     High
OVERALL ✔       -0.040  -0.045  -0.031  -0.084  0.004

Screencast speed 6:
File     Match   Avg     BDRate  Low     Mid     High
OVERALL ✔       1.115   -0.162  0.203   2.470   0.541

Change-Id: I46bbc11c89301015b5d3eac25294c709f23f0897
2016-04-11 15:39:52 -07:00
Marco
ac86dde3af vp9-denoiser: Avoid copy-block when denoising is at LowLow level.
Avoid copy-block when denoising is at LowLow level (i.e., no denoising is done).
Instead, don't enter denoiser at all, and when level goes back up over kLowLow
do a reset in denoiser.

Change-Id: I0544adf58f4dd51ecc4a4607fcb0353bfbbb7a59
2016-04-04 17:02:24 -07:00
Scott LaVarnway
ba962a5f37 VP9: Eliminate up_available and left_available
Use above_mi and left_mi instead.

Change-Id: I0b50e232c31d11da30aa2fb6f91a695aaf725e0c
2016-03-30 04:47:39 -07:00
Marco
48e7294e48 vp9-denoiser: Incorporate consec_zeromv in skin detection for denoising.
Change-Id: Ibe4e1207c0db1779e1358f4566da67c9a07bdb15
2016-03-28 10:57:54 -07:00
Marco
f03dfcb118 vp9-svc: Bugfix on usage of svc_force_zeromv in non-rd pickmode.
Only affects 1 pass cbr svc mode.

Change-Id: Ifc0948b4aabffa64a9701295d8dbc58a7208a30f
2016-03-16 16:12:16 -07:00
Marco
caa8de547f vp9 non-rd pickmode. Some adjustments for vbr mode.
Some adjustments to inter-mode selection for vbr mode.
Condition some of the bias to low/zero motion on cbr mode, and
don't use int_pro_motion_estimation for golden ref
(treat it same as last ref).

Change only affect 1 pass vbr mode, speed >=5 (non-rd pickmode).

Encoding time increase within ~5%.
Avg PSNR/SSIM on RTC set increase by ~2%, all clips up,
ranging from 0.5 to 4%.

Change-Id: I0048d0104a8816773d91a2b1484d601169d9bad7
2016-03-11 09:22:36 -08:00
Marco
2b5ab8095e vp9-svc: Non-rd pickmode, fix condition for testing zeromv for spatial ref.
Change-Id: I75bad321d5814a6df6735a8ead7700ab93b50763
2016-03-07 09:55:25 -08:00
Alex Converse
def519efb3 Fix an uninitialized warning.
Change-Id: I3ac1070df2948f6d8637ade7192400be53a7675c
2016-02-05 12:06:38 -08:00
Marco
1c2df9cd48 vp9: encoder-breakout: Modify UV thresholds.
Set UV thresholds to 0 for encoder breakout if
block is detected as skin.

Change-Id: Iaaa30b5cc8bae4cf611f8ea8eba693b730e32a3f
2016-01-28 12:12:40 -08:00
Marco
b39a599cef vp9 non-rd mode: Modification for detected skin areas.
If a superblock contains alot of "skin" then force split
of 64x64 partition, and make some adjustments in mode selection.

This helps to reduce artifacts on moving face/skin areas at low bitrates.

Little/no change in metrics: avgPSNR/SSIM down by ~0.12%.
Small encoding time increase < 1%.

Change-Id: Ic57f52148c3716f391419fab0530d916e4c1d186
2016-01-27 17:38:58 -08:00
Alex Converse
7da6324cab Short circuit flat blocks when coding screen content at realtime speed.
In inter mode search skip all modes except NEARESTMV and DC_PRED.

10% less encode latency for large frames using the chromium remoting_perftests.

+0.313% BDRATE on the screencast set at speed -6.

Change-Id: Ib97a39dd8bcdeab545509e0e02d78ce7033f8c63
2016-01-22 12:40:45 -08:00
Marco
dbc963a42f vp9 pickmode: Fix the shift on negative value.
Error only affected svc 1 pass mode, introduced in:
https://chromium-review.googlesource.com/#/c/319609/

Change-Id: Ifc078c82d38b742966a81baaf5ad532f7dbc13a4
2016-01-21 15:16:52 -08:00
Alex Converse
4326cffa65 Merge "Tie the bit cost scale to a define." 2016-01-21 19:17:56 +00:00
Marco
abf58ecf6a Vidyo patch: Changes to the scalability code.
Changes to mode selection for 1 pass SVC mode:
use base layer motion vector, changes to intra-prediction.

Change-Id: I3e883aa04db521cfa026a0b12c9478ea35a344c9
2016-01-20 08:53:51 -08:00