Commit Graph

404 Commits

Author SHA1 Message Date
Jingning Han
14011f037d Remove txfrm_block_to_raster_xy() from vp9 encoder
The transform block row and column positions are always available
outside the callees. There is no need to re-compute these values
again. This approach has been used by the decoder. This commit
removes txfrm_block_to_raster_xy() function.

Change-Id: I5b90f91a0d8b7c35cfa7d171da9edf8202630108
2016-07-04 18:41:47 -07:00
Yaowu Xu
b9ec759bc2 Fix ubsan warnings: vp9/encoder/vp9_pickmode.c
This commit fixes a number of integer out of range issue in HBD build.

BUG=webm:1219

Change-Id: Ib4192dc74a500e1b86c37a399114c7f6d4ed5185
2016-06-27 05:53:46 +00:00
James Zern
1c0a9f36f1 vp9_pickmode: revert rd modeling change for hbd
Avoids a segfault in high-bitdepth builds.
This restores the condition to its state prior to:
7991241 vp9: Change the scheme for modeling rd for bsize 32x32.

BUG=webm:1250

Change-Id: I6183d5b34cb89dfbf27b7bb589812148a72cd7de
2016-06-25 11:40:26 -07:00
jackychen
7991241a50 vp9: Change the scheme for modeling rd for bsize 32x32.
For real-time CBR mode, use model_rd_for_sb_y_large instead of
model_rd_for_sb_y for 32x32 block. In the former model, transform
might be skipped more aggressively in some condtions, which speeds
up encoding time with only a little PSNR/SSIM drop on rtc test set.
No obvious visual quality regression.

PSNR effect on different speed settings:
speed 8 rtc:  0.129% overall PSNR drop, 0.137% SSIM drop
speed 7 rtc:  0.135% overall PSNR drop, 0.062% SSIM drop
speed 5 rtc_derf: 0.105% overall PSNR drop, 0.095% SSIM drop

Speed up:
gips_motion_WHD, 1mbps: 3.29% faster on speed 7, 2.56% faster on speed8
gips_stat_WHD, 1mbps: 2.17% faster on speed 7, 1.62% faster on speed8

BUG=webm:1250

Change-Id: I818babce5b8549b4b1a7c3978df8591bffde7173
2016-06-24 12:09:13 -07:00
James Zern
d4596485be Revert "vp9: Change the scheme for modeling rd for bsize 32x32."
This reverts commit 5c29ee726e.

Causes segfaults in VP9/EndToEndTestLarge.EndtoEndPSNRTest.

BUG=webm:1250

Change-Id: I8a30e97be30589abdb76820b5c3c37c46cd6cafb
2016-06-23 15:59:25 -07:00
Angie Chiang
424982bc41 Merge "set interp_filter to SWITCHABLE_FILTER for intra block" 2016-06-23 18:56:27 +00:00
Angie Chiang
d9c417cb49 set interp_filter to SWITCHABLE_FILTER for intra block
In vp9_pick_inter_mode(), instead of using
vp9_get_pred_context_switchable_interp(xd) to assign filter_ref,
we use a less strict condition on assigning filter_ref.
This is to reduce the probabily of entering the flow of not
assigning filter_ref and then skipping filter search.

Overall PSNR gain 0.074% for rtc dataset

Details:
Low    Mid     High
0.185% -0.008% -0.082%

Change-Id: Id5c5ab38d3766c213d5681e17b4d1afd1529e676
2016-06-22 17:19:43 -07:00
Jacky Chen
8496390e73 Merge "vp9: Change the scheme for modeling rd for bsize 32x32." 2016-06-22 23:50:46 +00:00
jackychen
5c29ee726e vp9: Change the scheme for modeling rd for bsize 32x32.
For real-time CBR mode, use model_rd_for_sb_y_large instead of
model_rd_for_sb_y for 32x32 block. In the former model, transform
might be skipped more aggressively in some condtions, which speeds
up encoding time with only a little PSNR/SSIM drop on rtc test set.
No obvious visual quality regression.

PSNR effect on different speed setting:
speed 8 rtc:  0.129% overall PSNR drop, 0.137% SSIM drop
speed 7 rtc:  0.135% overall PSNR drop, 0.062% SSIM drop
speed 5 rtc_derf: 0.105% overall PSNR drop, 0.095% SSIM drop

Speed up:
gips_motion_WHD, 1mbps: 3.29% faster on speed 7, 2.56% faster on speed8
gips_stat_WHD, 1mbps: 2.17% faster on speed 7, 1.62% faster on speed8

Change-Id: I902f62def225ea01c145d7e5a93497398b8f5edf
2016-06-22 11:17:56 -07:00
Yaowu Xu
87bf1a149c Fix ubsan warnings: vp9/encoder/vp9_mcomp.c
This commit fixes a number of ubsan warnings in HBD build.

BUG=webm:1219

Change-Id: I05f0fd0ef50e93db4ba34205005c54af1ed32acc
2016-06-21 15:37:59 -07:00
James Zern
cf2bfcd9d8 Merge "vp9_pickmode: quiet int64->int conversion warning" 2016-06-17 03:58:46 +00:00
Marco Paniconi
8870756ac6 Merge "vp9: Adjustments to nonrd-pickmode for vbr" 2016-06-16 16:07:06 +00:00
James Zern
ff8edd3509 vp9_pickmode: quiet int64->int conversion warning
since:
7901a05 VP9: block_yrd(): Use RD_COST * instead

Change-Id: I808be7a7bc96b81eb7d6255ae96cea7e9b1ecf1f
2016-06-15 23:34:09 -07:00
Marco
8e070558b0 vp9: Adjustments to nonrd-pickmode for vbr
For VBR: (1) allow newmv mode for golden ref to
select interpolation filter (as in last ref case), and
(2) don't use the more aggressive tx-skip testing logic for large blocks.

Only affects 1 pass real-time vbr mode (speed >= 5).

PSNR/SSIM metrics on ytlive set are all positive, ~0.5-2% gain.

Change-Id: I0ffbb0a9755563a5acd6230c58236e4f19a47266
2016-06-15 14:31:28 -07:00
Scott LaVarnway
55d0ffeb54 Merge "VP9: block_yrd(): Use RD_COST * instead" 2016-06-15 21:02:45 +00:00
Jacky Chen
9c90830165 Merge "vp9: Code clean up for short circuit feature in low temp variance." 2016-06-15 16:49:32 +00:00
Scott LaVarnway
7901a051c9 VP9: block_yrd(): Use RD_COST * instead
of int *rate, int *dist.

Change-Id: I7cbaea5252615b6b45b33e896f637cf7d076588a
2016-06-15 05:39:00 -07:00
Scott LaVarnway
221fcdac85 Merge "VP9: minor block_yrd() refactoring" 2016-06-15 11:46:54 +00:00
JackyChen
17962ab2c0 vp9: Code clean up for short circuit feature in low temp variance.
Change-Id: I7573a5cf0ab79abed8d124019c0ed6d9531277f9
2016-06-14 16:53:13 -07:00
JackyChen
f9c0587200 vp9: Encoding cycle reduction for speed 8.
1. Skip golden non-zeromv and newmv-last for bsize >= 16x16 if the
temporal variance obtained from choose_partitioning is very low.
2. Skip horz and vert INTRA mode for speed 8.

This change works best on the clips with little noise and with some
motion (e.g. gips_motion which has > 5% speed up). PSNR drop is 1.78%
on rtc test set, no obvious visual quality regression found.

Change-Id: Ib43b5b20e67809d03c5a6890818ddff59e1fc94a
2016-06-13 09:33:22 -07:00
Scott LaVarnway
580317160c VP9: minor block_yrd() refactoring
Change-Id: Ic0a81fb5816c8252ca4876753320fde5c28d893f
2016-06-13 07:35:45 -07:00
jackychen
bacc67f4a8 vp9: Skip some modes when variance is low for big blocks, for 1 pass real-time.
Skip intra-mode and some inter-modes (newmv, nearmv, nearestmv) for
golden frame if the variance got from choose_partitioning is very low.
Only for 1 pass real-time CBR mode and bsize >= 32x32, it has ~2.5%
speed up with less than 0.1% PSNR drop for rtc test set. Don't see
visual regression.

Change-Id: I70efbc95a1007231ae36f02c5b2fbf6cd35077ad
2016-06-01 13:54:18 -07:00
Marco
bedf1c3af6 vp9: Skip computation of best_sad for newmv, unless needed.
For non-rd pickmode:
best_pred_sad, computed for NEWMV-last, is only used for
skipping golden non-zero modes. Add condition to avoid this
computation if not used (i.e, if golden nonzero modes are not used).

And remove code for computing best_pred_sad for NEWMV-golden,
since that sad is not used.

No change in behavior; small speed gain (~1%) for svc encodes.

Change-Id: Ic2cbdef6c4e9a233a57c0db0eeac8ad5fcead366
2016-05-31 10:29:00 -07:00
jackychen
6f397b8a5b vp9: Remove a redundent condition in sub-pixel filter choosing.
Change-Id: I5cbb0f452ec9622437482b3a9496ead1253acfe0
2016-05-20 14:38:45 -07:00
JackyChen
af0a9fcf70 vp9: Refactor some denoiser logic in vp9_pick_inter_mode.
Move the logic for rechecking zeromv on denoised block out to simplify
the function. To simplify the param passing, add a new structure
VP9_PICKMODE_CTX_DEN which is only used when denoiser is enabled.

Change-Id: Iaa9b4396dfcb8147236c02d4a1868a09103a4476
2016-05-19 15:28:25 -07:00
JackyChen
0dfa6b38dc Move non-zero mv bias on large block out of vp9_pick_inter_mode.
This is to simplify pick inter mode funtion and is bitexact.

Change-Id: Ib1a115bac5fbd212c626be194bcd4a6ce4af9bf2
2016-05-13 15:29:05 -07:00
Alex Converse
d8a18186ba pickmode: Fix a pair of unsigned overflows.
block_variance: This operates on 8x8s and would be safe with a int32 *
int32 to uint32 multiply, but this is potentially unsafe for 12-bit
input. Unfortunately the code already segfaults on 12-bit input:
https://bugs.chromium.org/p/webm/issues/detail?id=1223

calculate_variance: This operates on up to a 32x32 of 8x8s and can
overflow even with 8-bit input (log2((256*32*32)**2) == 36).

BUG=https://bugs.chromium.org/p/webm/issues/detail?id=1220

Change-Id: I1ca4ff6092db9a7580da371ee9a21f403fdadc40
2016-05-09 11:21:20 -07:00
James Zern
9bfa12764c vp9_pickmode.c: make function static
set_intra_cost_penalty() is local to this module

Change-Id: Ie277bfc5e8a26f05a04fd261bdb888388ad0e2a1
2016-05-04 15:10:00 -07:00
JackyChen
c69a141a9e vp9: Don't force block size to 8x8 in denoiser.
First, we only set use_4x4_partition for key frame where we don't
denoise; second, envision we have small partitions, we should pass the
actual block size to denoiser and make an early termination if needed.

Change-Id: I331f42046d792b17360723d17ff817d601394658
2016-04-27 22:18:11 -07:00
Alex Converse
a3a10a323b pickmode: only cost the skip flag once per prediction block
RTC speed 6:
File    Match   Avg     BDRate  Low     Mid     High
OVERALL ✔       -0.040  -0.045  -0.031  -0.084  0.004

Screencast speed 6:
File     Match   Avg     BDRate  Low     Mid     High
OVERALL ✔       1.115   -0.162  0.203   2.470   0.541

Change-Id: I46bbc11c89301015b5d3eac25294c709f23f0897
2016-04-11 15:39:52 -07:00
Marco
ac86dde3af vp9-denoiser: Avoid copy-block when denoising is at LowLow level.
Avoid copy-block when denoising is at LowLow level (i.e., no denoising is done).
Instead, don't enter denoiser at all, and when level goes back up over kLowLow
do a reset in denoiser.

Change-Id: I0544adf58f4dd51ecc4a4607fcb0353bfbbb7a59
2016-04-04 17:02:24 -07:00
Scott LaVarnway
ba962a5f37 VP9: Eliminate up_available and left_available
Use above_mi and left_mi instead.

Change-Id: I0b50e232c31d11da30aa2fb6f91a695aaf725e0c
2016-03-30 04:47:39 -07:00
Marco
48e7294e48 vp9-denoiser: Incorporate consec_zeromv in skin detection for denoising.
Change-Id: Ibe4e1207c0db1779e1358f4566da67c9a07bdb15
2016-03-28 10:57:54 -07:00
Marco
f03dfcb118 vp9-svc: Bugfix on usage of svc_force_zeromv in non-rd pickmode.
Only affects 1 pass cbr svc mode.

Change-Id: Ifc0948b4aabffa64a9701295d8dbc58a7208a30f
2016-03-16 16:12:16 -07:00
Marco
caa8de547f vp9 non-rd pickmode. Some adjustments for vbr mode.
Some adjustments to inter-mode selection for vbr mode.
Condition some of the bias to low/zero motion on cbr mode, and
don't use int_pro_motion_estimation for golden ref
(treat it same as last ref).

Change only affect 1 pass vbr mode, speed >=5 (non-rd pickmode).

Encoding time increase within ~5%.
Avg PSNR/SSIM on RTC set increase by ~2%, all clips up,
ranging from 0.5 to 4%.

Change-Id: I0048d0104a8816773d91a2b1484d601169d9bad7
2016-03-11 09:22:36 -08:00
Marco
2b5ab8095e vp9-svc: Non-rd pickmode, fix condition for testing zeromv for spatial ref.
Change-Id: I75bad321d5814a6df6735a8ead7700ab93b50763
2016-03-07 09:55:25 -08:00
Alex Converse
def519efb3 Fix an uninitialized warning.
Change-Id: I3ac1070df2948f6d8637ade7192400be53a7675c
2016-02-05 12:06:38 -08:00
Marco
1c2df9cd48 vp9: encoder-breakout: Modify UV thresholds.
Set UV thresholds to 0 for encoder breakout if
block is detected as skin.

Change-Id: Iaaa30b5cc8bae4cf611f8ea8eba693b730e32a3f
2016-01-28 12:12:40 -08:00
Marco
b39a599cef vp9 non-rd mode: Modification for detected skin areas.
If a superblock contains alot of "skin" then force split
of 64x64 partition, and make some adjustments in mode selection.

This helps to reduce artifacts on moving face/skin areas at low bitrates.

Little/no change in metrics: avgPSNR/SSIM down by ~0.12%.
Small encoding time increase < 1%.

Change-Id: Ic57f52148c3716f391419fab0530d916e4c1d186
2016-01-27 17:38:58 -08:00
Alex Converse
7da6324cab Short circuit flat blocks when coding screen content at realtime speed.
In inter mode search skip all modes except NEARESTMV and DC_PRED.

10% less encode latency for large frames using the chromium remoting_perftests.

+0.313% BDRATE on the screencast set at speed -6.

Change-Id: Ib97a39dd8bcdeab545509e0e02d78ce7033f8c63
2016-01-22 12:40:45 -08:00
Marco
dbc963a42f vp9 pickmode: Fix the shift on negative value.
Error only affected svc 1 pass mode, introduced in:
https://chromium-review.googlesource.com/#/c/319609/

Change-Id: Ifc078c82d38b742966a81baaf5ad532f7dbc13a4
2016-01-21 15:16:52 -08:00
Alex Converse
4326cffa65 Merge "Tie the bit cost scale to a define." 2016-01-21 19:17:56 +00:00
Marco
abf58ecf6a Vidyo patch: Changes to the scalability code.
Changes to mode selection for 1 pass SVC mode:
use base layer motion vector, changes to intra-prediction.

Change-Id: I3e883aa04db521cfa026a0b12c9478ea35a344c9
2016-01-20 08:53:51 -08:00
Scott LaVarnway
5232326716 VP9: Eliminate MB_MODE_INFO
Change-Id: Ifa607dd2bb366ce09fa16dfcad3cc45a2440c185
2016-01-19 16:40:20 -08:00
Alex Converse
269428e35c Tie the bit cost scale to a define.
This is a pure-refactor in preparation to potentially raise the bit-cost
resolution.

Verified at good speed 0 and rt speed -6.

Change-Id: I5347e6e8c28a9ad9dd0aae1d76a3d0f3c2335bb9
2016-01-15 15:59:31 -08:00
Scott LaVarnway
a85e552d95 VP9: Remove decoder args from find_mv_refs_idx()
The decoder does not use this function.

Change-Id: Ie67f909c0f4108ef286789c70df867d4b960a780
2016-01-13 13:30:40 -08:00
Yaowu Xu
4893d28186 Merge "Enable encoder to avoid 8x4 or 4x8 partitions" 2016-01-07 23:00:00 +00:00
Yaowu Xu
9cac17d157 Enable encoder to avoid 8x4 or 4x8 partitions
This commit enables encoder to avoid 8x4 and 4x8 partitions for
scaled reference frames when libvpx is configured and built with
--enable-better-hw-compatibility

Change-Id: I02ad65c386f5855f4325d72570c49164ed52f413
2016-01-07 09:53:14 -08:00
Marco
b520882f0e vp9-svc: Fix to avoid msan unitialized value.
Move the logic for forcing zero_mode after the
(ref_frame & flag_list) check.
This was causing an memory leak under msan:
https://bugs.chromium.org/p/webrtc/issues/detail?id=5402

Change-Id: Ie9d243369f8ed7c332f46178275945331da4fd85
2016-01-06 11:34:57 -08:00
Marco Paniconi
93c0b879d4 Merge "SVC 1 pass mode: Constrain inter mode search within superframe." 2015-12-15 16:25:20 +00:00