Commit Graph

242 Commits

Author SHA1 Message Date
James Zern
5d233390db vp10_cyclic_refresh_alloc: correct cleanup on error
previously only the CYCLIC_REFRESH allocation was being freed

Change-Id: I600eb496ec4b62accf1a6483c8170eabb046787d
2016-02-17 14:36:05 -08:00
James Zern
fdc977afc6 vp10,encoder: relocate setjmp
move to encoder_encode() as vp10_get_compressed_data() allocates data and
would require some modification to make its error return meaningful.

Change-Id: Ia5267c35d16ccd42b6da6d2136402b13e28f9159
2016-02-16 19:33:16 -08:00
James Zern
b5029a25a8 vp10_cx_iface,encode: remove redundant cpi check
this is allocated in init(), a NULL pointer on encode() should be
treated as a failure.

Change-Id: I09c206f088982716fd5acc39949ceb25576313fc
2016-02-16 18:10:30 -08:00
James Zern
8628898acf vp10_receive_raw_frame: add missing setjmp
allocations done within this function are protected with
vpx_internal_error; adding the setjmp fixes a crash in
vp10_lookahead_push() under low memory conditions.

Change-Id: I5515017cd71b218840c506791b3a517da7ffc93e
2016-02-11 19:21:28 -08:00
Debargha Mukherjee
f28ea3e830 Explicitly set tx_type for sub8x8 blocks
Fixes an issue where the tx_type was not set correctly for
sub8x8 inter and intra blocks. In the current syntax, for
sub8x8 blocks, there is still a single tx_type that is
transmitted. Ideally, this should be searched for the best
rd performance, albeit at the expense of encode speed.
For now, we just set it to DCT_DCT. Previously it was left
incorrectly as what was used for the previous non sub8x8
block.

derflr: BDRATE -0.277%

Change-Id: If76ba903bfbfd4d374cf1ac7d1daee50e92f0edd
2016-02-08 16:11:53 -08:00
Zoe Liu
0f7dd40324 Define a macro to replace hardcoded values
The definition is for the number of frames to check to determine the
recent decay rate, further to determine the next key frame in the
first pass of the encoder.

Change-Id: Ic696d6eb518a86fa296842273cf8767ef0b0e27a
2016-02-04 05:33:22 -08:00
James Zern
ebf258688e vp10: remove unused (read|write)_uniform
dead code since:
5d3327e Remove palette from VP10

Change-Id: I4a36575706ea6fffefe5bc778595112ef6ff37d8
2016-02-01 23:47:35 -08:00
hui su
580e815f3b Refactor prob_diff_update_savings_search_model
Change-Id: Icfab58e880285d5af00985dc20db2174a67d1fe8
2016-01-29 12:51:10 -08:00
Zoe Liu
bca1a350c9 Merge "Cleaned code in firstpass related to altref frame" 2016-01-29 16:31:41 +00:00
Zoe Liu
b6f1f369af Cleaned code in firstpass related to altref frame
Change-Id: I6dbddf500fd4f64216774685700efea972165470
2016-01-28 16:37:17 -08:00
James Zern
4f780e94a1 vp9/10: fix encoder crash on flush
the lookahead buffer allocation is deferred to receipt of the first
frame to allow profile changes. if the encoder was flushed before
supplying any frames the encoder would crash trying to dereference the
NULL buffer. vp8 is unaffected.

fixes mozilla bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1237848

Change-Id: Icee4b64de760476eee0d33b568f0a1010335ff13
2016-01-28 12:53:41 -08:00
Hui Su
cdb141dd94 Merge "Fix some typos." 2016-01-28 01:04:50 +00:00
hui su
5afc4e4c77 Fix some typos.
Change-Id: I32aacd014df6c927cf2893dc096cbe6ec7604b9b
2016-01-27 16:12:49 -08:00
Zoe Liu
9e612763f0 Cleaned a code in define_gf_group() for firstpass
Change-Id: I0b8eff9e74e3446024482d011ec4fb28b19136d3
2016-01-25 09:38:45 -08:00
Yaowu Xu
b6cefbf052 Fix a typo in ctrl_set_render_size()
Change-Id: I6a7fa6d0ae8ce8de7b426189d5eb00413f21bc17
2016-01-21 11:19:08 -08:00
Alex Converse
dea6f5217e Deduplicate sign cost for ONE/TWO/THREE/FOUR tokens
Change-Id: I9c383d604dfd240e38b198ac5bf86ddd3456f442
2016-01-15 13:56:34 -08:00
Yaowu Xu
d803e8a20e Fix encoder crashes and enc/dec mismatches
This commit adds the logic for segmentation map initialization and
disable temporal update of segmentation map when error-resilient
mode is on. It fixes the enc/dec mistmates (release build) and
assertions(debug) when both aq-mode and error-resilient are on.

Change-Id: Id2155e8b28962cf1f64494f4df0c8d79499b6890
2016-01-13 09:51:05 -08:00
Debargha Mukherjee
a0900fd0db Remove experimental flag for ext_tx
Also includes a bug fix.

Change-Id: Ia49ed00f8ffd1531c10bcf89b1f497310ee7cb82
2016-01-08 13:48:24 -08:00
Debargha Mukherjee
f7dfa4ece7 Modifies inter/intra coding to allow all tx types
The nominal tx_type for a given mode is used as a context
to encode the actual tx_type for intra.

Results:
derflr: -0.241% BDRATE
hevcmr: -0.366% BDRATE

Change-Id: Icfe7b0a58d79bc6497a06e3441779afec6e01e21
2016-01-08 11:13:46 -08:00
Ronald S. Bultje
53a11656cd vp10: only assume ONLY_4X4 if segmentation is disabled.
Otherwise, per-segment lossless might mean that some segments are not
lossless and they could still want to use another mode. The per-block
tx points remain uncoded on blocks where (per the segment id) the Q
value implies lossless.

Change-Id: If210206ab1fe3dd11976797370c77f961f13dfa0
2016-01-04 15:21:02 -05:00
Ronald S. Bultje
d9439fdc36 vp10: skip coding of txsz for lossless-segment blocks.
Change-Id: Ic23c10b6d2a9fed3abe69c6bf10e910832444f2c
2016-01-04 15:21:02 -05:00
Yaowu Xu
e650129683 Move bit_depth init out of setup_quantization
This also fixes a compiling error under --enable-vp9_highbitdepth.

Change-Id: I9d1dcb95d3336d797eb3c23a4702c30b04355357
2015-12-16 11:43:11 -08:00
Yaowu Xu
eace551c87 Merge changes Icf9b57c3,I9e12da84,Idf5ee179
* changes:
  Fixed interval, fixed Q 1 pass test patch.
  1 pass VBR mode bug fix.
  Fixed interval, fixed Q 1 pass test patch.
2015-12-15 17:51:33 +00:00
Yaowu Xu
9232f69b26 Merge "Fix a enc/dec mismatch under CONFIG_MISC_FIXES" 2015-12-15 16:02:39 +00:00
paulwilkins
99309004bf Fixed interval, fixed Q 1 pass test patch.
For testing implemented a fixed pattern and delta, 1 pass,
fixed Q, low delay mode.

This has not in any way been tuned or optimized.

Change-Id: Icf9b57c3bb16cc5c0726d5229009212af36eb6d9
2015-12-15 15:33:25 +00:00
paulwilkins
9ce611a764 1 pass VBR mode bug fix.
(copied from VP9)

The one pass VBR mode selects a Q range based on a
moving average of recent Q values. This calculation
should have been excluding arf overlay frames as these
are usually coded at the highest allowed value. Their
inclusion skews the average and can cause it to drift
upwards even when the clip as a whole is undershooting.

As such it can undermine correct adaptation of the allowed
Q range especially for easy content.

Change-Id: I9e12da84e12917e836b6e53ca4dfe4f150b9efb1
2015-12-15 15:02:40 +00:00
Yaowu Xu
c7101830a6 Fix a enc/dec mismatch under CONFIG_MISC_FIXES
The culprit is on the decode side xd->lossless[i] setup was in wrong
location where segment features are not yet decoded.

Also on the encoder side, transform mode was not set consistently
between when tx_mode is selected and how tx_mode is enforced in
tx size selection.

Change-Id: I4c4c32188fda7530cadab9b46d4201f33f7ceca3
2015-12-14 20:56:37 -08:00
James Zern
d36659cec7 move vp9_avg to vpx_dsp
Change-Id: I7bc991abea383db1f86c1bb0f2e849837b54d90f
2015-12-14 14:42:12 -08:00
Yaowu Xu
f0bef772be Merge "Proper fix of a msvc complier warning" 2015-12-11 00:53:28 +00:00
Yaowu Xu
4d2cfeab36 Proper fix of a msvc complier warning
Change-Id: I701ab4993be7cfb15b61a1adbbaf5565bd14ae27
2015-12-10 16:29:01 -08:00
Yaowu Xu
5a81c5c4be Merge changes Iece22223,Iefad9d8d
* changes:
  Fix two msvc build issues
  Fix enc/dec mismatches for aq-mode 1 and 2
2015-12-10 23:32:32 +00:00
Yaowu Xu
6786280807 Fix two msvc build issues
Change-Id: Iece22223773dd6d0f87f8f59827705acd2ebe2a4
2015-12-10 12:41:27 -08:00
Yaowu Xu
221ed5e47b Fix enc/dec mismatches for aq-mode 1 and 2
Change-Id: Iefad9d8d96a08dcc788a5efdca2df6a815d1205f
2015-12-10 11:45:26 -08:00
paulwilkins
4e692bbee2 Changes to exhaustive motion search.
This change has been imported from VP9 and
alters the nature and use of exhaustive motion search.

Firstly any exhaustive search is preceded by a normal step search.
The exhaustive search is only carried out if the distortion resulting
from the step search is above a threshold value.

Secondly the simple +/- 64 exhaustive search is replaced by a
multi stage mesh based search where each stage has a range
and step/interval size. Subsequent stages use the best position from
the previous stage as the center of the search but use a reduced range
and interval size.

For example:
  stage 1: Range +/- 64 interval 4
  stage 2: Range +/- 32 interval 2
  stage 3: Range +/- 15 interval 1

This process, especially when it follows on from a normal step
search, has shown itself to be almost as effective as a full range
exhaustive search with step 1 but greatly lowers the computational
complexity such that it can be used in some cases for speeds 0-2.

This patch also removes a double exhaustive search for sub 8x8 blocks
which also contained  a bug (the two searches used different distortion
metrics).

For best quality in my test animation sequence this patch has almost
no impact on quality but improves encode speed by more than 5X.

Restricted use in good quality speeds 0-2 yields significant quality gains
on the animation test of 0.2 - 0.5 db with only a small impact on encode
speed. On most natural video clips, however, where the step search
is performing well, the quality gain and speed impact are small.

Change-Id: Iac24152ae239f42a246f39ee5f00fe62d193cb98
2015-12-08 16:54:42 +00:00
paulwilkins
9d85ce8e0c Fix bug when overlaying middle arfs in multi-arf groups.
Fix copied over from VP9 master to VP10 master.
Do not reset the alt ref active flag when overlaying the middle
arf(s) of a multi arf group.

Change-Id: I1b7392107e7c675640d5ee1624012f39cc374c58
2015-12-07 15:23:46 +00:00
Angie Chiang
06bdcea606 Merge "comment out range_check of fdct in dct.c" 2015-12-04 23:38:35 +00:00
Angie Chiang
08b157da8e comment out range_check of fdct in dct.c
The range_check is not used because the bit range
in fdct# is not correct. Since we are going to merge in a new version
of fdct# from nextgenv2, we won't fix the incorrect bit range now.

Change-Id: I54f27a6507f27bf475af302b4dbedc71c5385118
2015-12-04 10:54:31 -08:00
hui su
5d3327e891 Remove palette from VP10
Store it in nextgenv2 for now.

Change-Id: Iab0af0e15246758e3b6e8bde4a74b13c410576fc
2015-12-03 12:30:47 -08:00
Alex Converse
b1fcd1751e Fix unsigned overflow in rd_variance_adjustment.
Found with clang -fsanitize=integer

Change-Id: I2538e7483cb2d5f06bceecbd3326bdd88bfecfa1
2015-11-19 15:00:59 -08:00
hui su
6ab6ac450b Use accurate bit cost for uv_mode in UV intra mode RD selection
On derflr, +0.1% for VP10; however, -0.03% on VP9.

Change-Id: I09c724232ede74254043d61d3cadc506256af0af
2015-11-06 14:45:43 -08:00
Hui Su
3cbe767972 Merge "Generate intra prediction reference values only when necessary" 2015-11-03 20:55:14 +00:00
Alex Converse
255bcf8697 Merge "misc fixes: Remove a wasted value." 2015-11-03 17:52:34 +00:00
hui su
16bf821dfc Move palette-based intra prediction out of misc-fixes
Change-Id: Ia59724413c4a4831390119a33d40a7d713b4b69f
2015-11-02 11:11:25 -08:00
hui su
e085fb643f Generate intra prediction reference values only when necessary
This can help increase encoding speed substantially.

Change-Id: Id0c009146e6e74d9365add71c7b10b9a57a84676
2015-11-02 10:26:50 -08:00
Alex Converse
989193c797 Make the zero handling in extend_to_full_distribution more explicit.
The old workaround "p = 0 ? 0 : p -1" is misleading.

?: happens before =
assigning back to p truncates to one byte.

Therefore it is equivalent to (p - 1) & 0xFF, but the check just exists
to work around a first pass bug, so let's make the work around more
clear.

https://bugs.chromium.org/p/webm/issues/detail?id=1089

Change-Id: I587c44dd61c1f3767543c0126376f881889935af
2015-10-29 14:46:55 -07:00
Alex Converse
6f229b3e62 Merge "Shrink probability remap tables." 2015-10-29 19:58:24 +00:00
Alex Converse
663960e757 Revert "Replace the zero handling in extend_to_full_distribution."
This reverts commit 7f56cb2978.

It causes uninitialized reads in the first pass setting up later cost tables.

Change-Id: I2df498df3f5c03eff359f79edf045aed0c618dc9
2015-10-28 11:51:40 -07:00
Alex Converse
0f059d6d65 misc fixes: Remove a wasted value.
Remove delta index 254 from probability remapping and subexp coding.
Saves 1-bit when the delta index is 129.

Change-Id: I88aba565fc766b1769165be458d2efd3ce45817e
2015-10-27 12:10:25 -07:00
Alex Converse
a736bf6bfb Shrink probability remap tables.
Saves 2288 bytes in vp8+vp9 libvpx.a.

Change-Id: Iaa5712e59a9693ed58cea63de63781a96827e44e
2015-10-27 12:08:23 -07:00
Yaowu Xu
c1b2d416d7 Merge "Reorder code to be consistent accross branches" 2015-10-27 17:07:50 +00:00