Commit Graph

137 Commits

Author SHA1 Message Date
Yaowu Xu
25110f2837 Fix an ubsan warning: vp9_quantizer.c
BUG=https://bugs.chromium.org/p/webm/issues/detail?id=1219

Change-Id: Ie1bfda4d51c1567f551515f0ed113eec1bc1d98c
2016-06-16 15:23:21 +00:00
Alex Converse
d3fe3b2abb Avoid an unsigned overflow in invert_quant
Change-Id: I16a570b2af66b6580d1cd6f8345a25f079009bf4
2016-04-27 15:07:22 -07:00
Alex Converse
fac947df77 Restore previous motion search bit-error scale.
The bit to error transformation got doubled as a result of going from
8-bit to 9-bit costs (change d13385c).

Use defines to derive the scale numbers and comment some of the fields.

derf: -0.023 BDRATE
hevcmr: +0.067 BDRATE
stdhd: +0.098 BDRATE
(These are substantially smaller than than the original gains from 8 to
9 bit costing.)

Change-Id: I6a2b3b029b2f1415e4f90a05709b2333ec0eea9b
2016-02-09 13:20:25 -08:00
Scott LaVarnway
5232326716 VP9: Eliminate MB_MODE_INFO
Change-Id: Ifa607dd2bb366ce09fa16dfcad3cc45a2440c185
2016-01-19 16:40:20 -08:00
Jingning Han
d621de7e8d Change vp9_quantize to vpx_quantize
This commit clears all the vp9_ prefix use case in vpx_dsp. It gets
the vp9 folder ready to branch out vp10.

Change-Id: I2906eec179ee792b4af8c9b4161313653050e931
2015-08-04 15:31:49 -07:00
Yunqing Wang
38f1fbbb75 Migrate quantization functions from vp9/ to vpx_dsp/
The following quantization functions were moved:
vp9_quantize_b
vp9_quantize_b_32x32
vp9_highbd_quantize_b
vp9_highbd_quantize_b_32x32

vp9_quantize_dc
vp9_quantize_dc_32x32
vp9_highbd_quantize_dc
vp9_highbd_quantize_dc_32x32

The purpose of doing that was to allow these functions to be shared
by multiple codecs.

Change-Id: Id8ab939f283353cdd07bd930d47db3d932a5d87f
2015-07-17 16:38:14 -07:00
Yaowu Xu
b58c99eb71 Remove clamp operations.
The clamp calls with INT32_MIN and INT32_MAX have no effect at all on
int values passed in, therefore this commit removes those effectless
clamps and also adds more const intermediate results to make the code
more readable.

Change-Id: I66d8811f58bb74ec31cbec9a6c441983a662352e
2015-07-08 17:44:19 -07:00
Scott LaVarnway
42c0b1b1f1 inline vp9_segfeature_active()
and changed name.

Change-Id: Ie023ca66cc2c823032f58d4faeb53fd1863c94f3
2015-06-11 04:20:55 -07:00
Johann
1d7ccd5325 Relocate memory operations for common code
With the sad functions, and hopefully the variance functions soon,
moving to the vpx_dsp location, place the defines used in the
reference C code in a common location.

Change-Id: I4c8ce7778eb38a0a3ee674d2f1c488eda01cfeca
2015-05-13 11:41:15 -07:00
James Zern
f58011ada5 vpx_mem: remove vpx_memset
vestigial. replace instances with memset() which they already were being
defined to.

Change-Id: Ie030cfaaa3e890dd92cf1a995fcb1927ba175201
2015-04-28 20:00:59 -07:00
Scott LaVarnway
8b17f7f4eb Revert "Remove mi_grid_* structures."
(see I3a05cf1610679fed26e0b2eadd315a9ae91afdd6)

For the test clip used, the decoder performance improved by ~2%.
This is also an intermediate step towards adding back the
mode_info streams.

Change-Id: Idddc4a3f46e4180fbebddc156c4bbf177d5c2e0d
2015-04-21 11:16:45 -07:00
Yunqing Wang
6e0ec0b2d9 vp9_ethread: fix me consts initialization to support aq_mode=3 encoding
While turning on "--aq_mode=3", the quantizers are updated by each
thread. Fixed the me consts initialization function to make sure
that the correct thread data are updated.

Change-Id: Ied27bb7bae76fc3fa2cda4f8c35ac0b46271bef4
2015-03-06 16:31:46 -08:00
Deb Mukherjee
6910e92d04 dc quantizer fix for 32x32 transforms
The rounding factor needs to be scaled down by a factor of 2.
Also, the quantized and dequantized coefficients are memset to 0
when dc quantizer is used.

Change-Id: Ifa68bab02addbf1b83d249c5b4cbd5cda796b1cf
2015-03-03 15:58:27 -08:00
Hangyu Kuang
8724d31d12 Move dequant table from VP9_COMMON to VP9_COMP as decoder
does not need it any more.

This reduces VP9_COMMON size from 25776 bytes to 17584 bytes(~31%).

Change-Id: Ic5daea732ccefb6d512b048af7983f0efe08589b
2015-02-20 11:12:42 -08:00
Jingning Han
d0f2377027 Revert "Revert "Removal of legacy zbin_extra / zbin_oq_value.""
This reverts commit 9946ee23e0.

Fix the ssse3 asm function.

Change-Id: I07f77a63aa98087626e45c4e87aa5dcafc0b0b07
2014-12-22 10:09:25 -08:00
Paul Wilkins
9946ee23e0 Revert "Removal of legacy zbin_extra / zbin_oq_value."
This reverts commit e9b586e21b.

Change-Id: I5b36e6727da6c05278d97e2c37b80c109f79bed4
2014-12-19 15:02:58 +00:00
Paul Wilkins
e9b586e21b Removal of legacy zbin_extra / zbin_oq_value.
zbin extra / zbin_oq_value was widely passed around,
hence removal touches a lot of code.

Change-Id: Idc94359735b60c38a160e4385ae09d5ca8b6b8e5
2014-12-18 16:49:11 +00:00
Paul Wilkins
60e9b731cf Remove mode dependent zbin boost.
Initial patch to remove get_zbin_mode_boost() and
cpi->zbin_mode_boost.

For now sets a dummy value of 0 for zbin extra pending
a further clean up patch.

Change-Id: I64a1e1eca2d39baa8ffb0871b515a0be05c9a6af
2014-12-18 16:45:52 +00:00
Yunqing Wang
edbd61e136 vp9_ethread: modify VP9_COMP structure
This patch modified struct VP9_COMP. Created a struct ThreadData
to include data that need to be copied for each thread. In
multiple thread case, one thread processes one tile. all threads
share one copy of VP9_COMP,
(refer to VP9_COMP *cpi in the code)
but each thread has its own copy of ThreadData,
(refer to ThreadData *td in the code).
Therefore, within the scope of encode_tiles(), both cpi and td
need to be passed as function parameters.

In single thread case, the FRAME_COUNTS pointer in ThreadData
points to "counts" in VP9_COMMON.

Change-Id: Ib37908b2d8e2c0f4f9c18f38017df5ce60e8b13e
2014-11-24 17:57:38 -08:00
Yunqing Wang
70c9d2983b Revert "vp9_ethread: include a pointer to mb in VP9_COMP"
This reverts commit 6906d218dd.

Another way will be used to handle mb struct.

Change-Id: Ic1111a46b2b1ee00f8f9e3fcd4cf3eb6030b2dc4
2014-11-20 08:31:12 -08:00
Yunqing Wang
6906d218dd vp9_ethread: include a pointer to mb in VP9_COMP
Modified VP9_COMP struct to include MACROBLOCK *mb. This change
makes it feasible in multi-thread case to allocate a mb for each
thread.

Change-Id: I624d6d1aa9c132362200753e5d90b581b1738d6e
2014-11-14 12:31:06 -08:00
Yaowu Xu
2c4fee17bc Fix visual studio 2013 compiler warnings
For configured with --enable-vp9-highbitdepth

Change-Id: I2b181519d7192f8d7a241ad5760c3578255f24e6
2014-11-05 13:47:28 -08:00
Deb Mukherjee
1929c9b391 Rename highbitdepth functions to use highbd prefix
Uses highbd_ prefix convention consistently.

Change-Id: I58f7f799a7ff8e32701bcd71c955bcf1cdd4581e
2014-10-09 14:40:40 -07:00
Yunqing Wang
e4aac6bb61 Modify block transform skipping check
Block transform skipping was implemented based on DCT's energy
conservation property. Modified the thresholds using zero bin
parameters. AC and DC coefficients were checked separately to
allow better identifying of skippable blocks.

Borg test at speed 3 showed:
stdhd set: psnr gain: 0.153%, ssim gain: 0.051%;
derf set: psnr gain: 0.023%, ssim gain: 0.036%

For most test clips, the encoding speedup is 1% - 2%.
parkrun(720p): 7.5% speedup, park_joy(1080p): 3.5% speedup.

Change-Id: If28eb81113a077414f5ca7b021c14f9069b373bb
2014-10-01 12:58:09 -07:00
hkuang
c70cea97ac Remove mi_grid_* structures.
mi_grid_* are arrays of pointer to pointer. They save the pointers that point
to the MIs in cm->mi. But they are unnecessary and complicated. The original
goal was to remove MODE_INFO_t copy. But with an extra MODE_INFO_t pointer
inside MODE_INFO_t, same goal could be achieved.

This commit totally removes the mi_grid_* structures. But there are still
many dummy MODE_INFO_t inside cm->mi which are a waste of memory. Next commit
will do on-demand MODE_INFO_t allocation in order to save these memories.

Change-Id: I3a05cf1610679fed26e0b2eadd315a9ae91afdd6
2014-09-19 21:27:11 -07:00
Deb Mukherjee
5cd0aab81a Adds high bitdepth quantization functions
Adds various high bitdepth quantization functions.

Change-Id: I36fc0bf75a1bd15128ed271df8723de0ac134b0c
2014-09-16 14:55:37 -07:00
Deb Mukherjee
10783d4f3a Adds high bitdepth transform functions and tests
Adds various high bitdepth transform functions and tests.
Much of the changes are related to using typedefs tran_low_t
and tran_high_t for the final transform cofficients and intermediate
stages of the transform computation respectively rather than fixed
types int16_t/int. When vp9_highbitdepth configure flag is off,
these map tp int16_t/int32_t, but when the flag is on, they map
to int32_t/int64_t to make space for needed extra precision.

Change-Id: I3c56de79e15b904d6f655b62ffae170729befdd8
2014-09-11 19:56:33 -07:00
Jingning Han
b4b09c9796 Enable fast forward txfm and quant for rate-distortion search
This commit enables encoder to select fast forward transform and
quantization path according to the prediction residual sse/variance,
in the rate-distortion optimization scheme.

Change-Id: Ief9fc3844fd4107166d401970e800c6e5ce2b5fe
2014-08-08 16:16:51 -07:00
Alex Converse
5926e7c0e8 Remove unfinished VP9 alpha channel.
Change-Id: Ic5d3a3a0dac10b49495771886a31e793bb78b5ca
2014-07-21 15:55:50 -07:00
Jingning Han
9ad1b9fc67 Re-design quantization process for 32x32 transform block
This commit enables a new quantization process for 32x32 2D-DCT
transform coefficient blocks. It improves the compression
performance of speed 5 by 1.4%. The overall compression gains of
speed 5 due to the new quantization scheme is 4.7%. It also includes
the SSSE3 implementation of the 32x32 quantization process.

Change-Id: I0855b124fd6462418683f783f5bcb44255c9993b
2014-07-08 16:55:28 -07:00
Alex Converse
03c276ea17 Split vp9_rdopt into vp9_rdopt and vp9_rd.
vp9_rdopt is for making rd optimal mode decisions. vp9_rd is for all
other rd related routines. Anything used outside of making an rd optimal
decision belongs in rd.

Change-Id: I772a3073f7588bdf139f551fb9810b6864d8e64b
2014-07-02 15:33:33 -07:00
Jingning Han
9ac2f66320 Re-design quantization process
This commit re-designs the quantization process for transform
coefficient blocks of size 4x4 to 16x16. It improves compression
performance for speed 7 by 3.85%. The SSSE3 version for the
new quantization process is included.

The average runtime of the 8x8 block quantization is reduced
from 285 cycles -> 255 cycles, i.e., over 10% faster.

Change-Id: I61278aa02efc70599b962d3314671db5b0446a50
2014-07-01 17:00:07 -07:00
Jingning Han
ccba289f8d Fast computation path for forward transform and quantization
This commit enables a fast path computational flow for forward
transformation. It checks the sse and variance of prediction
residuals and decides if the quantized coefficients are all
zero, dc only, or more. It then selects the corresponding coding
path in the forward transformation and quantization stage.

It is currently enabled in rtc coding mode. Will do it for rd
coding mode next.

In speed -6, the runtime for pedestrian_area 1080p at 1000 kbps
goes down from 14234 ms to 13704 ms, i.e., about 4% speed-up.
Overall coding performance for rtc set is changed by -0.18%.

Change-Id: I0452da1786d59bc8bcbe0a35fdae9f623d1d44e1
2014-06-12 11:10:54 -07:00
Yaowu Xu
c39a361b0f vp9_quantizer.c: cleanup -wextra warnings
Change-Id: If5a3c48a8c554018a5d63c1541a2900f15767a00
2014-05-14 09:37:45 -07:00
Dmitry Kovalev
ef003078e8 Renaming "onyx" to "encoder".
Actual renames:
  vp9_onyx_if.c -> vp9_encoder.c
  vp9_onyx_int.h -> vp9_encoder.h

Change-Id: I80532a80b118d0060518e6c6a0d640e3f411783c
2014-04-22 14:57:05 -07:00
Alex Converse
e6222b1a47 Fix the CONFIG_ALPHA build.
Change-Id: Ib89fe34812c17cd6294ce3c38f87d43a79abb16f
2014-04-22 11:23:49 -07:00
Paul Wilkins
e434d08fd9 Remove old activity masking code.
Delete code relating to the old VP8_TUNE_SSIM flag
as this code does not currently work and is largely made
redundant in VP9 by the various AQ modes.

Change-Id: I71f28e1f680573d296422254489000678552b17b
2014-04-16 12:22:41 -07:00
Dmitry Kovalev
d1a396d8b9 Moving q_trans[] table to vp9_quantize.{c, h}.
Change-Id: I1324c339815a47004ddccdaf651d24c60382b92f
2014-04-09 13:35:39 -07:00
Dmitry Kovalev
86f44a91f4 Renaming two members in MACROBLOCKD struct.
Renames:
  mi_8x8 -> mi
  mode_info_stride -> mi_stride

Change-Id: I66f3e5fd1e7b7f46f108af5bb711c5fd9493c1be
2014-04-01 17:46:40 -07:00
Dmitry Kovalev
7bcfa31750 Moving encoder quantization parameters into separate struct.
Change-Id: I2a169535489aeda3943fb5a46ab53e7a12abaa36
2014-03-28 16:46:41 -07:00
Dmitry Kovalev
c9513e1dfb Fixing include order in vp9_quantize.c
Change-Id: Ic32eb103d0d7f98c0a16c4e7bdec117faf05df02
2014-02-28 11:30:51 -08:00
Dmitry Kovalev
f527c46f20 Cleaning up vp9_quantize.c.
Change-Id: I9a38af32f16f196b83dd69755eafb9543edf5691
2014-02-28 10:11:31 -08:00
Paul Wilkins
8618c70683 A couple more V.S. warnings silenced.
Change-Id: Ica1b583d69810182f621de757d2543b2a3b35566
2014-02-14 20:34:14 -08:00
Dmitry Kovalev
803a5c67dd Merge "Encoder quantization cleanup." 2014-02-10 21:32:04 -08:00
Dmitry Kovalev
220b8f8644 Encoder quantization cleanup.
Change-Id: I633205c95f0e81ce0589580501d0be4425a3cb8e
2014-02-03 14:57:28 -08:00
Dmitry Kovalev
70cde0af3d Removing ENC_DEBUG.
Change-Id: I101017621003314f000a454725ea13fc9db43177
2014-01-29 12:58:57 -08:00
Dmitry Kovalev
f00d157c12 Moving eob array to the encoder.
In the decoder we don't need to save eobs, we can pass eob as an argument.
That's why removing eob arrays from VP9Decompressor and TileWorkerData,
and moving eob pointer from macroblockd_plane to macroblock_plane.

Change-Id: I8eb919acc837acfb3abdd8319af63d1bbca8217a
2013-12-03 17:59:32 -08:00
Alex Converse
2360a5f093 Remove plane_block_idx.
Its last remaining caller can be passed its results directly without any
additional work. Also, it's not non-4:2:0 safe.

Change-Id: Ia5089ba5f7f66c7617270483c619c9271aefd868
2013-12-02 18:33:50 -08:00
Dmitry Kovalev
d3a2e55af4 Removing qcoeff buffers from the decoder.
We only need qcoeff buffers in the encoder. Reducing TileWorkerData struct
and VP9Decompressor struct sizes by 24K.

Change-Id: Id148868461f7ffa3d3dd634b371503ae9c57e207
2013-11-26 18:52:10 -08:00
Dmitry Kovalev
3f3d14e1d3 Moving q_index from MACROBLOCKD to MACROBLOCK.
Moving because q_index is used only by encoder.

Change-Id: I0b96175614ed4fd3d76ee56a0ba36258e1e896f6
2013-11-12 18:13:19 -08:00