Commit Graph

142 Commits

Author SHA1 Message Date
Jingning Han
9ac2f66320 Re-design quantization process
This commit re-designs the quantization process for transform
coefficient blocks of size 4x4 to 16x16. It improves compression
performance for speed 7 by 3.85%. The SSSE3 version for the
new quantization process is included.

The average runtime of the 8x8 block quantization is reduced
from 285 cycles -> 255 cycles, i.e., over 10% faster.

Change-Id: I61278aa02efc70599b962d3314671db5b0446a50
2014-07-01 17:00:07 -07:00
Yunqing Wang
9d41313e4b Decide the partitioning threshold from the variance histogram
Before encoding a frame, calculate and store each 16x16 block's
variance of source difference between last and current frame.
Find partitioning threshold T for the frame from its variance
histogram, and then use T to make partition decisions.

Comparing with fixed 16x16 partitioning, rtc set test showed an
overall psnr gain of 3.242%, and ssim gain of 3.751%. The best
psnr gain is 8.653%.

The overall encoding speed didn't change much. It got faster for
some clips(for example, 12% speedup for vidyo1), and a little
slower for others.

Also, a minor modification was made in datarate unit test.

Change-Id: Ie290743aa3814e83607b93831b667a2a49d0932c
2014-06-30 09:36:23 -07:00
Alex Converse
aeacaac574 Switch active map implementation to segment based.
Change-Id: Ibb841a1fa4d08d164cf5461246ec290f582b1f80
2014-06-20 13:13:23 -07:00
Dmitry Kovalev
f80a346e0e Merge "Replacing txfm_size with tx_size." 2014-06-12 13:07:11 -07:00
Dmitry Kovalev
4345d12d28 Replacing txfm_size with tx_size.
Change-Id: Ifa6374e9db5919322733b656e0865f5f19ee6f2c
2014-06-12 11:57:26 -07:00
Jingning Han
ccba289f8d Fast computation path for forward transform and quantization
This commit enables a fast path computational flow for forward
transformation. It checks the sse and variance of prediction
residuals and decides if the quantized coefficients are all
zero, dc only, or more. It then selects the corresponding coding
path in the forward transformation and quantization stage.

It is currently enabled in rtc coding mode. Will do it for rd
coding mode next.

In speed -6, the runtime for pedestrian_area 1080p at 1000 kbps
goes down from 14234 ms to 13704 ms, i.e., about 4% speed-up.
Overall coding performance for rtc set is changed by -0.18%.

Change-Id: I0452da1786d59bc8bcbe0a35fdae9f623d1d44e1
2014-06-12 11:10:54 -07:00
Dmitry Kovalev
35a83677a5 Moving itxm_add pointer from MACROBLOCKD to MACROBLOCK.
The final goal is eventually to get rid of both itxm_add and fwd_txm4x4.
This patch does it in the decoder.

Change-Id: Ibb3db57efbcbb1ac387c6742538a9fcf2c6f24a5
2014-05-21 11:09:44 -07:00
Dmitry Kovalev
81e03394d6 Replacing int_mv with MV.
Change-Id: Icd7eea20e944e3e28e5eb20cdc088866a54d53b4
2014-05-19 11:43:07 -07:00
Dmitry Kovalev
51545f5753 Moving PC_TREE from MACROBLOCK to VP9_COMP.
Because PC_TREE is encoder-level data, not MACROBLOCK-level data.

Change-Id: I4f620c0781acd3a2744860610117e74948e0b2b5
2014-05-16 10:17:13 -07:00
Dmitry Kovalev
66307bf2c8 Moving costs from MACROBLOCK to VP9_COMP.
Change-Id: I61471dd0f77d1547abec13cbf9670e1c4eb9131a
2014-05-01 16:12:23 -07:00
Dmitry Kovalev
aa464eca5e Adding search_site_config struct.
Change-Id: I2ad333553e673dbabcdc0f0366aea311e90849bf
2014-04-29 10:34:53 -07:00
Jingning Han
80a4f55989 Enable background detection for adaptive quantizer control
This commit enables a background detection approach for adaptive
quantizer control. It combines the cyclic refresh pattern and the
background information to determine the segment id for adaptive
quantizer selection, prior to the non-RD mode decision process.
It hence allows proper quantization information update for a more
precise rate-distortion modeling in the non-RD mode decision.

The compression performance of speed -5 for rtc set is improved
by 2.5%, at no speed change.

Change-Id: Ic3713e8ed9185b403b5b1679d19dabd57506d452
2014-04-21 08:57:53 -07:00
Jim Bankoski
e890c2579b add a context tree structure to encoder
This patch sets up a quad_tree structure (pc_tree) for holding all of
pick_mode_context data we use at any square block size during encoding
or picking modes.  That includes contexts for 2 horizontal and 2 vertical
splits, one none, and pointers to 4 sub pc_tree nodes corresponding
to split.  It also includes a pointer to the current chosen partitioning.

This replaces code that held an index for every level in the pick
modes array including:  sb_index, mb_index,
b_index, ab_index.

These were used as stateful indexes that pointed to the current pick mode
contexts you had at each level stored in the following arrays

array ab4x4_context[][][],
sb8x4_context[][][], sb4x8_context[][][], sb8x8_context[][][],
sb8x16_context[][][], sb16x8_context[][][], mb_context[][], sb32x16[][],
sb16x32[],  sb32_context[], sb32x64_context[], sb64x32_context[],
sb64_context

and the partitioning that had been stored in the following:
b_partitioning, mb_partitioning, sb_partitioning, and sb64_partitioning.

Prior to this patch before doing an encode you had to set the appropriate
index for your block size ( switch statement),  update it ( up to 3
lookups for the index array value) and then make your call into a recursive
function at which point you'd have to call get_context which then
had to do a switch statement based on the blocksize,  and then up to 3
lookups based upon the block size to find the context to use.

With the new code the context for the block size is passed around directly
avoiding the extraneous switch statements and multi dimensional array
look ups that were listed above.   At any level in the search all of the
contexts are local to the pc_tree you are working on (in?).

In addition in most places code that used to call sub functions and
then check if the block size was 4x4 and index was > 0 and return
now don't preferring instead to call the right none function on the inside.



Change-Id: I06e39318269d9af2ce37961b3f95e181b57f5ed9
2014-04-17 07:30:55 -07:00
Paul Wilkins
e434d08fd9 Remove old activity masking code.
Delete code relating to the old VP8_TUNE_SSIM flag
as this code does not currently work and is largely made
redundant in VP9 by the various AQ modes.

Change-Id: I71f28e1f680573d296422254489000678552b17b
2014-04-16 12:22:41 -07:00
Dmitry Kovalev
ad4ce92d92 Merge "Using local variable for token_cache." 2014-03-24 15:54:38 -07:00
Jim Bankoski
35af423e8e vp9_block.h static reconverted to inline
Change-Id: I0e7d2815839d8a64250116a5486570d03659a4c0
2014-03-24 06:30:39 -07:00
Dmitry Kovalev
6b32e5f04a Using local variable for token_cache.
We use local variable for token_cache in the decoder.

Change-Id: I032763fa7894313cffe73e3f14863ae1d0527665
2014-03-21 12:14:05 -07:00
Yunqing Wang
cf07d3e332 Remove unused mode_sad
Removed mode_sad.

Change-Id: I230b42ac9b617ae2c375e297057aa0756bd355fe
2014-03-20 09:28:16 -07:00
Alex Converse
29a487c77f Add a conservative RD based active map in vp9.
Change-Id: I47b3c38aadfd8f3ea08515a18a5948aa1375c650
2014-03-10 15:48:43 -07:00
Dmitry Kovalev
3f1ab25812 Removing vp9_onyx.h and moving its content to the encoder.
Change-Id: I03451c88536bc498edddbe0cd9773ff79da085c2
2014-03-05 23:33:22 -08:00
Dmitry Kovalev
627720fa81 Cleaning up mode cost manipulations.
Change-Id: If175d97990454b171b6abeddb76d142497484487
2014-03-05 12:29:44 -08:00
Alex Converse
463ba70581 vp9_rd_pick_inter_mode_sb() reorganization
* Reduce the number of short cirtcuit checks by pre-computing and combining like checks.
* Postpone non-trivial initializations until after the shortcircuits are evaluated.
* Add some consts and const pointers.

No change to the actual results of the call or output of the encoder.

Change-Id: Ie44c4702aec6e08cfe0b8b0ba3cd6b57206478d1
2014-02-20 18:06:25 -08:00
Jingning Han
0eecccc51e Remove inactive control parameters
Change-Id: Ic5692af975fe6bd2d8ec82bbae103c6f7c2fc13e
2014-02-12 12:48:15 -08:00
Alex Converse
e78c174e54 Cleanup block_rd_txfm.
* Avoid unnecessary type erasure
* Prune unused/duplicate fields from struct rdcost_block_args
* Make struct rdcost_block_args a local

Change-Id: I4f1fd4837ccd028bbfe727191ee8d69f0463b7e5
2014-01-31 12:13:18 -08:00
Dmitry Kovalev
4264c93844 Renaming INTERPOLATION_TYPE to INTERP_FILTER.
Corresponding renames:
  subpel_kernel              => interp_kernel
  vp9_get_filter_kernel()    => vp9_get_interp_kernel()
  pred_filter_type           => pred_interp_filter
  adaptive_pred_filter_type  => adaptive_pred_interp_filter
  mcomp_filter_type          => interp_filter
  read_interp_filter_type()  => read_interp_filter()
  write_interp_filter_type() => write_interp_filter()
  fix_mcomp_filter_type()    => fix_interp_filter()

Change-Id: I1fa61fa1dc81ebbf043457c3ee2d8d4515bee6d3
2014-01-24 15:57:28 -08:00
James Zern
b453941caf vp9/encoder: add extern "C" to headers
Change-Id: I4f51ce859a97bf1b8fd2b37ac585b7c643232b69
2014-01-23 16:21:24 -08:00
Jingning Han
2f52decd22 Inter-frame non-RD mode decision
This commit setups a test framework for real-time coding. It enables
a light motion search for non-RD mode decision purpose.

Change-Id: I8bec656331539e963c2b685a70e43e0ae32a6e9d
2014-01-16 12:35:04 -08:00
Jingning Han
393a8ccef9 Remove avoid_frame_with_high_error from RD loop
The feature undergoes prior assumption that the recursive partition
size search from 4x4 to 64x64, hence utilizing information from small
blocks to determine early termination in large block rate-distortion
optimization search. The current codebase is now going from top down.
The previous function might go with not properly initialized values,
hence removed.

Tested on pedestrian_area_1080p at 4000 kbps running under speed 2.
No visible difference in runtime observed.

Change-Id: I553df415c6191413762db7ae34e8790c71d8118e
2014-01-06 13:34:07 -08:00
Dmitry Kovalev
5b04962cf4 Merging best_ref_mv and second_best_ref_mv into best_ref_mv[2].
Change-Id: If04b57828847cee09a79c94e1098d1aa4990ea0d
2014-01-03 11:31:00 -08:00
Jingning Han
a4ce53f14d Adaptive motion control on ref and search range
This commit takes a preliminary attempt to refine the motion search
control. It detects the SAD associated with mv predictor per reference
frame, and based on which to determine whether the encoder wants to
reduce the motion search range (if the predicted mv provides fairly
small SAD), or to skip the current reference frame (if there exists
another ref frame that gives much smaller SAD cost).

This feature is turned on in the settings of speed 1 and above.

In speed 1, compression performance changed
derf  -0.018%
yt    -0.043%
hd    -0.045%
stdhd -0.281%

speed-up
pedestrian_area_1080p at 4000 kbps 100 frames
199651ms -> 188846ms (5.5% speed-up)
blue_sky_1080p at 6000 kbps
443531ms -> 415239ms (6.3% speed-up)

In speed 2, compression performance changed
derf  -0.026%
yt    -0.090%
hd    -0.055%
stdhd -0.210%

speed-up
pedstrian 113949ms -> 108855ms (4.5% speed-up)
blue_sky  271057ms -> 257322ms (5% speed-up)

Change-Id: I1b74ea28278c94fea329d971d706d573983d810d
2013-12-27 12:43:06 -08:00
Jingning Han
243327f43c Store the SSE of prediction residuals
Buffer the SSE of prediction residuals in the rate-distortion
optimization loop of a given block. This information would be used
for later encoding control.

Change-Id: If4e63f3462490513c48be9407d3327c8dd438367
2013-12-19 18:45:28 -08:00
Jingning Han
3b5a90bd86 Enable adaptive pred filter type for sub8x8
This commit enables an adaptive prediction filter type selection
for sub8x8 block sizes. In speed 1, it re-uses the filter type of
collocated 8x8 block if it is tested in the rate-distortion optimization
loop, for the sub8x8 blocks. Otherwise, it runs the normal test
over all the three filter types. In speed 2, it re-uses the 8x8
block's prediction filter type, if available. Otherwise, force it
to be EIGHTTAP.

Compression and speed performance wise:
speed 1
derf -0.266%
yt   -0.138%

bus at 2000 kbps: 33766ms -> 30451ms (10% speed-up)
football at 600 kbps: 48173ms -> 43786ms (9% speed-up)

speed 2
derf -0.026%
yt   +0.134%

bus at 2000 kbps: 18973ms -> 17698ms (6% speed-up)
football at 600 kbps: 26748ms -> 25096ms (6% speed-up)

Change-Id: I77e097533b969fd3472147225fa79fc98095d342
2013-12-12 17:54:34 -08:00
Dmitry Kovalev
d6b159d4a6 Removing BLOCK_TYPES and adding PLANE_TYPES constant instead.
Change-Id: Ic3bb862e93aedf6a489a33ea6f7e5097d96855ee
2013-12-06 10:54:00 -08:00
Dmitry Kovalev
377fa8aff8 Renaming PREV_COEF_CONTEXTS to COEFF_CONTEXTS.
Also adding BAND_COEFF_CONTEXTS macro to simplify for loop logic.

Change-Id: I12a78a49cf1addf81e6b3fe2a3736ec2b79bd79e
2013-12-05 17:08:06 -08:00
Dmitry Kovalev
3712b58c2f Merge "Cleaning up vp9_entropy.h file." 2013-12-04 16:46:41 -08:00
Dmitry Kovalev
f00d157c12 Moving eob array to the encoder.
In the decoder we don't need to save eobs, we can pass eob as an argument.
That's why removing eob arrays from VP9Decompressor and TileWorkerData,
and moving eob pointer from macroblockd_plane to macroblock_plane.

Change-Id: I8eb919acc837acfb3abdd8319af63d1bbca8217a
2013-12-03 17:59:32 -08:00
Dmitry Kovalev
8e89e2f2e0 Cleaning up vp9_entropy.h file.
Renaming constants for consistency:
  DCT_VAL_CATEGORY1 => CATEGORY1_TOKEN
  DCT_VAL_CATEGORY2 => CATEGORY2_TOKEN
  DCT_VAL_CATEGORY3 => CATEGORY3_TOKEN
  DCT_VAL_CATEGORY4 => CATEGORY4_TOKEN
  DCT_VAL_CATEGORY5 => CATEGORY5_TOKEN
  DCT_VAL_CATEGORY6 => CATEGORY6_TOKEN
  DCT_EOB_TOKEN     => EOB_TOKEN
  DCT_EOB_MODEL_TOKEN => EOB_MODEL_TOKEN
  MAX_ENTROPY_TOKENS => ENTROPY_TOKENS

Moving constants:
  INTER_MODE_CONTEXTS from vp9_entropy.h to vp9_blockd.h.
  EOSB_TOKEN from vp9_entropy.h to vp9_tokenize.h

Change-Id: I5fcbf081318e1d365792b6d290a930c6cb0f3fc2
2013-12-03 17:23:03 -08:00
Dmitry Kovalev
d3a2e55af4 Removing qcoeff buffers from the decoder.
We only need qcoeff buffers in the encoder. Reducing TileWorkerData struct
and VP9Decompressor struct sizes by 24K.

Change-Id: Id148868461f7ffa3d3dd634b371503ae9c57e207
2013-11-26 18:52:10 -08:00
Deb Mukherjee
cfcd5c4f61 Simplifies band-getting with a static array
Simplifies the code by implementing band mapping with static arrays.
A lot of the code complexity introduced in a previous patch
disappears.

Change-Id: Ia3fac36e594fb5ad2d55ae141c58bba4c55c2d28
2013-11-13 22:15:16 -08:00
Jingning Han
b6b9143218 Dual buffer encoding for intra modes
Overall change (using dual buffer scheme for superblocks of both inter
and intra modes) reduces speed 2 runtime:
bluesky_1080p at 6000kbps:   263553ms -> 257441ms
riverbed_1080p at 8000kbps:  233230ms -> 225308ms.

Change-Id: Idf8d70f768a4b0d97b2a8506372c57b7b4022119
2013-11-13 12:57:03 -08:00
Dmitry Kovalev
3f3d14e1d3 Moving q_index from MACROBLOCKD to MACROBLOCK.
Moving because q_index is used only by encoder.

Change-Id: I0b96175614ed4fd3d76ee56a0ba36258e1e896f6
2013-11-12 18:13:19 -08:00
Jingning Han
e69461593d Merge "Enable dual buffer rd search and encoding scheme" 2013-11-12 18:11:41 -08:00
Dmitry Kovalev
3a2ea76469 Merge "Moving {sb, mb, b, ab}_index from MACROBLOCKD to MACROBLOCK." 2013-11-12 15:59:28 -08:00
Deb Mukherjee
5ade423774 Removes conditional statements from band getting
Implements scan order to band map with arrays in both the encoder
and decoder to remove conditional statements.

Encoding seems to be about 1% faster at speed 0, tested on football.
Decoding seems to be about 0.5-1% faster on a set of 25 videos.

Change-Id: Idb233ca0b9e0efd790e30880642e8717e1c5c8dd
2013-11-12 10:13:27 -08:00
Jingning Han
34b6abefa2 Enable dual buffer rd search and encoding scheme
This commit enables the dual buffer rate-distortion optimization
and encoding scheme. It stacks the original transform coefficients,
quantized levels, and reconstructed coefficients, in the rate-
distortion optimization search process, hence eliminates the need
to re-run residual generation, forward transform, and quantization
in the encoding stage.

Change-Id: I011bfad3a59a380a869ee552e91dae0394ec492e
2013-11-11 18:32:55 -08:00
Jingning Han
3b3aea6834 Allocate dual buffer sets for encoding
Allocate memory space of dual buffer sets that store the coeff, qcoeff,
dqcoeff, and eobs. Connect the pointers of macroblock_plane and
macroblockd_plane to the actual buffer in use accordingly.

Change-Id: I2f0b5f482ca879fae39095013eaf8901db20a5a4
2013-11-11 16:24:39 -08:00
Dmitry Kovalev
3551e25099 Moving {sb, mb, b, ab}_index from MACROBLOCKD to MACROBLOCK.
We use {sb, mb, b, ab}_index only inside encoder, so moving them into
appropriate data structure.

Change-Id: Ib5c1036716354d9d321e11a60c1634c1cb8f9716
2013-11-11 15:58:57 -08:00
Deb Mukherjee
8810db303c Removes stack allocation of token_cache
Removes stack-allocation of token_cache in the tokenize
function in the encoder.

Change-Id: Ifdbab76dc2b23384da0080d2e9390533e489db8c
2013-11-06 11:50:27 -08:00
Dmitry Kovalev
6761872e49 Replacing (SWITCHABLE_FILTERS + 1) with SWITCHABLE_FILTER_CONTEXTS.
Change-Id: I9781a62bc1a4cd9176554d1271d87dbcafda9cb0
2013-10-30 14:40:34 -07:00
Dmitry Kovalev
600a3860a4 Making input pointer constant for all fdct/fht functions.
Change-Id: I78f7012f967a777ddd39bae6671eb501df6bbfe8
2013-10-24 11:48:25 -07:00