Commit Graph

666 Commits

Author SHA1 Message Date
Ronald S. Bultje
38d7945345 Slight simplification of SB RD loop recursion conditions.
Change-Id: I87a406fcd18ab043253ca0c009d1182fdc5c3046
2013-04-11 09:14:55 -07:00
Ronald S. Bultje
4eb537c0e6 A few more cases where sb_type was used arithmetically.
With these fixed, the codec produces identical results regardless of
what literal values are used for the enum members in BLOCK_SIZE_*.

Change-Id: I26db8e08019b58ba432af1f0950ebe6b0eb4ad8c
2013-04-10 18:04:57 -07:00
Ronald S. Bultje
33d94a843f Remove copying of coefficients and predictor in i8x8 RD loop.
The resulting values are never used.

Change-Id: I688caf30da9aab87aa280cce913eda4f33172293
2013-04-10 17:39:03 -07:00
Ronald S. Bultje
8fb5be48a6 Make usage of sb_type independent of literal values.
Change-Id: I0d12f9ef9d960df0172a1377f8e5236eb6d90492
2013-04-10 17:38:57 -07:00
Ronald S. Bultje
b4f6098ef7 Make RD superblock mode search size-agnostic.
Merge various super_block_yrd and super_block_uvrd versions into one
common function that works for all sizes. Make transform size selection
size-agnostic also. This fixes a slight bug in the intra UV superblock
code where it used the wrong transform size for txsz > 8x8, and stores
the txsz selection for superblocks properly (instead of forgetting it).
Lastly, it removes the trellis search that was done for 16x16 intra
predictors, since trellis is relatively expensive and should thus only
be done after RD mode selection.

Gives basically identical results on derf (+0.009%).

Change-Id: If4485c6f0a0fe4038b3172f7a238477c35a6f8d3
2013-04-10 16:50:30 -07:00
Yaowu Xu
8e9819230d Merge "Remove obselete code" into experimental 2013-04-10 14:56:28 -07:00
Yaowu Xu
2da90fddc2 Remove obselete code
The strategy to run fast loop filter picking for encoder speed-up
should be revisited at a later stage.

Change-Id: I3b75e06d767cff41be952a42e63b3292f4eab996
2013-04-10 13:45:22 -07:00
Dmitry Kovalev
0cef7234e1 Merge "Fixing upper case names." into experimental 2013-04-10 13:29:38 -07:00
Dmitry Kovalev
1c6df34c06 Merge "Code cleanup in bitstream code." into experimental 2013-04-10 10:18:50 -07:00
Ronald S. Bultje
1932828d19 Merge "Make SB coding size-independent." into experimental 2013-04-10 08:51:58 -07:00
Ronald S. Bultje
a3874850dd Make SB coding size-independent.
Merge sb32x32 and sb64x64 functions; allow for rectangular sizes. Code
gives identical encoder results before and after. There are a few
macros for rectangular block sizes under the sbsegment experiment; this
experiment is not yet functional and should not yet be used.

Change-Id: I71f93b5d2a1596e99a6f01f29c3f0a456694d728
2013-04-09 21:28:27 -07:00
Dmitry Kovalev
c34f6fcb54 Fixing upper case names.
Renaming Y1dequant to y_dequant, UVdequant to uv_dequant, QIndex to qindex.

Change-Id: I1c356e5f886deb3f8807dc212de9799b55b09d58
2013-04-09 10:46:57 -07:00
Dmitry Kovalev
d1cff2deb1 Code cleanup in bitstream code.
Lower case variable names, less code.

Change-Id: I1abc8f592ad2343ab5c76fe2d16262741a4a894a
2013-04-08 19:07:29 -07:00
Jingning Han
12bf0796e6 Clamp inferred motion vectors only
Clamp only the motion vectors inferred from neighboring reference
macroblocks. The motion vectors obtained through motion search in
NEWMV mode are constrained during the search process, which allows
a relatively larger referencing region than the inferred mvs.
Hence further clamping the best mv provided by the motion search may
affect the efficacy of NEWMV mode.

Synchronized the decoding process. The decoded mvs in NEWMV modes
should be guaranteed to fit in the effective range. Put a mv range
clamping function there for security purpose.

This improves the coding performance of high motion sequences, e.g.,
derf set:
foreman 0.233%
husky   0.175%
icd     0.135%
mother_daughter 0.337%
pamphlet        0.561%

stdhd set:
blue_sky 0.408%
city     0.455%
also saw sunflower goes down by -0.469%.

Change-Id: I3fcbba669e56dab779857a8126a91b926e899cb5
2013-04-08 11:37:03 -07:00
John Koleszar
fa135d7b9e Merge changes Ibbfa68d6,Idb76a0e2 into experimental
* changes:
  Move EOB to per-plane data
  Move qcoeff, dqcoeff from BLOCKD to per-plane data
2013-04-05 15:56:50 -07:00
Ronald S. Bultje
9161127ee9 Merge "Remove full-pixel-related code." into experimental 2013-04-05 13:46:07 -07:00
Ronald S. Bultje
36c3a67c20 Remove full-pixel-related code.
This is a VP8-only feature (part of profile 3) that is unsupported in
VP9.

Change-Id: I78016eede8d9c834d44d4c517f3e8b8fc2a378b1
2013-04-05 12:50:19 -07:00
Yaowu Xu
e79a3ff5f3 Removed a speed feature no longer used
Change-Id: Id0c2e44daa936f1d6fb76469fd1bd72a4d7c19fd
2013-04-05 10:43:20 -07:00
John Koleszar
05a79f2fbf Move EOB to per-plane data
Continue migrating data from BLOCKD/MACROBLOCKD to the per-plane
structures.

Change-Id: Ibbfa68d6da438d32dcbe8df68245ee28b0a2fa2c
2013-04-04 21:30:23 -07:00
John Koleszar
4c05a051ab Move qcoeff, dqcoeff from BLOCKD to per-plane data
Start grouping data per-plane, as part of refactoring to support
additional planes, and chroma planes with other-than 4:2:0
subsampling.

Change-Id: Idb76a0e23ab239180c818025bae1f36f1608bb23
2013-04-04 16:30:57 -07:00
Yaowu Xu
9780d58e94 make one_shot_q an experiment
so it is configurable to faciliate testings

Change-Id: I247b62736c3a08ec2934793959d1ae605a05efa3
2013-04-04 14:14:51 -07:00
Deb Mukherjee
73031aaa7d Bugfix in encode_inter_mb_segment_8x8
Fixes an indexing bug. Looks like the bug has been there for a while.

Change-Id: I9fc04b0c30754bcb47366ad94a08112925600c4d
2013-04-04 11:07:19 -07:00
Paul Wilkins
9b9136f8a2 Fixed incorrect use of compute_qdelta()
This function expects real Q values as inputs
not index values.

The use-age her impacts the Q chosen for force key
frames. Though this is a bug fix I have not yet verified
whether following the bug fix the q multiplier value used is
correct.

Change-Id: I49f6da894d90baeb1e86c820c335f02dc80d3b66
2013-04-04 10:19:16 +01:00
Yunqing Wang
4ca882f32f Modify vp9_setup_interp_filters function
Took vp9_setup_scale_factors_for_frame() out from
vp9_setup_interp_filters(), so that it is only called once per
frame instead of per macroblock. Decoder tests showed a 1.5%
performance gain.

Change-Id: I770cb09eb2140ab85132f82aed388ac0bdd3a0aa
2013-04-03 13:49:55 -07:00
John Koleszar
8b71b8a6de Merge "Renaming sb32_coded and sb64_coded fields." into experimental 2013-04-02 21:49:03 -07:00
John Koleszar
ede03dfa48 Merge "Code cleanup in vp9_onyx_if.c." into experimental 2013-04-02 20:16:56 -07:00
Dmitry Kovalev
dca8ad178c Renaming sb32_coded and sb64_coded fields.
Renaming sb32_coded to prob_sb32_coded and sb64_coded to prob_sb64_coded.

Change-Id: I6de5cad00a57c3e066d53467f8c38cb6073dce11
2013-04-02 18:21:55 -07:00
Dmitry Kovalev
6f53eee531 Code cleanup in vp9_onyx_if.c.
Using clamp and MIN/MAX functions instead of plain C code. Lower case
variable names. Removing redundant parenthesis.

Change-Id: Ibf7cc5fbe4fbdb5029049a599af71534176e6f42
2013-04-02 10:24:56 -07:00
John Koleszar
49bc402a94 Merge "Code cleanup." into experimental 2013-04-01 21:12:56 -07:00
John Koleszar
a417a6e32c Merge "Removing redundant function arguments." into experimental 2013-04-01 21:09:48 -07:00
Dmitry Kovalev
50e54c112d Code cleanup.
Adding multiple16 function, removing redundant code, better formatting.

Change-Id: I50195b78ac8ab803e3d05c8fb05a7ca134fab386
2013-04-01 18:23:04 -07:00
Ronald S. Bultje
cdac4ad4e6 Merge "Calculate SSIM over both reconstruction as well as postproc buffer." into experimental 2013-04-01 17:22:29 -07:00
Ronald S. Bultje
6dd6ffb0bb Calculate SSIM over both reconstruction as well as postproc buffer.
We used to calculate SSIM only over the postproc buffer, whereas we
calculate PSNR for both. Compared to postproc-SSIM, this is about 0.3%
higher for derf, 1.4% lower for hd and 0.5% lower for stdhd, although
it is highly variable on a per-clip basis.

Change-Id: I8dd491f0f5b4201dedfb15d288c854d5d4caa10f
2013-04-01 09:10:27 -07:00
Deb Mukherjee
e3955007df Merge "Framework changes in nzc to allow more flexibility" into experimental 2013-03-29 15:57:27 -07:00
John Koleszar
2e181c2d0b Merge "General code cleanup." into experimental 2013-03-29 10:40:34 -07:00
Paul Wilkins
0b4deea896 Merge "Adjust mv_ratio_accumulator threshold." into experimental 2013-03-28 12:53:23 -07:00
Ronald S. Bultje
6cb2fcf601 Merge "Fix mix-up in pt token indexing." into experimental 2013-03-28 12:53:00 -07:00
Yaowu Xu
e071fe15b2 Merge "Fix crash when --tune=ssim is selected." into experimental 2013-03-28 11:23:44 -07:00
Ronald S. Bultje
ed78d1439f Merge "Save nzcstats." into experimental 2013-03-28 09:36:58 -07:00
Deb Mukherjee
fe9b5143ba Framework changes in nzc to allow more flexibility
The patch adds the flexibility to use standard EOB based coding
on smaller block sizes and nzc based coding on larger blocksizes.
The tx-sizes that use nzc based coding and those that use EOB based
coding are controlled by a function get_nzc_used().
By default, this function uses nzc based coding for 16x16 and 32x32
transform blocks, which seem to bridge the performance gap
substantially.

All sets are now lower by 0.5% to 0.7%, as opposed to ~1.8% before.

Change-Id: I06abed3df57b52d241ea1f51b0d571c71e38fd0b
2013-03-28 09:33:50 -07:00
Ronald S. Bultje
9eea9fa206 Fix mix-up in pt token indexing.
This fixes uninitialized reads in the trellis, and probably makes the
trellis do something again.

Change-Id: Ifac8dae9aa77574bde0954a71d4571c5c556df3c
2013-03-28 09:24:29 -07:00
Paul Wilkins
17ef6a8dfd Adjust mv_ratio_accumulator threshold.
This threshold effectively limits the amount of motion
from one end of a GF/ARF group to the other.
This patch makes the threshold depend on image size.

Change-Id: Id45d1d7bced815f86ddd037be53164894b00b82f
2013-03-28 12:49:02 +00:00
Paul Wilkins
befb0393c5 Fix crash when --tune=ssim is selected.
Crash fix only. No functional change or testing.

Change-Id: I0c6d114d024c29fc11ae61666f5938f11b01dd6a
2013-03-28 12:48:30 +00:00
Dmitry Kovalev
17cddb4e26 Removing redundant function arguments.
Almost all arguments for vp9_build_inter32x32_predictors_sb and
vp9_build_inter64x64_predictors_sb can be deduced from the first macroblock
argument.

Change-Id: I5d477a607586d05698d5b3b9b9bc03891dd3fe83
2013-03-27 16:19:27 -07:00
Dmitry Kovalev
180cd5faa5 General code cleanup.
Removing redundant code, lower case variable names, better indentation,
better parameter names, adding const to readonly parameters.

Change-Id: Ibfdee00f60316fdc5b3f024028c7aaa76a627483
2013-03-27 14:22:30 -07:00
John Koleszar
28d9202ed4 Merge "Cleaning up rate control code." into experimental 2013-03-27 12:29:00 -07:00
Ronald S. Bultje
35dc9f5546 Save nzcstats.
Change-Id: I4a3a9eb9f9d17218a0f0d7e148123d34dae879c2
2013-03-27 09:44:47 -07:00
Ronald S. Bultje
7c70145914 Merge "Add col/row-based coefficient scanning patterns for 1D 8x8/16x16 ADSTs." into experimental 2013-03-26 19:17:08 -07:00
Ronald S. Bultje
3c77ab4c0f Merge "Redo banding for all transforms." into experimental 2013-03-26 19:16:44 -07:00
Ronald S. Bultje
c6efbbcfe4 Merge "Use above/left (instead of previous in scan-order) as token context." into experimental 2013-03-26 19:16:24 -07:00
Deb Mukherjee
23144d2345 Implicit weighted prediction experiment
Adds an experiment to use a weighted prediction of two INTER
predictors, where the weight is one of (1/4, 3/4), (3/8, 5/8),
(1/2, 1/2), (5/8, 3/8) or (3/4, 1/4), and is chosen implicitly
based on consistency of the predictors to the already
reconstructed pixels to the top and left of the current macroblock
or superblock.

Currently the weighting is not applied to SPLITMV modes, which
default to the usual (1/2, 1/2) weighting. However the code is in
place controlled by a macro. The same weighting is used for Y and
UV components, where the weight is derived from analyzing the Y
component only.

Results (over compound inter-intra experiment)
derf: +0.18%
yt: +0.34%
hd: +0.49%
stdhd: +0.23%

The experiment suggests bigger benefit for explicitly signaled weights.

Change-Id: I5438539ff4485c5752874cd1eb078ff14bf5235a
2013-03-26 16:58:56 -07:00
Ronald S. Bultje
d9094d8fd3 Add col/row-based coefficient scanning patterns for 1D 8x8/16x16 ADSTs.
These are mostly just for experimental purposes. I saw small gains (in
the 0.1% range) when playing with this on derf.

Change-Id: Ib21eed477bbb46bddcd73b21c5c708a5b46abedc
2013-03-26 16:46:13 -07:00
Ronald S. Bultje
3120dbddb1 Redo banding for all transforms.
Now that the first AC coefficient in both directions use the same DC
as their context, there no longer is a purpose in letting both have
their own band. Merging these two bands allows us to split bands for
some of the very high-frequency AC bands.

In addition, I'm redoing the banding for the 1D-ADST col/row scans. I
don't think the old banding made any sense at all (it merged the last
coefficient of the first row/col in the same band as the first two of
the second row/col), which was clearly an oversight from the band being
applied in scan-order (rather than in their actual position). Now,
coefficients at the same position will be in the same band, regardless
what scan order is used. I think this makes most sense for the purpose
of banding, which is basically "predict energy for this coefficient
depending on the energy of context coefficients" (i.e. pt).

After full re-training, together with previous patch, derf gains about
1.2-1.3%, and hd/stdhd gain about 0.9-1.0%.

Change-Id: I7a0cc12ba724e88b278034113cb4adaaebf87e0c
2013-03-26 16:46:13 -07:00
Ronald S. Bultje
790fb13215 Use above/left (instead of previous in scan-order) as token context.
Pearson correlation for above or left is significantly higher than for
previous-in-scan-order (absolute values depend on position in scan, but
in general, we gain about 0.1-0.2 by using either above or left; using
both basically just makes this even better). For eob branch skipping,
we continue to use the previous token in scan order.

This helps about 0.9% on derf after re-training on a limited data set.
Full re-training and results on larger-resolution clips are pending.

Note that this commit breaks trellis, so we can probably get further
gains out of it by fixing trellis at some later point.

Change-Id: Iead68e296fc3a105cca746b5e3da9555d6010cfe
2013-03-26 16:46:09 -07:00
Dmitry Kovalev
77c664ade3 Cleaning up rate control code.
Lower case variable names, declaration and initialization on the same line,
removing redundant casts to double.

Change-Id: I7ea3905bed827aa6faac11a78401b85e448b57f9
2013-03-26 11:25:58 -07:00
John Koleszar
441e2eab1b Add an in-loop deringing experiment
Adds a per-frame, strength adjustable, in loop deringing filter. Uses
the existing vp9_post_proc_down_and_across 5 tap thresholded blur
code, with a brute force search for the threshold.

Results almost strictly positive on the YT HD set, either having no
effect or helping PSNR in the range of 1-3% (overall average 0.8%).
Results more mixed for the CIF set, (-0.5 min, 1.4 max, 0.1 avg).
This has an almost strictly negative impact to SSIM, so examining a
different filter or a more balanced search heuristic is in order.

Other test set results pending.

Change-Id: I5ca6ee8fe292dfa3f2eab7f65332423fa1710b58
2013-03-26 08:23:24 -07:00
Deb Mukherjee
49dcc71493 Merge "Modeling default coef probs with distribution" into experimental 2013-03-26 07:13:13 -07:00
Deb Mukherjee
fd18d5dffe Modeling default coef probs with distribution
Replaces the default tables for single coefficient magnitudes with
those obtained from an appropriate distribution. The EOB node
is left unchanged. The model is represeted as a 256-size codebook
where the index corresponds to the probability of the Zero or the
One node. Two variations are implemented corresponding to whether
the Zero node or the One-node is used as the peg. The main advantage
is that the default prob tables will become considerably smaller and
manageable. Besides there is substantially less risk of over-fitting
for a training set.

Various distributions are tried and the one that gives the best
results is the family of Generalized Gaussian distributions with
shape parameter 0.75. The results are within about 0.2% of fully
trained tables for the Zero peg variant, and within 0.1% of the
One peg variant.

The forward updates are optionally (controlled by a macro)
model-based, i.e. restricted to only convey probabilities from the
codebook. Backward updates can also be optionally (controlled by
another macro) model-based, but is turned off by default. Currently
model-based forward updates work about the same as unconstrained
updates, but there is a drop in performance with backward-updates
being model based.

The model based approach also allows the probabilities for the key
frames to be adjusted from the defaults based on the base_qindex of
the frame. Currently the adjustment function is a placeholder that
adjusts the prob of EOB and Zero node from the nominal one at higher
quality (lower qindex) or lower quality (higher qindex) ends of the
range. The rest of the probabilities are then derived based on the
model from the adjusted prob of zero.

Change-Id: Iae050f3cbcc6d8b3f204e8dc395ae47b3b2192c9
2013-03-25 23:43:38 -07:00
Paul Wilkins
99a4939ec3 Merge "Disable zero bin mode boost." into experimental 2013-03-22 10:59:43 -07:00
Paul Wilkins
bfe7666142 Merge "Minor code clean up" into experimental 2013-03-22 10:53:12 -07:00
Paul Wilkins
ec080fa9de Disable zero bin mode boost.
As things stand the zero bin mode boost is hurting somewhat.
In part this seems to be because the boost applied as is
interferes with the rd mode selection loop.

Average gains (derf 0.072, yt 0.243, ythd 0.179 std-hd 0.212%)

Change-Id: Icaecea3908d9a7352370e49b8fa822f2c2c49dc1
2013-03-22 17:43:43 +00:00
Paul Wilkins
815734e5fb Minor code clean up
Change-Id: Ifa864e0acb253b238b03cdeed0fe5d6ee30a45d8
2013-03-22 17:42:45 +00:00
Dmitry Kovalev
56f3a2c663 Code cleanup: lower case variable names.
Renaming Width to width, Height to height and Version to version in
several structs and function signatures.

Change-Id: I084c3f7e747cb2ce3345aff27a3dff9b13a87543
2013-03-20 16:41:30 -07:00
Paul Wilkins
d8ffee4526 Changes to rd error_per_bit calculation.
Specifically changes to retain more precision
especially at low Q through to the point of use.

Change-Id: Ief5f010f2ca4daaabef49520e7edb46c35daf397
2013-03-18 23:07:51 +00:00
Ronald S. Bultje
a5b54d73e4 Merge "Fix ENTROPY_STATS code in vp9_tokenize.c." into experimental 2013-03-18 15:58:33 -07:00
Ronald S. Bultje
b99dce6881 Fix ENTROPY_STATS code in vp9_tokenize.c.
Change-Id: I9b4cb1e2ce6c6a99cffd473ff2fa7579bd318fcd
2013-03-18 15:39:04 -07:00
Yunqing Wang
6344c84c82 Optimize 8x8 idct function
Wrote sse2 functions of vp9_short_idct8x8 and vp9_short_idct10_8x8.
Compared to c version, the sse2 version is 2X faster. The decoder
test didn't show noticeable gain since 8x8 idct doesn't take much
of decoding time (less than 1% in my test).

Change-Id: I56313e18cd481700b3b52c4eda5ca204ca6365f3
2013-03-18 15:34:14 -07:00
John Koleszar
93529bd7c1 Merge "Replace scaling byte with explicit display size" into experimental 2013-03-18 13:02:07 -07:00
John Koleszar
8a3f55f2d4 Replace scaling byte with explicit display size
If the intended display size is different than the size the frame is
coded at, then send that size explicitly in the bitstream. Adds a new
bit to the frame header to indicate whether the extra size fields
are present.

Change-Id: I525c66f22d207efaf1e5f903c6a2a91b80245854
2013-03-18 12:02:20 -07:00
Paul Wilkins
ef179bce61 Merge "Adapt ARNR filter length and strength." into experimental 2013-03-18 12:00:39 -07:00
John Koleszar
c5b317057b Merge "Fix pulsing issue with scaling" into experimental 2013-03-18 11:57:36 -07:00
John Koleszar
e5d7542447 Merge "Add VP9_GET_REFERENCE control" into experimental 2013-03-18 11:57:31 -07:00
Paul Wilkins
cdb322dd72 Adapt ARNR filter length and strength.
Adjust the filter length and strength for each
ARF group based on a measure of difficulty (the boost)
and the active q range.

Remove lower limit on RDMULT value.

Average gains on the different sets in range 0.4%-0.9%.
However the ARNR changes give a very big boost on a
few clips.

Eg. Soccer ~5%, in derf set and Cyclist ~ 10% in the std-hd set

Change-Id: I2078d78798e27ad2bcc2b32d703ea37b67412ec4
2013-03-18 16:17:04 +00:00
Yaowu Xu
12ade55719 Merge "removed reference to "LLM" and "x8"" into experimental 2013-03-18 08:51:19 -07:00
John Koleszar
9a56ea7e46 Merge "Remove some unused rate control variables" into experimental 2013-03-18 08:36:23 -07:00
John Koleszar
571fce6546 Merge "Fix use of NaN in firstpass" into experimental 2013-03-18 08:36:18 -07:00
Deb Mukherjee
bf7387f6b7 Merge "Context-pred fix to not use top/left on edges" into experimental 2013-03-16 19:09:25 -07:00
Deb Mukherjee
b1921b2f08 Context-pred fix to not use top/left on edges
This fix resolves some of the mismatch issues being seen
recently. While this is the right thing to do when tiling
is used for this experiment, it is not the underlying cause
of the the mismatches.
Something else is causing writing outside of the allowable
frame area in the encoder leading to this mismatch.

Change-Id: If52c6f67555aa18ab8762865384e323b47237277
2013-03-16 09:26:52 -07:00
John Koleszar
b8ac9f2f2c Remove some unused rate control variables
These variables are unused, and are subject to overflowing, causing
assertions when built with -ftrapv.

Change-Id: Ia00a3201af309906c05bcd4b23a643925ed6ea86
2013-03-15 17:53:45 -07:00
John Koleszar
db5f2cb57b Fix use of NaN in firstpass
If the second reference is better than the first in the long term,
it was possible to try to take the fractional exponent of a
negative number, giving an undefined result.

Change-Id: I1dd08286747ceae960eb03bb5d98a383cc9d253b
2013-03-15 17:53:38 -07:00
Christian Duvivier
4418b790a7 Faster vp9_short_fdct16x16.
Scalar path is about 1.5x faster (3.1% overall encoder speedup).
SSE2 path is about 7.2x faster (7.8% overall encoder speedup).

Change-Id: I06da5ad0cdae2488431eabf002b0d898d66d8289
2013-03-15 15:55:31 -07:00
Yaowu Xu
82fe8c9f36 Merge "force lossless coding at very high quality end" into experimental 2013-03-14 19:05:17 -07:00
Yaowu Xu
5d9ba7938e Merge "Remove leftover reference to 2nd order dc/ac quant" into experimental 2013-03-14 19:05:11 -07:00
Yaowu Xu
374a17366e force lossless coding at very high quality end
Change-Id: I75fc4eee10bee9efd419d248827290cce8e6d637
2013-03-14 12:31:27 -07:00
Yaowu Xu
f4d2ad6915 Remove leftover reference to 2nd order dc/ac quant
Change-Id: Ib8dacf1d2797743569771b8f699e40e1aeb085cb
2013-03-14 10:46:15 -07:00
John Koleszar
9b7be88883 Fix pulsing issue with scaling
Updates the YV12_BUFFER_CONFIG structure to be crop-aware. The
exiting width/height parameters are left unchanged, storing the
width and height algined to a 16 byte boundary. The cropped
dimensions are added as new fields.

This fixes a nasty visual pulse when switching between scaled and
unscaled frame dimensions due to a mismatch between the scaling
ratio and the 16-byte aligned sizes.

Change-Id: Id4a3f6aea6b9b9ae38bdfa1b87b7eb2cfcdd57b6
2013-03-13 19:10:10 -07:00
John Koleszar
b3c350a1a9 Add VP9_GET_REFERENCE control
This is like VP8_COPY_REFERENCE, but returns a pointer to the reference
frame rather than a copy of it. This is useful when the application
doesn't know what the size of the reference is, as is the case when
scaling is in effect.

Change-Id: I63667109f65510364d0e397ebe56217140772085
2013-03-13 19:08:06 -07:00
Jingning Han
76c12ab9c9 Support +/-2048 motion vector coding
Enable entropy coding of motion vectors up to +/-2048. Also
extend the motion search range accordingly.

Change-Id: Iac2bb015e8934521cef83a19edbe967d9f097436
2013-03-13 14:08:27 -07:00
Yaowu Xu
005552639b removed reference to "LLM" and "x8"
The commit changed the name of files and function to remove obselete
reference to LLM and x8.

Change-Id: I973b20fc1a55149ed68b5408b3874768e6f88516
2013-03-13 08:35:46 -07:00
Paul Wilkins
a2c6f6e945 Merge "disambiguate superframe index in vp9_stop_encode()" into experimental 2013-03-12 16:00:30 -07:00
John Koleszar
872fc3ded8 disambiguate superframe index in vp9_stop_encode()
If the bool-coded partition naturally ends in a byte that matches the
superframe index marker, it could lead to a parse error. This commit
ensures that if such a marker is seen, it is padded out with an
additional zero byte to disambiguate it.

Change-Id: Id977de05745b6fa9ef08afb71e210a2a3ecca02e
2013-03-12 14:30:23 -07:00
Paul Wilkins
21ba242bfd Merge "Change buffer update rules on ARF overlay." into experimental 2013-03-12 11:07:20 -07:00
Paul Wilkins
49d1425d19 Merge "Changes to maximum gf/arf interval." into experimental 2013-03-12 09:59:43 -07:00
Paul Wilkins
8be3056c45 Change buffer update rules on ARF overlay.
When coding the frame that corresponds to the midpoint frame
defining an ARF, do not update the last reference frame buffer.
Previously this buffer was updated meaning that when coding the next
ARF all the reference buffers were the same (or nearly so).
Turning the update off means that the frame before is still available
as an alternative predictor and for use in compound prediction.

Also fixed inconsistency in test for mismatch (patch from JK).

Net average gains (derf 0.049, yt 0.163, yt-hd 0.207, std-hd 0.286)

Change-Id: Ifee21da21ccbb1648ac2eafe890d3ce60562c7bc
2013-03-12 16:57:39 +00:00
Dmitry Kovalev
ff553ba113 Merge "Code cleanup." into experimental 2013-03-11 17:22:21 -07:00
Dmitry Kovalev
2891d70b23 Code cleanup.
Removing redundant code, introducing new functions for better
decomposition, adding 'clamp' function to vp9_common.h.

Change-Id: Ic3b8ca13bbc38f60f0c9c43910b5802005e31aaf
2013-03-11 17:02:27 -07:00
John Koleszar
a07eb47b25 Merge "Reinitialize motion search tables on frame size change" into experimental 2013-03-11 16:32:03 -07:00
John Koleszar
0a18228274 Merge "Add 'superframe' index" into experimental 2013-03-11 16:31:48 -07:00
Paul Wilkins
08d2c3829a Changes to maximum gf/arf interval.
This patch puts in an adjustment to the maximum gf/arf
interval based on the active q range.  It sets a fixed
baseline maximum of 16 but can drop this down to 12 at
lower q. This required some re-ordering in the first pass
code to insure we have a Q range estimate before defining
the first gf sequence.

The main gains seed are int he STD hd set on 50fps clips
where previously the interval could rise as high as 25.
On the std hd clip the gains are around 2.8% with limit set
to 300 frames.

When combined with the one shot rate control flags we get
combined of:

derf 1.55% (limit300), yt 7.25%, hd 5.17% std-hd 5.84% (limit300)

Change-Id: Ib380d51354511f2ff0f171a8df4e74291c0421f9
2013-03-11 19:25:10 +00:00
John Koleszar
9b4095c537 Fix vp9_tree_probs_from_distribution with CONFIG_CODE_NONZEROCOUNT
The automatic merge result was incomplete.

Change-Id: I8976318bfc346d867660a013a302c80edb25fc29
2013-03-11 11:03:36 -07:00
John Koleszar
e6257342b1 Merge "Optimize vp9_tree_probs_from_distribution" into experimental 2013-03-11 09:32:11 -07:00
John Koleszar
bd84685f78 Optimize vp9_tree_probs_from_distribution
The previous implementation visited each node in the tree multiple times
because it used each symbol's encoding to revisit the branches taken and
increment its count. Instead, we can traverse the tree depth first and
calculate the probabilities and branch counts as we walk back up. The
complexity goes from somewhere between O(nlogn) and O(n^2) (depending on
how balanced the tree is) to O(n).

Only tested one clip (256kbps, CIF), saw 13% decoding perf improvement.

Note that this optimization should port trivially to VP8 as well. In VP8,
the decoder doesn't use this function, but it does routinely show up
on the profile for realtime encoding.

Change-Id: I4f2848e4f41dc9a7694f73f3e75034bce08d1b12
2013-03-10 13:39:30 -07:00
Deb Mukherjee
a28139c849 Continued experiment with nonzero count
Adds probability updates for extra bits for the nzcs, code for
getting nzc stats, plus some minor cleanups and fixes.

Change-Id: If2814e7f04fb52f5025ad9f400f3e6c50a00b543
2013-03-08 16:37:08 -08:00
Ronald S. Bultje
0643c3f133 Merge "Add support for tx_select in i8x8 encoding in keyframes." into experimental 2013-03-08 16:25:27 -08:00
Dmitry Kovalev
3edbc77ae3 Merge "Consistent usage of ROUND_POWER_OF_TWO macro." into experimental 2013-03-08 11:35:22 -08:00
Jingning Han
2a5278bdbd Extend diff MV limit from +/-256 to +/-1024
Increase the motion search range by 4x. Change MV_CLASS tree of the
entropy coding to allow two additional mv classes to cover the
extended motion vector limit. The codec determines the effective
motion search range conditioned on the actual frame dimension.

It provides coding gains:

stdhd 0.39%
yt    0.56%
hd    0.47%

Major coding performance gains are packed in several sequences with
intense motion activities, e.g., ped_1080p gains 7% at high bit-rates,
and on average 3%.

TODO: Need to further tune the rate control and motion search units.

Change-Id: Ib842540a6796fbee5a797809433ef6a477c6d78d
2013-03-08 10:04:36 -08:00
Ronald S. Bultje
b41dee8428 Add support for tx_select in i8x8 encoding in keyframes.
Also enable tx_select for keyframes.

Change-Id: Iadb1231d9fa7af0c8dce3d9b41830b93a302479e
2013-03-08 09:28:46 -08:00
Dmitry Kovalev
3603dfb62c Consistent usage of ROUND_POWER_OF_TWO macro.
Change-Id: I44660975e9985310d8c654c158ee7a61291b5a08
2013-03-07 12:24:35 -08:00
Ronald S. Bultje
d3724abe9f Re-add support for ADST in superblocks.
This also changes the RD search to take account of the correct block
index when searching (this is required for ADST positioning to work
correctly in combination with tx_select).

Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-07 11:19:10 -08:00
Deb Mukherjee
eb6ef2417f Coding con-zero count rather than EOB for coeffs
This patch revamps the entropy coding of coefficients to code first
a non-zero count per coded block and correspondingly remove the EOB
token from the token set.

STATUS:
Main encode/decode code achieving encode/decode sync - done.
Forward and backward probability updates to the nzcs - done.
Rd costing updates for nzcs - done.
Note: The dynamic progrmaming apporach used in trellis quantization
is not exactly compatible with nzcs. A suboptimal approach has been
used instead where branch costs are updated to account for changes
in the nzcs.

TODO:
Training the default probs/counts for nzcs

Change-Id: I951bc1e22f47885077a7453a09b0493daa77883d
2013-03-07 07:20:30 -08:00
Dmitry Kovalev
a9961fa819 Merge "Code cleanup." into experimental 2013-03-06 16:57:34 -08:00
Paul Wilkins
72a6201050 Merge "Added stricter Q control flag." into experimental 2013-03-06 04:32:22 -08:00
Paul Wilkins
db6ad0138c Added stricter Q control flag.
Added a variant of the one shot maxQ flag
for two pass that forces a fixed Q for the
normal inter frames. Disabled by default.
Also small adjustment to the Bits per MB
estimation.
Change-Id: I87efdfb2d094fe1340ca9ddae37470d7b278c8b8
2013-03-06 12:05:49 +00:00
Dmitry Kovalev
7f99c3c59a Code cleanup.
Removing redundant 'extern' keywords, fixing formatting and #include order,
code simplification.

Change-Id: I0e5fdc8009010f3f885f13b5d76859b9da511758
2013-03-05 14:12:16 -08:00
John Koleszar
522d4bf852 Add 'superframe' index
A 'superframe' is a group of frames that share the same PTS, but have a
defined decoding order. This commit adds the ability to append an index
to such a group of frames, allowing for random access to the constituent
frames. This could be useful for frame-level parallelism or partial
decoding in a multilayer scenario.

Decoding the stream serially without such an index should work as a
fallback, and VP9/TestSuperframeIndexIsOptional verifies that.

Change-Id: Idff83b7560e1a7077d8fb067bfbc45b567e78b1c
2013-03-05 12:45:40 -08:00
Ronald S. Bultje
4209bba462 Merge changes Ifacbf5a0,Ibad7c3dd into experimental
* changes:
  vpxenc: actually report mismatch on stderr.
  Make superblocks independent of macroblock code and data.
2013-03-05 11:17:14 -08:00
Ronald S. Bultje
111ca42133 Make superblocks independent of macroblock code and data.
Split macroblock and superblock tokenization and detokenization
functions and coefficient-related data structs so that the bitstream
layout and related code of superblock coefficients looks less like it's
a hack to fit macroblocks in superblocks.

In addition, unify chroma transform size selection from luma transform
size (i.e. always use the same size, as long as it fits the predictor);
in practice, this means 32x32 and 64x64 superblocks using the 16x16 luma
transform will now use the 16x16 (instead of the 8x8) chroma transform,
and 64x64 superblocks using the 32x32 luma transform will now use the
32x32 (instead of the 16x16) chroma transform.

Lastly, add a trellis optimize function for 32x32 transform blocks.

HD gains about 0.3%, STDHD about 0.15% and derf about 0.1%. There's
a few negative points here and there that I might want to analyze
a little closer.

Change-Id: Ibad7c3ddfe1acfc52771dfc27c03e9783e054430
2013-03-04 16:34:36 -08:00
John Koleszar
daa9b29ea1 Reinitialize motion search tables on frame size change
Make sure the motion search is done with the offsets calculated from
the correct stride.

Change-Id: Ifbcc0f742eda3399c255bfcfa1cdee9a4bb4b4e7
2013-03-04 16:00:01 -08:00
Dmitry Kovalev
49b697d327 Merge "Code cleanup." into experimental 2013-03-04 15:41:15 -08:00
Jingning Han
5957b2b514 Support 16K sequence coding
Fixed a couple of variable/function definitions, as well as header
handling to support 16K sequence coding at high bit-rates.

The width and height are each specified by two bytes in the header.
Use an extra byte to explicitly indicate the scaling factors in
both directions, each ranging from 0 to 15.

Tested coding up to 16400x16400 dimension.

Change-Id: Ibc2225c6036620270f2c0cf5172d1760aaec10ec
2013-03-04 11:08:41 -08:00
John Koleszar
1cfc86ebe0 Add unit test for x4 multi-SAD functions
Update the function prototypes to match between VP9 and VP8.

Change-Id: If58965073989e87df3b62b67a030ec6ce23ca04f
2013-03-01 18:14:02 -08:00
Dmitry Kovalev
135428e954 Code cleanup.
Removing redundant 'extern' keyword, lowercase variable names.

Change-Id: I608e8d8579aba8981f5fac3493f77b4481b13808
2013-03-01 17:39:31 -08:00
John Koleszar
69c67c9531 Merge master branch into experimental
Picks up some build system changes, compiler warning fixes, etc.

Change-Id: I2712f99e653502818a101a72696ad54018152d4e
2013-03-01 11:06:05 -08:00
Yaowu Xu
db4dc6f0c0 Merge "Adjust the max_gf_interval initialization" into experimental 2013-03-01 11:02:23 -08:00
Yaowu Xu
cea8cd08d3 Adjust the max_gf_interval initialization
to be a fixed value of 15.

Test results:
cif:  .124%, .068%, .081%
std-hd: 2.809%, 3.174%, 2.705%

Change-Id: I380c8152c973506094da15eab59e3aa22b75a983
2013-03-01 06:38:35 -08:00
Dmitry Kovalev
852ca19e4b Merge "Code cleanup." into experimental 2013-02-28 17:22:51 -08:00
Dmitry Kovalev
253886413a Merge changes I9be9c990,Ic3b97339 into experimental
* changes:
  Ignoring test video sequences in the source tree.
  Code cleanup.
2013-02-28 16:07:45 -08:00
James Zern
a07bed2b2b firstpass.c: correct casting around gf_group_bits
gf_group_bits is int64_t remove casts to int.

Change-Id: I3b4225905041fac9af9fdfcbcb6f1c357ea4b593
2013-02-28 15:45:29 -08:00
Jim Bankoski
078f5bf439 Merge "mv dct_sse2.c dct_sse2_intrinsics.c to avoid collision" into experimental 2013-02-28 15:16:44 -08:00
Dmitry Kovalev
dcbdda8e15 Code cleanup.
Lower case variable names, converting while loops to for loops.

Change-Id: Ic3b973391eef7472a99d18d02fe79cfef5e04e62
2013-02-28 14:40:20 -08:00
Jim Bankoski
8f270acfb2 mv dct_sse2.c dct_sse2_intrinsics.c to avoid collision
Change-Id: Id786be31da3c91d95d2955aa569ecdc6e66650df
2013-02-28 13:58:15 -08:00
Dmitry Kovalev
0d9cc0a9f0 Code cleanup.
Removing redundant 'extern' keyword, better formatting, code
simplification.

Change-Id: I132fea14f08c706ee9ea147d19464d03f833f25b
2013-02-28 13:18:02 -08:00
John Koleszar
b6a3062d81 Fix incorrect comparison of frame size
The width and height stored in the reference frames are padded out to
a multiple of 16. The Width and Height variables in common are the
displayed size, which may be smaller. The incorrect comparison was
causing scaling related code to be called when it shouldn't have
been. A notable case where this happens is 1080p, since 1088 != 1080.

Change-Id: I55f743eeeeaefbf2e777e193bc9a77ff726e16b5
2013-02-28 11:33:02 -08:00
Jim Bankoski
714aa9f3c0 this commit converts all sad ptrs to uint32
sse4_1 code used uint16_t for returning sad, but that
won't work for 32x32 or 64x64.   This code fixes the
assembly for those and also reenables sse4_1 on linux

Change-Id: I5ce7288d581db870a148e5f7c5092826f59edd81
2013-02-28 08:46:35 -08:00
Jim Bankoski
b715e371c0 fix to parameters to match rtcd
Change-Id: I919e2dd72292fe44f2e53ada56bd42287d50cdeb
Signed-off-by: Jim Bankoski <jimbankoski@google.com>
2013-02-28 08:10:08 -08:00
Christian Duvivier
c129203f7e Faster vp9_short_fdct8x8.
Scalar path is about 1.4x faster (4% overall encoder speedup).
SSE2 path is about 7x faster (13% overall encoder speedup).

Change-Id: I7e85d8225a914a74c61ea370210414696560094d
2013-02-27 17:23:08 -08:00
Dmitry Kovalev
347f3a0aa8 Code cleanup.
Fixing code style, using array lookup instead of switch statements for
forward hybrid transforms (in the same way as for their inverses).
Consistent usage of ROUND_POWER_OF_TWO macro in appropriate places.

Change-Id: I0d3822ae11f928905fdbfbe4158f91d97c71015f
2013-02-27 13:51:04 -08:00
John Koleszar
5ac141187a Merge "Remove unused vp9_copy32xn" into experimental 2013-02-27 12:23:45 -08:00
Ronald S. Bultje
90932399b4 Merge "Move eob from BLOCKD to MACROBLOCKD." into experimental 2013-02-27 11:39:16 -08:00
John Koleszar
09be534f13 Merge "give vp9 variance struct a unique name" 2013-02-27 11:22:36 -08:00
Ronald S. Bultje
e8c74e2b70 Move eob from BLOCKD to MACROBLOCKD.
Consistent with VP8.

Change-Id: I8c316ee49f072e15abbb033a80e9c36617891f07
2013-02-27 11:00:55 -08:00
John Koleszar
0921bfb749 Merge "Use ref_frame_map vice active_ref_idx on the encoder" into experimental 2013-02-27 10:59:08 -08:00
John Koleszar
9615fd8f39 Merge "Test upscaling as well as downscaling" into experimental 2013-02-27 10:25:51 -08:00
John Koleszar
7ad8dbe417 Remove unused vp9_copy32xn
This function was part of an optimization used in VP8 that required
caching two macroblocks. This is unused in VP9, and might not
survive refactoring to support superblocks, so removing it for now.

Change-Id: I744e585206ccc1ef9a402665c33863fc9fb46f0d
2013-02-27 10:24:56 -08:00
Jan Kratochvil
82ed3f9a41 Fix --as=nasm compatibility for new asm code.
s/movd/movq/

Change-Id: Id1a56de91551f8dc796f14f1056c565dfc1ba626
2013-02-27 09:55:38 -08:00
John Koleszar
350ba5f30e Merge "Combined motion compensation with scaled predictors" into experimental 2013-02-27 09:46:12 -08:00
John Koleszar
800ad0b886 Use ref_frame_map vice active_ref_idx on the encoder
This patch makes the encoder's use of ref_frame_map and active_ref_idx
consistent with the decoder. ref_frame_map[] maps a reference buffer
index to its actual location in the yv12_fb array, since many
references may share an underlying buffer. active_ref_idx[] mirrors
cpi->{lst,gld,alt}_fb_idx, holding the active references in each
slot.

This also fixes a bug in setup_buffer_inter() where the incorrect
reference was used to populate the scaling factors.

Change-Id: Id3728f6d77cffcd27c248903bf51f9c3e594287e
2013-02-27 08:22:40 -08:00
John Koleszar
b683eecf6d Test upscaling as well as downscaling
Fixes a bug in vp9_set_internal_size() that prevented returning to
the unscaled state. Updated the ResizeInternalTest to scale both
down and up. Added a check that all frames are within 2.5% of the
quality of the initial keyframe.

Change-Id: I3b7ef17cdac144ed05b9148dce6badfa75cff5c8
2013-02-27 08:22:40 -08:00
John Koleszar
77f88e97fa Combined motion compensation with scaled predictors
This patch extends the previous support for using references of a
different resolution in ZEROMV mode to all inter prediction modes.
Subpixel based best-mv scoring is disabled when the reference frame
differs in resolution from the current frame.

Change-Id: Id4dc3e5e6692de98d9857fd56bfad3ac57e944ac
2013-02-27 08:22:39 -08:00
Yaowu Xu
858b60e8d0 Merge "Improve 32x32 forward dct" into experimental 2013-02-27 07:56:42 -08:00
John Koleszar
eb939f45b8 Spatial resamping of ZEROMV predictors
This patch allows coding frames using references of different
resolution, in ZEROMV mode. For compound prediction, either
reference may be scaled.

To test, I use the resize_test and enable WRITE_RECON_BUFFER
in vp9_onyxd_if.c. It's also useful to apply this patch to
test/i420_video_source.h:

  --- a/test/i420_video_source.h
  +++ b/test/i420_video_source.h
  @@ -93,6 +93,7 @@ class I420VideoSource : public VideoSource {

     virtual void FillFrame() {
       // Read a frame from input_file.
  +    if (frame_ != 3)
       if (fread(img_->img_data, raw_sz_, 1, input_file_) == 0) {
         limit_ = frame_;
       }

This forces the frame that the resolution changes on to be coded
with no motion, only scaling, and improves the quality of the
result.

Change-Id: I1ee75d19a437ff801192f767fd02a36bcbd1d496
2013-02-26 23:54:23 -08:00
Ronald S. Bultje
96d260515a Merge "Merge cnvcontext experiment." into experimental 2013-02-26 19:39:39 -08:00
Ronald S. Bultje
1a0533958b Merge "Fix modes.stt output printf format string." into experimental 2013-02-26 19:39:33 -08:00
Ronald S. Bultje
db54e6774f Merge "Minor cosmetics in rdopt." into experimental 2013-02-26 19:39:28 -08:00
James Zern
4446af78f0 Merge "vp9: promote gf_group_bits calculation to 64-bit" into experimental 2013-02-26 16:27:45 -08:00
John Koleszar
25686fc22d Merge "Refactor inter recon functions to support scaling" into experimental 2013-02-26 11:45:28 -08:00
Dmitry Kovalev
998bed1d2c Merge "Changing pitch value meaning for fht and iht transforms." into experimental 2013-02-26 10:44:15 -08:00
Ronald S. Bultje
b1641150b1 Merge cnvcontext experiment.
Change-Id: I35e64998b25694a3bb4a62164bba3c03c1db4bc7
2013-02-26 10:40:15 -08:00
Ronald S. Bultje
f3fdb4c37d Fix modes.stt output printf format string.
Change-Id: I17e2d2f6a4da86d9e4af7bebdea0bf5d154da084
2013-02-26 10:40:15 -08:00
Ronald S. Bultje
71539eae2a Minor cosmetics in rdopt.
Change-Id: I62497dcf2074b4bb4787bf660e727e5cf1bf3472
2013-02-26 10:40:11 -08:00
Ronald S. Bultje
c4ae97911a Merge "make cost_coeffs to use combined context" into experimental 2013-02-26 10:32:01 -08:00
John Koleszar
6a4f708c25 Refactor inter recon functions to support scaling
Ensure that all inter prediction goes through a common code path
that takes scaling into account. Removes a bunch of duplicate
1st/2nd predictor code. Also introduces a 16x8 mode for 8x8
MVs, similar to the 8x4 trick we were doing before. This has an
unexpected effect with EIGHTTAP_SMOOTH, so it's disabled in that
case for now.

Change-Id: Ia053e823a8bc616a988a0af30452e1e75a739cba
2013-02-26 10:03:29 -08:00
Yaowu Xu
66d94ac13c Improve 32x32 forward dct
The commit improves the 32x32 forward dct implementation:
1. change to use same constants and rounding as other forward dcts
2. select rounding to specifically minimize the roundtrip error, which
improved average 19/block to .77/block using 100000 random input.

Test showed a small but consistent gain on all test sets, about .15%

Change-Id: If0afd6a71880a522f60c1c234be0462092c2eb53
2013-02-26 09:23:01 -08:00
Dmitry Kovalev
9bf3f75168 Changing pitch value meaning for fht and iht transforms.
Pitch now means the number of elements, not the number of bytes.

Change-Id: Idb9f2f012e39b09d596a3cc1802305a80b7c13af
2013-02-25 18:19:55 -08:00
Yaowu Xu
ecb03e9a3f make cost_coeffs to use combined context
Change-Id: Ia15f4244595fab49bffda0c651a750a8a9481d28
2013-02-25 17:01:33 -08:00
Jingning Han
e7b67d33a9 Merge "Improving the forward 16x16 ADST/DCT accuracy" into experimental 2013-02-25 13:38:33 -08:00
James Zern
b2fc3ca066 vp9: promote gf_group_bits calculation to 64-bit
avoids signed integer overflow

Change-Id: I9ffcdba90b21edb324d1b173fd11d613e0592931
2013-02-25 13:00:18 -08:00
Paul Wilkins
0e36158c70 Merge "Minor rate control refactoring and experiments." into experimental 2013-02-25 12:49:54 -08:00
Jingning Han
65821d6680 Improving the forward 16x16 ADST/DCT accuracy
Increase the first stage dynamic range by 4 times, and reduce it
back with proper rounding before applying the second stage. Hence
it still fits in the given dynamic range and slightly improves
the key frame coding performance.

Change-Id: Ia4c5907446f20a95dc3de079c314b3ad1221d8aa
2013-02-25 12:13:37 -08:00
Jingning Han
77a3becf92 clean up forward and inverse hybrid transform
Rebased.

Remove the old matrix multiplication transform computation. The 16x16
ADST/DCT can be switched on/off and evaluated by setting ACTIVE_HT16
300/0 in vp9/common/vp9_blockd.h.

Change-Id: Icab2dbd18538987e1dc4e88c45abfc4cfc6e133f
2013-02-25 09:16:12 -08:00
Paul Wilkins
97da8b8c33 Minor rate control refactoring and experiments.
Some minor refactoring code relating to estimates of
bits per MB at a given Q and estimating the allowed Q range.

Most of the changes here were included in a previous commit.
This commit seeks to separate out the refactoring from more
the material changes.

Two #define control flags have been added for experimentation.

ONE_SHOT_Q_ESTIMATE force the two pass encoder to
use its initial Q range estimate for the whole clip even if this results
in a miss on the target data rate. In effect this tightens the Q range
seen at the expense of rate control accuracy.

DISABLE_RC_LONG_TERM_MEM is a related flag that disables the
long term memory in the rate control. Local adjustments are still
made to try and better hit the rate target on a per frame basis but
the impact of rate control misses is not propagated to the remainder
of the clip. This means that for example an overshoot early on will not
cause frames later in the clip to be starved of bits. Again the result
of this relaxation amy be less rate control accuracy especially on short
clips.

The flags are disabled by default for now.

Change-Id: I7482f980146d8ea033b5d50cc689f772e4bd119e
2013-02-25 17:07:45 +00:00
Yaowu Xu
499fe05dc0 optimize forward 16x16 DCT for accuracy
This commit added pre/post scaling for first half of fDCT16x16 to
reduce error, by simulation of 100,000 blocks for random inputs,
the average sse reduced from 2.1/block to 0.0498/block.

also enabled tests for 16x16 fDCT and iDCT

Change-Id: Id2a95f0464c6dd4118797d456237ae90274c0f02
2013-02-25 07:47:27 -08:00
Ronald S. Bultje
0c9e2e9a1d Split coefficient token tables intra vs. inter.
Change-Id: I5416455f8f129ca0f450d00e48358d2012605072
2013-02-23 07:33:46 -08:00
Paul Wilkins
c17672a33d Further changes to coefficient contexts.
This patch alters the balance of context between the
coefficient bands (reflecting the position of coefficients
within a transform blocks) and the energy of the previous
token (or tokens) within a block.

In this case the number of coefficient bands is reduced
but more previous token energy bands are supported.

Some initial rebalancing of the default tables has been
by running multiple derf clips at multiple data rates using
the ENTOPY_STATS macro. Further balancing needs to be
done using larger image formatsd especially in regard to
the bigger transform sizes which are not as well represented
in encodings of smaller image formats.

Change-Id: If9736e95c391e711b04aef6393d26f60f36e1f8a
2013-02-23 07:29:09 -08:00
Yaowu Xu
bf0570a7e6 Merge "optimize 8x8 fdct rounding for accuracy" into experimental 2013-02-22 22:20:57 -08:00
Yaowu Xu
22012ee994 optimize 8x8 fdct rounding for accuracy
The commit added a final rounding choice for 8x8 forward dct to get
rid of a sign bias at DC position and improve the accuracry in term
of round trip error for 8x8 fDCT/iDCT.

This commit also enabled forward 8x8 dct test.

Change-Id: Ib67f99b0a24d513e230c7812bc04569d472fdc50
2013-02-22 16:55:30 -08:00
James Zern
e5fb6321a1 give vp9 variance struct a unique name
variance_vtable clashed with vp8/common/variance.h

Change-Id: I09c1de44d5519f1bd13f58c01144c0de4706de6f
2013-02-22 16:25:13 -08:00
James Zern
c21226b638 Merge "vp8: make gf_group_bits 64-bit" 2013-02-22 15:31:28 -08:00
James Zern
5e0724abad Merge "vp8_first_pass(): avoid floating point div by 0" 2013-02-22 15:30:14 -08:00
James Zern
4e00060d29 vp8: make gf_group_bits 64-bit
avoids signed integer overflow; matches kf_group_bits

Change-Id: I193145cdc4fa53e70fba0a1731a03eb1a574931d
2013-02-22 12:45:28 -08:00
James Zern
fba9772dd2 vp8_first_pass(): avoid floating point div by 0
Change-Id: Id1e6a12db6b0c1d3f64ead8fd8834aadc30fbed2
2013-02-22 12:41:59 -08:00
Jingning Han
936aa281b5 Fixed the buffer overflow issue
The issue that potentially broke the encoding process was due to the fact
that the length of token link is calculated from the total number of tokens
coded, while it is possible, in high bit-rate setting, this length is
greater than the buffer length initially assigned to the cpi->tok.

This patch increases the initially allocated buffer length assigned to
cpi->tok from
(mb_rows * mb_cols * 24 * 16) to (mb_rows * mb_cols * (1 + 24 * 16)).

It resolves the buffer overflow problem.

Change-Id: I8661a8d39ea0a3c24303e3f71a170787a1d5b1df
2013-02-22 12:30:35 -08:00
John Koleszar
606a2561d6 Merge "Code cleanup." into experimental 2013-02-22 11:20:20 -08:00
Dmitry Kovalev
548b4dd5f2 Code cleanup.
Removing redundant 'extern' keywords and parentheses, fixing indentation,
making variable names lower case, using short expressions x *= c
instead of x = x * c, minor code simplifications.

Change-Id: If6a25fcf306d1db26e90d27e3c24a32735c607de
2013-02-22 11:03:14 -08:00
Jingning Han
c67a20994f Merge "Forward butterfly hybrid transform" into experimental 2013-02-22 09:20:26 -08:00
Paul Wilkins
b5f3cb6e37 Merge "Experimental removal of over quant code" into experimental 2013-02-22 08:44:40 -08:00
Paul Wilkins
dbf4942046 Experimental removal of over quant code
The over quant code was added in VP8 post
bitstream freeze to allow compression to lower
data rates

In VP9 the real qualtizer range has been greatly
extended anyway.

Change-Id: I5d384fa5e9a83ef75a3df34ee30627bd21901526
2013-02-22 14:00:51 +00:00
Jingning Han
babbd5d170 Forward butterfly hybrid transform
This patch includes 4x4, 8x8, and 16x16 forward butterfly ADST/DCT
hybrid transform. The kernel of 4x4 ADST is sin((2k+1)*(n+1)/(2N+1)).
The kernel of 8x8/16x16 ADST is of the form sin((2k+1)*(2n+1)/4N).

Change-Id: I8f1ab3843ce32eb287ab766f92e0611e1c5cb4c1
2013-02-21 18:24:28 -08:00
Dmitry Kovalev
5a18106fb7 Code cleanup.
Removing redundant 'extern' keywords. Moving VP9DX_BOOL_DECODER from .h
to .c file.

Change-Id: I5a3056cb3d33db7ed3c3f4629675aa8e21014e66
2013-02-21 13:50:15 -08:00
John Koleszar
4674312382 Merge "Code cleanup." into experimental 2013-02-21 10:56:17 -08:00
Dmitry Kovalev
5da8534963 Code cleanup.
Removing redundant 'extern' keyword from function declarations and making
function arguments lower case.

Change-Id: Idae9a2183b067f2b6c85ad84738d275e8bbff9d9
2013-02-21 10:34:33 -08:00
Deb Mukherjee
048f593703 Merge "Refactoring of switchable filter search for speed" into experimental 2013-02-21 09:23:50 -08:00
Deb Mukherjee
28b1db9278 Refactoring of switchable filter search for speed
Refactors the switchable filter search in the rd loop to
improve encode speed.

Uses a piecewise approximation to a closed form expression to estimate
rd cost for a Laplacian source with a given variance and quantization
step-size.

About 40% encode time reduction is achieved.

Results (on a feb 12 baseline) show a slight drop:

derf: -0.019%
yt: +0.010%
std-hd: -0.162%
hd: -0.050%

Change-Id: Ie861badf5bba1e3b1052e29a0ef1b7e256edbcd0
2013-02-20 18:34:42 -08:00
Jingning Han
abfd2a4880 Merge "Fixed the buffer overflow issue" into experimental 2013-02-20 16:27:27 -08:00
Jingning Han
232ccc2fbe Fixed the buffer overflow issue
The issue that potentially broke the encoding process was due to the fact
that the length of token link is calculated from the total number of tokens
coded, while it is possible, in high bit-rate setting, this length is
greater than the buffer length initially assigned to the cpi->tok.

This patch increases the initially allocated buffer length assigned to
cpi->tok from
(mb_rows * mb_cols * 24 * 16) to (mb_rows * mb_cols * (1 + 24 * 16)).

It resolves the buffer overflow problem.

Change-Id: I8661a8d39ea0a3c24303e3f71a170787a1d5b1df
2013-02-20 15:41:48 -08:00
Yaowu Xu
441f24de3d Merge "Merge lossless experiment" into experimental 2013-02-20 12:27:26 -08:00
Yaowu Xu
d262e26cc7 Merge lossless experiment
Change-Id: I7b7b8d4fda3a23699e0c920d727f8c15d37d43aa
2013-02-20 07:54:28 -08:00
Paul Wilkins
ef01b956d8 Entropy stats output code.
Fixes to make Entropy stats code work again

Change-Id: I62e380481a4eb4c170076ac6ab36f0c2b203e914
2013-02-20 14:33:19 +00:00
Ronald S. Bultje
ae81d3a03f Merge "Minor cosmetic cleanups." into experimental 2013-02-19 08:54:44 -08:00
Ronald S. Bultje
0694ea0ed6 Merge "Prevent filling transform size cache with uninitialized values." into experimental 2013-02-19 08:54:35 -08:00
Yaowu Xu
93d6b86cfd Use lossless for Q0
The commit changes the coding mode to lossless whenever the lowest
quantizer is choosen.

As expected, test results showed no difference for cif and std-hd
set where Q0 is rarely used. For yt and yt-hd set, Q0 is used for
a number of clips, where this commit helped a lot in the high end.

Average over all clips in the sets:
yt: 2.391% 1.017% 1.066%
hd: 1.937%  .764%  .787%

Change-Id: I9fa9df8646fd70cb09ffe9e4202b86b67da16765
2013-02-19 06:18:42 -08:00
Ronald S. Bultje
aa84c16da2 Minor cosmetic cleanups.
Change-Id: I13d8ae754827368755575dd699a087b3b11f5b16
2013-02-15 17:21:16 -08:00
Ronald S. Bultje
ebfdaa0e0b Prevent filling transform size cache with uninitialized values.
The 32x32 value in case of splitmv was uninitialized. this leads to
all kind of erratic behaviour down the line. Also fill in dummy values
for superblocks in keyframes (the values are currently unused, but we
run into integer overflows anyway, which makes detecting bad cases
harder). Lastly, in case we did not find any RD value at all, don't
set tx_diff to INT_MIN, but instead set it to zero (since if we couldn't
find a mode, it's unlikely that any particular transform would have made
that worse or better; rather, it's likely equally bad for all tx_sizes).

Change-Id: If236fd3aa2037e5b398d03f3b1978fbbc5ce740e
2013-02-15 17:21:16 -08:00
Ronald S. Bultje
5bb103c486 Merge "Remove Y2 and Y-no-DC token types from the bitstream." into experimental 2013-02-15 17:11:20 -08:00
Jingning Han
e343732a92 Fixed a subtle issue that breaks encoding process
This issue breaks the encoding process of the codebase. The effect
emerges only in particular test sequence at certain bit-rates and
frame limits.

Change-Id: I02e080f2a49624eef9a21c424053dc2a1d902452
2013-02-15 14:49:30 -08:00
Ronald S. Bultje
3af36ea8cc Remove Y2 and Y-no-DC token types from the bitstream.
Change-Id: I7a5314daca993d46b8666ba1ec2ff3766c1e5042
2013-02-15 14:06:30 -08:00
Ronald S. Bultje
48598e30b1 Remove y2dc/ac Q delta values from the bitstream.
Since there is no Y2, these values are always zero. This changes the
bitstream results slightly, hence a separate commit.

Change-Id: I2f838f184341868f35113ec77ca89da53c4644e0
2013-02-15 14:06:30 -08:00
Ronald S. Bultje
46dff5d233 Remove some Y2-related code.
Change-Id: I4f46d142c2a8d1e8a880cfac63702dcbfb999b78
2013-02-15 14:06:25 -08:00
John Koleszar
716db10f0d Merge "Moved vp9_get_coef_band to header file" into experimental 2013-02-14 18:02:55 -08:00
Scott LaVarnway
ae886d6bff Moved vp9_get_coef_band to header file
allowing the compiler to inline.

Change-Id: I66e5caf5e7fefa68a223ff0603aa3f9e11e35dbb
2013-02-14 12:27:25 -08:00
Yaowu Xu
03f28c0a12 Merge "Rewrote fdct16x16" into experimental 2013-02-14 09:06:37 -08:00
Paul Wilkins
45712dc8c8 Merge "Abstract selection of coef band." into experimental 2013-02-14 03:23:31 -08:00
Yunqing Wang
048b9d41a6 Rewrote fdct16x16
Used same algorithm as others.

Change-Id: Ifdac560762aec9735cb4bb6f1dbf549e415c38a0
2013-02-13 16:19:10 -08:00
Ronald S. Bultje
89a206ef2f Add support for tile rows.
These allow sending partial bitstream packets over the network before
encoding a complete frame is completed, thus lowering end-to-end
latency. The tile-rows are not independent.

Change-Id: I99986595cbcbff9153e2a14f49b4aa7dee4768e2
2013-02-13 12:31:00 -08:00
Paul Wilkins
9255ad107f Abstract selection of coef band.
This patch abstracts the selection of the coefficient band
context into a function as a precursor to further experiments
with the coefficient context.

It also removes the large per TX size coefficient band structures
and uses a single matrix for all block sizes within the test function.

This may have an impact on quality (results to follow) but is only an
intermediate step in the process of redefining the context. Also the
quality impact will be larger initially because the default tables will
be out of step with the new banding.

In particular the 4x4 will in this case only use 7 bands. If needed we
can add back block size dependency localized within the function, but
this can follow on after the other changes to the definition of the
context.

Change-Id: Id7009c2f4f9bb1d02b861af85fd8223d4285bde5
2013-02-13 19:01:25 +00:00
Paul Wilkins
56049d9488 Fixed encoder decoder mismatch.
Reverted part of change
I19981d1ef0b33e4e5732739574f367fe82771a84

That gives rise to an enc/dec mismatch.
As things stand the memsets are still needed.

Change-Id: I9fa076a703909aa0c4da0059ac6ae19aa530db30
2013-02-13 18:56:56 +00:00
Paul Wilkins
0d284ffed1 Abstract the selection of coefficient context.
This is an initial step to facilitate experimentation
with changes to the prior token context used to code
coefficients to take better account of the energy of
preceding tokens.

This patch merely abstracts the selection of context into
two functions and does not alter the output.

Change-Id: I117fff0b49c61da83aed641e36620442f86def86
2013-02-13 18:56:30 +00:00
Paul Wilkins
afa57bfc97 Merge "Remove NEWCOEFCONTEXT experiment." into experimental 2013-02-13 10:41:13 -08:00
Yaowu Xu
f01b08c96c Merge "enable bitstream lossless support" into experimental 2013-02-13 10:26:58 -08:00
Yaowu Xu
d3de97794f Merge "fix the lossless experiment" into experimental 2013-02-13 09:54:35 -08:00
Yaowu Xu
17db5d00be enable bitstream lossless support
1. Added a bit in frame header to  to indicate if a frame is encoded
in lossless mode, so decoder does not make the decision based on Q0
2. Minor changes to make sure that lossy coding works same as when
the lossless experiment is not enabled.
3. Renamed function pointers for transforms to be consistent, using
prefix fwd_txm and inv_txm for forward and inverse respectively

To encode in lossless mode, using "--lossless=1 --min-q=0 --max-q=0"
with vpxenc.

Change-Id: Ifae53b26d2ffbe378d707e29d96817b8a5e6c068
2013-02-13 09:24:39 -08:00
Yaowu Xu
16f25f9dc8 fix the lossless experiment
Change-Id: I95acfc1417634b52d344586ab97f0abaa9a4b256
2013-02-13 09:20:26 -08:00
Paul Wilkins
6a9f0c61a4 Remove NEWCOEFCONTEXT experiment.
Removal of the  NEWCOEFCONTEXT experiment to
reduce code clutter and make it easier to experiment with
some other changes to the coefficient coding context.

Change-Id: Icd17b421384c354df6117cc714747647c5eb7e98
2013-02-13 15:12:17 +00:00
Paul Wilkins
649be94cf0 Removal of Hybrid DWT/DCT experiment.
Removal of experiment to simplify code base for other
changes.

Change-Id: If0a33952504558511926ad212bc311fc2bffb19a
2013-02-13 15:08:48 +00:00
Christian Duvivier
097f205289 Merge "Faster vp9_regular_quantize_b_8x8." into experimental 2013-02-12 17:08:00 -08:00
Christian Duvivier
0e4397f0cd Faster vp9_regular_quantize_b_8x8.
A couple of scalar optimizations speeding up quantization by about 1.6x. Overall encoder speedup is around 3%.

Change-Id: I19981d1ef0b33e4e5732739574f367fe82771a84
2013-02-12 15:55:58 -08:00
Yunqing Wang
7630cf0c3f Merge "Rewrote fdct8x8" into experimental 2013-02-12 15:52:31 -08:00
John Koleszar
1d60b6bcb5 Merge "Replace as_mv struct with array" into experimental 2013-02-12 13:59:04 -08:00
Ronald S. Bultje
f496f601fb Add tile column size limits (256 pixels min, 4096 pixels max).
This is after discussion with the hardware team. Update the unit test
to take these sizes into account. Split out some duplicate code into
a separate file so it can be shared.

Change-Id: I8311d11b0191d8bb37e8eb4ac962beb217e1bff5
2013-02-12 10:33:34 -08:00
Yunqing Wang
aa295918ed Rewrote fdct8x8
Use consistent algorithm.

Change-Id: Ib8484821ebc454b9d3380a3d6571798decd037f3
2013-02-11 22:28:05 -08:00
Jingning Han
f1060e4cd8 Merge "butterfly inverse 4x4 ADST" into experimental 2013-02-11 14:46:06 -08:00
Yunqing Wang
ab2dc6ae57 Merge "Integerization of dct32x32" into experimental 2013-02-11 12:15:26 -08:00
Jingning Han
57e995ff9c butterfly inverse 4x4 ADST
fixed format issues.

Implement the inverse 4x4 ADST using 9 multiplications. For this
particular dimension, the original ADST transform can be
factorized into simpler operations, hence is retained.

Change-Id: Ie5d9749942468df299ab74e90d92cd899569e960
2013-02-11 10:42:39 -08:00
Ronald S. Bultje
5f2e8449b7 Merge "Port sadNxNx4d functions to x86inc.asm." into experimental 2013-02-11 08:20:12 -08:00
Paul Wilkins
aec5bed3db Change rd thresholds and add speed trade off flags.
Experimental tweaks to various thresholds to measure
quality / speed trade off.

Add flag that allows static segmentation to be turned off
and disables it unless in the second pass of a two pass
encode.

Change-Id: I219702ffe858412a83db801cbbbd869924b8c61b
2013-02-11 11:54:36 +00:00
Paul Wilkins
e4f949b55a Merge "Nearest / Zero Mv default entropy tweak." into experimental 2013-02-09 04:21:08 -08:00
John Koleszar
7ca517f755 Replace as_mv struct with array
Replace as_mv.{first, second} with a two element array, so that they
can easily be processed with an index variable.

Change-Id: I1e429155544d2a94a5b72a5b467c53d8b8728190
2013-02-08 20:23:35 -08:00
John Koleszar
dc836109e4 Merge "Pass macroblock index to pick inter functions" into experimental 2013-02-08 20:20:37 -08:00
Ronald S. Bultje
c0ce2ab349 Port sadNxNx4d functions to x86inc.asm.
Change-Id: Ic639f5742f7a007753d7a3fa5c66235172eb31d8
2013-02-08 17:59:32 -08:00
Ronald S. Bultje
02ff360b33 Add sad64x64 and sad32x32 SSE2 versions.
Also port the 4x4, 16x16, 8x16 and 16x8 versions to x86inc.asm; this
makes them all slightly faster, particularly on x86-64. Remove SSE3
sad16x16 version, since the SSE2 version is now faster.

About 1.5% overall encoding speedup.

Change-Id: Id4011a78cce7839f554b301d0800d5ca021af797
2013-02-08 16:32:25 -08:00
Ronald S. Bultje
639b863d22 Make cost_coeffs() more efficient.
Cache the constant offset in one variable to prevent re-loading that
in each loop iteration, and mark the function as inline so we can use
the fact that the transform size is always known in the caller.

Almost 1% faster encoding overall.

Change-Id: Id78325a60b025057d8f4ecd9003a74086ccbf85a
2013-02-08 16:32:24 -08:00
John Koleszar
6125a1ed81 Pass macroblock index to pick inter functions
Pass the current mb row and column around rather than the
recon_yoffset and recon_uvoffset, since those offsets will
change from predictor to predictor, based on the reference
frame selection.

Change-Id: If3f9df059e00f5048ca729d3d083ff428e1859c1
2013-02-08 14:25:40 -08:00
John Koleszar
6dfc95fe63 Merge changes Icd1a2a5a,I204d17a1,I3ed92117 into experimental
* changes:
  Initial support for resolution changes on P-frames
  Avoid allocating memory when resizing frames
  Adds a test for the VP8E_SET_SCALEMODE control
2013-02-08 14:20:05 -08:00
John Koleszar
3de8ee6ba1 Merge changes Ife0d8147,I7d469716,Ic9a5615f into experimental
* changes:
  Restore SSSE3 subpixel filters in new convolve framework
  Convert subpixel filters to use convolve framework
  Add 8-tap generic convolver
2013-02-08 13:19:47 -08:00
John Koleszar
393b485627 Initial support for resolution changes on P-frames
Allows inter-frames to change resolution. Currently these are
almost equivalent to keyframes, as only intra prediction modes
are allowed, but without the other context resets that occur on
keyframes.

Change-Id: Icd1a2a5af0d9462cc792588427b0a1f5b12e40d3
2013-02-08 12:20:30 -08:00
John Koleszar
c03d45def9 Avoid allocating memory when resizing frames
As long as the new frame is smaller than the size that was originally
allocated, we don't need to free and reallocate the memory allocated.
Instead, do the allocation on the size of the first frame. We could
make this passed in from the application instead, if we wanted to
support external upscaling.

Change-Id: I204d17a130728bbd91155bb4bd863a99bb99b038
2013-02-08 12:20:30 -08:00
John Koleszar
88f99f4ec2 Adds a test for the VP8E_SET_SCALEMODE control
Tests that the external interface to set the internal codec scaling
works as expected. Also updates the test to pull the height from
the decoded frame size rather than parsing the keyframe header,
in anticipation of allowing resolution changes on non-keyframes.

Change-Id: I3ed92117d8e5288fbbd1e7b618f2f233d0fe2c17
2013-02-08 12:20:30 -08:00
Yunqing Wang
dbccffe299 Integerization of dct32x32
Test on derf set showed 0.047% overall psnr change.

Change-Id: Id16c276c251a3943850ac9b95e9b09a56cf42b19
2013-02-08 08:50:47 -08:00
Paul Wilkins
bbede82f24 Nearest / Zero Mv default entropy tweak.
Tweak to default mode context to account for the fact
that when there are no non zero motion candidates
Nearest is now the preferred mode for coding a 0,0
vector.

Also resolve duplicate function name and typos.

Change-Id: I76802788d46c84e3d1c771be216a537ab7b12817
2013-02-08 10:16:13 +00:00
Yaowu Xu
e6ad9ab02c move dct/idct constants to a header file
also removed some un-unsed functions.

Change-Id: Ie363bcc8d94441d054137d2ef7c4fe59f56027e5
2013-02-07 13:51:45 -08:00