Commit Graph

511 Commits

Author SHA1 Message Date
Jim Bankoski
b95338c7ab Merge "fixes --disable-vp9-encoder" into vp9-preview 2012-12-03 12:41:31 -08:00
Jim Bankoski
d9038b3c60 fixes --disable-vp9-encoder
Change-Id: I467bf0fdf3b35326bcce58d5459e6d2dbfd6c5e5
2012-12-03 12:21:16 -08:00
Frank Galligan
0d687ed22b vp9: Fix assert check.
Change-Id: If0cc1ab60dff6abd67dae7c7b3dc83a1afd7fe65
2012-12-03 12:18:59 -08:00
Frank Galligan
3e0ea7f6e1 vp9: Remove superfluous command.
- vpx_calloc is called on arf_not_zz above.
- Note The removed vpx_memset call had an issue with sizeof.

Change-Id: I86fd7a167d0a042e581e613e2a6c0b5e63073fc6
2012-12-03 10:26:15 -08:00
Deb Mukherjee
8b92f1e023 Supports inter-intra prediction with superblocks
Adds support for compound inter-intra prediction with superblocks.
Also, fixes a bug that disabled intra modes for superblocks.

Change-Id: I4d711317e1bc19df8c2f32dc645429f7fff31036
2012-12-01 15:19:55 -08:00
Deb Mukherjee
6632330702 Adds switchable filters with superblocks
Allows switchbale filters to be used without mismatch when the
superblock experiment is on.

Also removes a spurious clamping code in decodemv.c which causes
rare encode/decode mismatches.

Change-Id: I809d9ee0b2859552b613500b539a615515b863ae
2012-11-30 09:37:08 -08:00
Jim Bankoski
9f9370425b warnings in various experiments
Change-Id: Ib5106d4772450f8026f823dd743f162ab833b1d6
2012-11-30 07:31:37 -08:00
Jim Bankoski
2b8dc065d1 google style guide include guards
Change-Id: I2c252f3ddcc99e96c1f5d3dab8bcb25a2a3637ea
2012-11-30 07:30:59 -08:00
Yunqing Wang
eebc0b49f1 Merge "Further improve macroblock loop filters" into experimental 2012-11-29 16:07:14 -08:00
Deb Mukherjee
d7489ea45e Merge "Minor refactoring of superblock decoding" into experimental 2012-11-29 15:33:42 -08:00
Deb Mukherjee
be08b5af1a Minor refactoring of superblock decoding
Refactoring for improved readability - no bitstream or
performance change.

Change-Id: I4488ed4715f8dbe38c66431106478669041b8b33
2012-11-29 15:26:56 -08:00
Jim Bankoski
e3bdae1fc7 intrinsic warnings begone
Change-Id: I6a224c590b6a2c5b91f9084ffb8083d18223a206
2012-11-29 14:14:26 -08:00
Jim Bankoski
d0a20fd22c last remaining warning
Change-Id: I1f49d96cdb5e342041c9a72ef31df361a1b609eb
2012-11-29 14:07:21 -08:00
Jim Bankoski
51e770deb1 fix implicit warnings idct etc
Change-Id: I54a122cc8c0b6ed2dbc3c6ecfcd44736cd40b687
2012-11-29 11:23:02 -08:00
Jim Bankoski
ef3c01ed67 Additional warning message cleanup.
Change-Id: I429a97ac57db3de0bf67ce3f3fe0c6b409f77a9e
2012-11-29 10:10:51 -08:00
Yaowu Xu
ff2f9de828 Merge changes Iaa67bcf1,Ibea3bc80 into experimental
* changes:
  more warning cleanup
  unused variables & warnings
2012-11-29 09:34:10 -08:00
Yaowu Xu
b3055ec020 Merge "more unused variables." into experimental 2012-11-29 09:33:59 -08:00
Yaowu Xu
8422ef772d Merge "unused variable" into experimental 2012-11-29 09:33:52 -08:00
Yaowu Xu
e007eb89cf Merge "unused var removed" into experimental 2012-11-29 09:33:41 -08:00
Yaowu Xu
6431007df3 Merge "minor fix to eob check for setting CONTEXT" into experimental 2012-11-29 09:27:00 -08:00
Yaowu Xu
7ab1d3e49f minor fix to eob check for setting CONTEXT
Previously, the "!=" check is logically incorrect when eob is at 0 and
effective coefficient starting position is 1. This commit should have
no effect on bitstream.

Change-Id: I6ce3a847c7e72bfbe4f7c74f88e3310c6b9b6d30
2012-11-29 09:10:15 -08:00
Jim Bankoski
00b27a3647 more warning cleanup
Change-Id: Iaa67bcf1e866dfe255c4e458d4e51e9c708ffcf4
2012-11-29 09:07:12 -08:00
Jim Bankoski
a802f5e783 unused variables & warnings
Change-Id: Ibea3bc80eb26a975faaa60268bbc93237f82bc57
2012-11-29 09:02:47 -08:00
Jim Bankoski
cf671e2756 more unused variables.
Change-Id: Ibe11e9275949b26a77fa9c8ac2e7c356ae533d5d
2012-11-29 08:54:59 -08:00
Jim Bankoski
6e02947e29 unused variable
Change-Id: I1302a6eaa840d419e8bb9ad0673e42ef139d3fee
2012-11-29 08:51:19 -08:00
Jim Bankoski
705220ee71 unused var removed
Change-Id: I9d0efdff0c79ea4bdd660098106b64776bdd4483
2012-11-29 08:50:20 -08:00
Jim Bankoski
245fba74b7 signed mismatch mvrefcount
Change-Id: Ie34820c1b6eaba9cf9316415a46f48af79c41646
2012-11-29 08:13:18 -08:00
Jim Bankoski
abd74ed594 warning error missing void
Change-Id: I914bcc669297d3414261486bf1bfb716c2ecc804
2012-11-29 07:47:50 -08:00
Jim Bankoski
030e268a90 ihtllm moves to rtcd
clears up some warnings

Change-Id: I9899637497c6ad7519f098e055ab98580ae6d688
2012-11-29 07:19:38 -08:00
Jim Bankoski
e69b5258fd fix vp9_vp8 files renamed
Change-Id: I20c426e91ee49666db42e20eb074095ab6b8ec5d
2012-11-29 06:53:08 -08:00
Jim Bankoski
13dbf1fb17 more rtcd cleanup
Change-Id: Ieefd76e164ca4aa87597da0412977614ddfbacb7
2012-11-28 17:27:15 -08:00
Deb Mukherjee
0de214260b Merge "Fixing 8x8/4x4 ADST for intra modes with tx select" into experimental 2012-11-28 16:59:17 -08:00
Deb Mukherjee
0742b1e4ae Fixing 8x8/4x4 ADST for intra modes with tx select
This patch allows use of 8x8 and 4x4 ADST correctly for Intra
16x16 modes and Intra 8x8 modes when the block size selected
is smaller than the prediction mode. Also includes some cleanups
and refactoring.

Rebase.

Change-Id: Ie3257bdf07bdb9c6e9476915e3a80183c8fa005a
2012-11-28 16:21:12 -08:00
Yaowu Xu
b2f27d909a Merge "remove the vp9_default_mode_contexts_a" into experimental 2012-11-28 13:56:42 -08:00
Yaowu Xu
1cc5739669 remove the vp9_default_mode_contexts_a
Given the way mode_context is updated, the benefit of an additional
default is not signficant.

Change-Id: I67489453e8781340b18e26a1cc2f04e9221004a2
2012-11-28 11:14:30 -08:00
Jim Bankoski
c67873989f fixed includes to be fully specified
Change-Id: Ia1cce221f8511561b9cbd8edb7726fbc286ff243
2012-11-28 10:53:17 -08:00
Jim Bankoski
926d95cd84 Merge "remove postproc invokes" into experimental 2012-11-28 10:30:42 -08:00
John Koleszar
00e2c6bf7a Merge "Clamp decoded feature data" into experimental 2012-11-28 10:08:37 -08:00
John Koleszar
b07fcf5f6f Merge "Revert "make: flatten object file directories"" into experimental 2012-11-28 10:08:22 -08:00
Jim Bankoski
85cba19e16 remove postproc invokes
and some miscellaneous invoke left overs

Change-Id: I63191b1bfd3bea4ce30cceaeb686ec850570fc43
2012-11-28 10:00:25 -08:00
Yaowu Xu
3e976bba21 Localize Y2 entropy coding context
This commit makes sure Y2 entropy coding context is always updated on
every macroblock even there is no Y2 block.

Change-Id: Ie307cfc46526efe55613be39f9f178d2531b56ba
2012-11-28 09:27:36 -08:00
Yunqing Wang
d202138621 Further improve macroblock loop filters
This change included:
1. Aligned reads in vp9_mbloop_filter_vertical_edge function.
Since we actually read 16 bytes, we can align the reads to read
starting at (s - 8) instead of (s - 5).
2. Combined u, v loop filters.
3. Added 8x16 transpose.

This gave 2% decoder performance gain (tulip clip).

Change-Id: Ib14c2f1645c4a3436df17fe2f24789506bf0bb58
2012-11-28 09:27:07 -08:00
Yaowu Xu
12da793d00 removed redundant mode_context data structures
This commit removed a couple of redundant data structures in frame
coding contextsm, mode_context and mode_context_a, and changed to
use vp9_mode_contexts only. The switch of the context for different
frame type now relies on the switch of frame coding context between
lfc and lfc_a. This commit also removed a number of memcpy among
these redundant data structure.

Change-Id: I42e8174bd60f466b0860afc44c1263896471b0f3
2012-11-28 09:24:30 -08:00
John Koleszar
a1f15814be Clamp decoded feature data
Not all segment feature data elements are full-range powers of two, so
there are values that can be encoded that are invalid. Add a new function
to clamp values to the maximum allowed.

Change-Id: Ie47cb80ef2d54292e6b8db9f699c57214a915bc4
2012-11-27 16:38:31 -08:00
John Koleszar
1760c39bce Revert "make: flatten object file directories"
This reverts commit b72373de79.

Change-Id: Ic1601160e11df1a018ef12da25967cfb5eebd5ba
2012-11-27 16:36:39 -08:00
John Koleszar
fcccbcbb39 Add vp9_ prefix to all vp9 files
Support for gyp which doesn't support multiple objects in the same
static library having the same basename.

Change-Id: Ib947eefbaf68f8b177a796d23f875ccdfa6bc9dc
2012-11-27 14:12:30 -08:00
Yunqing Wang
3bf7b131c8 Merge "Improve sad3x16 SSE2 function" into experimental 2012-11-26 10:15:35 -08:00
Paul Wilkins
fbc8e8f9ae Merge "Modified mv prediction." into experimental 2012-11-26 09:59:24 -08:00
Paul Wilkins
d22f3d9f42 Modified mv prediction.
Modified the  mv_pred() fuunction that chooses a centre
point from which to start step searches to use the top
candidate vectors chosen previously.

Some gains (mainly on HD and tested with SB off).
Std_hd 0.874%, YT-hd 0.174%, YT 0.05%, Derf 0.036%

Change-Id: Ie232284f561838b8ecee0e28dcbb07a9cd46cf56
2012-11-26 17:55:19 +00:00
Yunqing Wang
e7cd80718b Improve sad3x16 SSE2 function
Vp9_sad3x16_sse2() is heavily called in decoder, in which the
unaligned reads consume lots of cpu cycles. When CONFIG_SUBPELREFMV
is off, the unaligned offset is 1. In this situation,
we can adjust the src_ptr to be 4-byte aligned, and then do the
aligned reads. This reduced the reading time significantly. Tests
on 1080p clip showed over 2% decoder performance gain with
CONFIG_SUBPELREFM off.

Change-Id: I953afe3ac5406107933ef49d0b695eafba9a6507
2012-11-26 09:53:50 -08:00
Yaowu Xu
89d62e3b04 remove the dependency on idct.h
Change-Id: Idcf827d8ae6429ee5b673c3398f838dbeacb4e74
2012-11-26 09:12:04 -08:00
Jim Bankoski
f42e41f2ef Merge "removed the idct rtcd idct calls" into experimental 2012-11-24 21:38:36 -08:00
Ronald S. Bultje
25b609b62b Move switch(tx_size) around txsize to detokenize.c.
Add a new function vp9_decode_mb_tokens() that handles the switch
between different per-tx-size detokenize functions. Make actual
implementations (vp9_decode_mb_tokens_NxN()) static.

Change-Id: I9e0c4ef410bfa90128a02b472c079a955776816d
2012-11-24 21:22:42 -08:00
Ronald S. Bultje
9dc7d4fb97 Fix crash in pick_inter_mode_sb().
It didn't handle rd_thresh == INT_MAX, which means the reference is
unavailable.

Change-Id: Ie6fa8b2577437411db81a8c24e8dcdfd856a0e8d
2012-11-24 21:20:32 -08:00
Jim Bankoski
510557e2eb removed the idct rtcd idct calls
More cleanup to do after this,  but this is a good chunk of removing rtcd.

Change-Id: I551db75e341a0a85c3ad650df1e9a60dc305681a
2012-11-24 19:33:58 -08:00
Ronald S. Bultje
9970d8b662 Restructure vp9_decode_mb_tokens_8x8() a bit.
Don't declare variables if they only ever have a single value and are
used only as argument to another function call; instead, just hardcode
the value in the function call directly. Split out UV and Y coefficient
loops for clarity. Use xd->block[].qcoeff instead of xd->qcoeff + magic
to remove use of magic offset variables.

Change-Id: I5b17eda1bb666c69c2b7ea957d5525cd78192e33
2012-11-23 09:43:13 -08:00
Ronald S. Bultje
f090b6b47b Restructure vp9_decode_mb_tokens_16x16() a bit.
Don't declare variables if they only ever have a single value and are
used only as argument to another function call; instead, just hardcode
the value in the function call directly. Also remove unneeded brackets
around a code block, and remove the magic offsets 64 and 256 for chroma
values in the coefficient memory block.

Change-Id: I14fc14120a81ea1d6fb862674e8bf8cf6ba3d114
2012-11-23 09:11:12 -08:00
Ronald S. Bultje
0312c3d6d9 Make get_eob() function static.
Change-Id: Idde3ab97960eda7022367c1f91a873a479bc9d7b
2012-11-23 08:17:06 -08:00
Ronald S. Bultje
4422847143 Rename "block_type" function argument to "txfm_size".
Also fix the type (TX_SIZE instead of int).

Change-Id: Ib9b3f33835e58a6e758ed5f37bb64543e62b6a86
2012-11-23 08:15:00 -08:00
Jim Bankoski
91d703b2b2 Merge "remove subpixel invoke functions" into experimental 2012-11-21 19:55:16 -08:00
Ronald S. Bultje
a5e542e74b Fix enc/dec mismatch with b_context_pred experiment enabled.
Change-Id: I1272ae3f0fdfb7ed8eb364ef0c6dd1818d3179d7
2012-11-21 12:39:55 -08:00
Jim Bankoski
3338af4109 remove subpixel invoke functions
Removed the rtcd subpixel invoke functions.

Change-Id: I8b7618bd5813333fac66b2817bdf807616e0fb33
2012-11-21 09:16:30 -08:00
Jim Bankoski
e25bd474ad fixed const problem
NEEDED FOR BUILD

Change-Id: I56a3e68f15dff480b34de048e30231ba821b1ee2
2012-11-21 06:46:25 -08:00
Jim Bankoski
4ad2f08c72 Merge "clean out some of the rtcd code." into experimental 2012-11-21 06:41:37 -08:00
John Koleszar
414f68d266 Merge "Pack invisible frames without lengths" into experimental 2012-11-20 17:22:50 -08:00
Yunqing Wang
bbe5e032a4 Fix ref_stride in sad function
Used ref_stride.

Change-Id: I31f0a3bb935520f54d11a1d87315627f162ae845
2012-11-20 10:01:20 -08:00
Jim Bankoski
f4871b6a3f clean out some of the rtcd code.
This removes functions that are no longer needed and cleans up some warnings.

Change-Id: I292a4c3694e9c1d68ce99cea390905b198434719
2012-11-18 12:33:18 -08:00
Ronald S. Bultje
4db08237e0 Merge "Assign above/left context in decode_coefs() instead of in caller." into experimental 2012-11-17 14:41:15 -08:00
Ronald S. Bultje
18e42dddf2 Merge "Remove unused argument from decode_coefs() function prototype." into experimental 2012-11-17 14:41:07 -08:00
Ronald S. Bultje
d0b525656b Merge "Remove coef_bands_x[] array and related machinery in decode_coefs()." into experimental 2012-11-17 14:40:56 -08:00
Ronald S. Bultje
825b20b0ae Merge "Inline count_tokens() in decode_coefs()." into experimental 2012-11-17 14:40:48 -08:00
Ronald S. Bultje
4db4f98b52 Merge "Merge various count_token() functions into a single one." into experimental 2012-11-17 14:40:41 -08:00
Ronald S. Bultje
5d7cb59035 Assign above/left context in decode_coefs() instead of in caller.
this prevents duplicating the same line of code in each caller of
decode_coefs().

Change-Id: Id7996ad394828bf77ef3d5e03002f577c9f79609
2012-11-17 11:22:38 -08:00
Ronald S. Bultje
3bdf302ce7 Remove unused argument from decode_coefs() function prototype.
Change-Id: I8d2539ba1046012c948520ac23a1f1978be921c5
2012-11-17 11:11:06 -08:00
Ronald S. Bultje
a253b3791b Remove coef_bands_x[] array and related machinery in decode_coefs().
Change-Id: I0a36d1efb3bb81a54005b10316550ec67100559e
2012-11-17 11:07:23 -08:00
Ronald S. Bultje
511ef2072c Inline count_tokens() in decode_coefs().
This prevents the relatively expensive token-from-coefficient lookup
function get_token(), plus a duplicate loop..

Change-Id: Ibecd407b2a91d3593d439ec4646e43fa26d2ff91
2012-11-17 10:35:47 -08:00
Ronald S. Bultje
56352f189d Merge various count_token() functions into a single one.
Change-Id: I1970f43e2cb5f7d9744c7249099eed226f16f162
2012-11-17 10:18:41 -08:00
Jim Bankoski
b38b6abccc Merge "removal of temporal invoke" into experimental 2012-11-17 09:53:02 -08:00
Ronald S. Bultje
166d24d07e Remove unused function count_tokens() in detokenize.c.
Change-Id: I178f250b1a4d41d5a9c1619091f5ae51cebffb10
2012-11-17 07:45:46 -08:00
Jim Bankoski
cb98b83239 removal of temporal invoke
Change-Id: I18ca713b02a5241bdb20dddcde0216467b55b596
2012-11-17 06:11:01 -08:00
Ronald S. Bultje
f19a1cafed Remove special-case inline detokenization in b_pred reconstruction.
Just like for all other block modes, b_pred tokens can be read together
before starting macroblock reconstruction. This removes special cases
for b_pred in decode_macroblock() and allows to make decode_coefs_4x4()
static in detokenize.c.

While at it, remove the redundant handling and checking of plane_type
and block_index (i) in decode_coefs_4x4(). Since the function is static,
and is called only from decode_mb_tokens_4x4(), we don't need to worry
that the arguments ever go out of sync.

Change-Id: I2d415da0b51b89d0490a6b9e24cc86363c2090f7
2012-11-16 22:26:12 -08:00
Yunqing Wang
0eb5590425 Merge "Add const before the dequant(dq)" into experimental 2012-11-16 12:35:17 -08:00
Yunqing Wang
4c7c15ee69 Merge "Optimize 8x8 dequant and idct" into experimental 2012-11-16 12:23:06 -08:00
Yunqing Wang
47d9d48fa4 Add const before the dequant(dq)
Modified code to use const before dq.

Change-Id: I6fa59c2ed9743ded33ad08df70e15c2fe1ae7b99
2012-11-16 12:13:13 -08:00
Ronald S. Bultje
5b11052ac1 Support 32x32 intra modes in non-keyframe superblocks.
Change-Id: Icf8ad313c543462e523bff89690e5daa8d49bcc0
2012-11-16 09:54:43 -08:00
Paul Wilkins
a57dbd957b Further experimentation with the mode context
Experiments with a larger set of contexts and some
clean up to replace magic numbers regarding the
number of contexts.

The starting values and rate of backwards adaption
are still suspect and based on a small set of tests.
Added forwards adjustment of probabilities.

The net result of adding the new context and forward
update is small compared to the old context from the
legacy find_near function.  (down a little on derf but
up by a similar amount for HD)

HOWEVER.... with the new context and forward update
the impact of disabling the reverse update (which may be
necessary in some use cases to facilitate parallel decoding)
is hugely reduced.

For the old context without forward update, the impact of
turning off reverse update (Experiment was with SB off) was
Derf - 0.9, Yt -1.89, ythd -2.75 and sthd -8.35. The impact was
mainly at low data rates.

With the new context and forward update enabled the impact
for all the test sets was no more than 0.5-1% (again most at
the low end).

Change-Id: Ic751b414c8ce7f7f3ebc6f19a741d774d2b4b556
2012-11-16 16:58:00 +00:00
John Koleszar
6bca6decbf Merge "Don't write recon.yuv by default" into experimental 2012-11-16 08:41:40 -08:00
Deb Mukherjee
cb2d06ceac Merge "Compound inter-intra experiment" into experimental 2012-11-16 08:30:34 -08:00
Yaowu Xu
170305dcd3 Merge "changed mv candidate search for superblocks" into experimental 2012-11-16 07:21:55 -08:00
Yaowu Xu
415e6bff4d changed mv candidate search for superblocks
added additional motion vectors at close neighborhood of a superblock
to the list of candiate motion vectors, and removed a couple that are
further away.

The change helped std-hd set about .8% (all metrics) and smaller gain
for derf set.

Change-Id: Iaa69b98614db43420ed3fd4738d0ca5587b90045
2012-11-16 07:01:13 -08:00
Deb Mukherjee
0c917fc975 Compound inter-intra experiment
A patch on compound inter-intra prediction.

In compound inter-intra prediction, a new predictor for
16x16 inter coded MBs are obtained by combining a single
inter predictor with a 16x16 intra predictor, in a manner
that the weight varies with distance from the top/left
boundary. The current search strategy is to combine the best
inter mode with the best intra mode obtained independently.

Results so far:

derf +0.31%
yt +0.32%
std-hd +0.35%
hd +0.42%

It is conceivable that the results would improve somewhat
with a more thorough search strategy where all intra modes
are searched given the best mv, or even a joint search for
the best mv and the best intra mode.

Change-Id: I7951f1ed0d6eb31ca32ac24d120f1585bcd8d79b
2012-11-16 06:56:29 -08:00
Yaowu Xu
1c56946ec1 Merge "subpelrefmv for superblocks" into experimental 2012-11-16 05:49:32 -08:00
John Koleszar
64bcffc1ec Pack invisible frames without lengths
Modify the decoder to return the ending position of the bool decoder and
use that as the starting position for the next frame.

The constant-space algorithm for parsing the appended frame lengths is
O(n^2), which is a potential DoS concern if n is unbounded. Revisit
the appended lengths for use as partition lengths when multipartition
support is added.

In addition, this allows decoding of raw streams outside of a container
without additional framing information, though it's insufficient to
be able to remux said stream into a container.

Change-Id: I71e801a9c3e37abe559a56a597635b0cbae1934b
2012-11-15 15:48:07 -08:00
Yaowu Xu
61416aedc2 subpelrefmv for superblocks
duplicate code clean-up and variable name corrections

Change-Id: Ibc4703228e652ec425125de5e7bc038fa46595c5
2012-11-15 13:46:52 -08:00
John Koleszar
a9c7597adc support building vp8 and vp9 into a single lib
Change-Id: Ib8f8a66c9fd31e508cdc9caa662192f38433aa3d
2012-11-15 10:46:17 -08:00
John Koleszar
b72373de79 make: flatten object file directories
Rather than building an object file directory heirarchy matching the
source tree's layout, rename the object files so that the object
file name contains the path in the source file tree. The intent here
is to allow two files in different parts of the source tree to have
the same name and still not collide when put into an ar archive.

Change-Id: Id627737dc95ffc65b738501215f34a995148c5a2
2012-11-15 10:44:58 -08:00
John Koleszar
6becad426c detokenize: use SEG_LVL_EOB feature consistently
Update decode_coefs() to break when c >= eob, since it's possible that
c starts the loop from 1 and eob is 0. The loop won't terminate in that
case.

Add new get_eob() function to consistently clamp the eob based on the
segment level EOB and the block size. It's possible to code a segment
level EOB that's greater than the block size, and that leads to an
out of bounds access.

Change-Id: I859563b30414615cf1b30dcc2aef8a1de358c42d
2012-11-15 11:44:29 +00:00
pascal massimino
5a955973d9 Merge changes I63348ae3,I658ea409 into experimental
* changes:
  Segment mode coding bug.
  Silenced a few warnings.
2012-11-15 00:24:57 -08:00
Ronald S. Bultje
120690989b Merge "fix costing bug in pick_uv_sb_mode." into experimental 2012-11-14 17:05:46 -08:00
Ronald S. Bultje
d7290d4974 Merge "Merge a few mostly-duplicate code fragments in SB/MB encoding." into experimental 2012-11-14 17:05:40 -08:00
Ronald S. Bultje
a77df0c473 Merge "Prevent overflow in variance32x32." into experimental 2012-11-14 15:43:19 -08:00
Ronald S. Bultje
a653c9d286 fix costing bug in pick_uv_sb_mode.
Change-Id: Ia24e0fddcca9125f8e41e95dbb22444dc51767c7
2012-11-14 15:19:45 -08:00
Ronald S. Bultje
fa1b356e4e Merge a few mostly-duplicate code fragments in SB/MB encoding.
Change-Id: I8e12fbab7ec4732b6400ae3a6964749d818c90c9
2012-11-14 15:19:45 -08:00
Ronald S. Bultje
a099370370 Prevent overflow in variance32x32.
Change-Id: I478878c78ef8a770186622d987d318176827ef5f
2012-11-14 15:18:21 -08:00
John Koleszar
16e2686682 Merge "SEG_LVL_MODE: don't code ref_frame if it's implicit" into experimental 2012-11-14 09:39:25 -08:00
Ronald S. Bultje
127836d11f Merge "Don't use hybrid transform (ADST) for superblocks." into experimental 2012-11-14 09:18:34 -08:00
Ronald S. Bultje
1e3dd49fe3 Don't use hybrid transform (ADST) for superblocks.
This is in line with other cases where we disable ADST if prediction
size and transform size don't match. Before this patch, the RD loop
will use ADST for superblocks, but frame encoding/decoding won't.

Change-Id: I700368c632eb72b5e089c22ef25649d99d7697d0
2012-11-14 08:58:24 -08:00
Paul Wilkins
b527c4dbb7 Segment mode coding bug.
There are now more than 16 possible modes so 5
bits required for segment mode feature.

Note that it is likely that the mode feature and how it is
coded will change but for now the 4 bits was a bug.

Change-Id: I63348ae3a9cc31566a656c2dc78f09f5e1a9dcc9
2012-11-14 14:38:03 +00:00
Paul Wilkins
19a1ba1e91 Silenced a few warnings.
Silenced a few VS compiler warnings.

Change-Id: I658ea409c36c05cd11042675e2e42ccde0ef2420
2012-11-14 14:27:37 +00:00
John Koleszar
854e41f057 Don't write recon.yuv by default
CONFIG_DEBUG was turning on some code to dump the reconstructed frame
to a buffer from within the decoder. Move this code to a more specific
debugging define.

Change-Id: I3ca9ea634bdbd186f2470bd644d3695ee0ab3037
2012-11-13 15:22:35 -08:00
John Koleszar
6d482706ef SEG_LVL_MODE: don't code ref_frame if it's implicit
If the SEG_LVL_MODE is an intra mode, then the reference frame must be
INTRA_FRAME.

Change-Id: I2cdeeac3780c077c74b39ce89a528bc280674231
2012-11-13 15:22:09 -08:00
Yaowu Xu
3fa1348d5f fix a few typos
Change-Id: I7b6f27826052eb706fc6080d4e3a940dff7d3a58
2012-11-13 14:45:53 -08:00
Ronald S. Bultje
1761a6b55a Merge "Use full 32-pixel edge for superblock bestrefmv motion vector ordering." into experimental 2012-11-13 14:12:58 -08:00
Ronald S. Bultje
b147c64c16 Merge "Fix edge MV handling in SBs." into experimental 2012-11-13 14:12:48 -08:00
Deb Mukherjee
7de64f35d3 A fix in MV_REF experiment
This fix ensures that the forward prob update is not turned off for
motion vectors.

Change-Id: I0b63c9401155926763c6294df6cca68b32bac340
2012-11-13 08:27:04 -08:00
Yunqing Wang
e60478d46d Optimize 8x8 dequant and idct
Similar to 16x16 dequant and idct, based on the value of eobs, the
8x8 dequant and idct calculation was simplified to improve decorder
performance.

Combined vp9_dequant_idct_add_8x8 and vp9_dequant_dc_idct_add_8x8
to eliminate duplicate code.

Change-Id: Ia58e50ab27f7012b7379c495837c9c0b5ba9cf7f
2012-11-12 17:41:53 -08:00
Ronald S. Bultje
c79ae1713c Use full 32-pixel edge for superblock bestrefmv motion vector ordering.
Change-Id: I417e39867c020a17d85370972446a8ce2bbe9a6d
2012-11-12 17:06:56 -08:00
Ronald S. Bultje
722972454c Fix edge MV handling in SBs.
Change-Id: Ia1eddb108ec463835e9de8769572d698e21bca49
2012-11-12 17:06:52 -08:00
Paul Wilkins
5d65614fdd Merge "New inter mode context" into experimental 2012-11-12 09:24:14 -08:00
Paul Wilkins
2669f42b0d New inter mode context
This change is a fix / extension of the newbestrefmv
experiment. As such it is presented without IFDEF.

The change creates a new context for coding inter modes
in vp9_find_mv_refs(). This replaces the context that
was previously calculated in vp9_find_near_mvs().
The new context is unoptimized and not necessarily
any better at this stage (results pending), but eliminates
the need for a legacy call to vp9_find_near_mvs().
Based on numbers from Scott, this could help decode
speed by several %.

In a later patch I will add support for forward update of
context (assuming this helps) and refine the context as
necessary.

Change-Id: I1cd991b82c8df86cc02237a34185e6d67510698a
2012-11-12 15:50:02 +00:00
Ronald S. Bultje
3a08b033b0 Merge "Fix data type for eobs[] array in SB 4x4 IDCT code." into experimental 2012-11-12 07:40:54 -08:00
Ronald S. Bultje
11fec1863d Merge "Remove 'thismb' data pointer when superblock experiment is on." into experimental 2012-11-12 07:22:22 -08:00
Paul Wilkins
6fb8953c19 Restrict ref mv search range.
Experiment to test speed trade off of reducing the
extent of the ref mv search.

Reducing the maximum number of tested candidates to 9 had
minimal net effect on quality in any of the tests sets.

Reduction to 7 has a small negative impact (worst was STD-HD
at about -0.2%).

This change is in response to the apparently high number of
decode cycles reported in regard to mv-ref selection.

Change-Id: I0e92e92e324337689358495a1ec9ccdeb23dc774
2012-11-12 11:31:12 +00:00
Ronald S. Bultje
dd9d4f9e1a Fix data type for eobs[] array in SB 4x4 IDCT code.
This fixes encoder/decoder mismatches with the superblock experiment
turned on whenever a superblock is encoded using the 4x4 transform.

Change-Id: Iefec7055e8d25f8efdbba66c4261bbd322d335a3
2012-11-10 12:08:27 -08:00
Ronald S. Bultje
73987d140a Remove 'thismb' data pointer when superblock experiment is on.
This should prevent inconsistent results between identical encodes with
the superblock experiment turned on.

Change-Id: I41a005fae53f2eb59736cc70041185fb7d63cfca
2012-11-10 08:39:51 -08:00
Deb Mukherjee
d01357bbad New b-intra mode where direction is contextual
Preliminary patch on a new 4x4 intra mode B_CONTEXT_PRED where the
dominant direction from the context is used to encode. Various decoder
changes are needed to support decoding of B_CONTEXT_PRED in conjunction
with hybrid transforms since the scan order and tokenization depends on
the actual direction of prediction obtained from the context. Currently
the traditional directional modes are used in conjunction with the
B_CONTEXT_PRED, which also seems to provide the best results.

The gains are small - in the 0.1% range.

Change-Id: I5a7ea80b5218f42a9c0dfb42d3f79a68c7f0cdc2
2012-11-10 07:12:30 -08:00
Deb Mukherjee
3f7182cb0d Build fix in decoder/decodframe.c
Missing eobs agrument in vp9_dequant_idct_add_16x16_c

Change-Id: I826b1afa0a4ee6398f7373325aa0c75e6a866937
2012-11-09 12:48:35 -08:00
John Koleszar
3a0cfb3617 Merge "Packing Altref along with succeeding frame and length encoding frames" into experimental 2012-11-09 12:31:37 -08:00
Vignesh Venkatasubramanian
bc9670eee0 Packing Altref along with succeeding frame and length encoding frames
The altref frame is packed along with the next P frame. So that
outside of the codec there are now only two types of frames P and I.
Also, now it is one frame in and one frame out with respect to the
codec. Apart from that, all the frames are length encoded with the
length of each frame appended to the frame itself. There are
two categories of frames and each of them will look as follows:

  - Packed frames (an altref along with the succeeding p frame)
    - altref_frame_data | altref_lenngth | frame_data | length
  - Unpacked frames (all frames other than the above)
    - frame_data | length

Change-Id: If1eabf5c473f7d46b3f2d026bd30c803588c5330
2012-11-09 12:04:53 -08:00
Yunqing Wang
71b1885403 Merge "Optimize 16x16 dequant and idct" into experimental 2012-11-09 08:30:53 -08:00
Jim Bankoski
a186eb7f1b Merge "remove macros obfuscating mv costing" into experimental 2012-11-08 15:51:21 -08:00
Jim Bankoski
c72be96b0a remove macros obfuscating mv costing
cleanup

Change-Id: I565eee40d900e0441ad211b65ac829fc5b93d94a
2012-11-08 15:44:39 -08:00
Ronald S. Bultje
1d4fbeb32a Implement tx_select for superblock encoding.
Also split superblock handling code out of decode_macroblock() into
a new function decode_superblock(), for easier readability.

Derf +0.05%, HD +0.2%, STDHD +0.1%. We can likely get further gains
by allowing to select mb_skip_coeff for a subset of the complete SB
or something along those lines, because although this change allows
coding smaller transforms for bigger predictors, it increases the
overhead of coding EOBs to skip the parts where the residual is
near-zero, and thus the overall gain is not as high as we'd expect.

Change-Id: I552ce1286487267f504e3090b683e15515791efa
2012-11-08 11:03:00 -08:00
Yunqing Wang
6c17c9fae0 Optimize 16x16 dequant and idct
As suggested by Yaowu, simplified 16x16 dequant and idct. In decoder,
after detoken step, we know the number of non-zero dct coefficients
(eobs) in a macroblock. Idct calculation can be skipped or simplified
based on eobs, which improves the decoder performance.

Change-Id: I9ffa1cb134bcb5a7d64fcf90c81871a96d1b4018
2012-11-07 20:04:09 -08:00
John Koleszar
8959c8b11d Merge with upstream experimental changes (2)
Include upstream changes (variance fixes) into the merged code base.

Change-Id: I4182654c1411c1b15cd23235d3822702613abce1
2012-11-07 14:32:26 -08:00
James Zern
5338d983d6 Merge "Fix variance (signed integer) overflow" into experimental 2012-11-07 12:49:36 -08:00
John Koleszar
2c08c28191 Merge with upstream experimental changes
Include upstream changes (unit test fixes, in particular) into the
merged code base.

Change-Id: I096f8a9d09e2532fbec0c95d7a995ab22fa54b29
2012-11-07 11:46:23 -08:00
John Koleszar
7b8dfcb5a2 Rough merge of master into experimental
Creates a merge between the master and experimental branches. Fixes a
number of conflicts in the build system to allow *either* VP8 or VP9
to be built. Specifically either:

  $ configure --disable-vp9 $ configure --disable-vp8
  --disable-unit-tests

VP9 still exports its symbols and files as VP8, so that will be
resolved in the next commit.

Unit tests are broken in VP9, but this isn't a new issue. They are
fixed upstream on origin/experimental as of this writing, but rebasing
this merge proved difficult, so will tackle that in a second merge
commit.

Change-Id: I2b7d852c18efd58d1ebc621b8041fe0260442c21
2012-11-07 11:30:16 -08:00
Yaowu Xu
0cedaa3631 merge full pixel refmv experiment
Change-Id: Ib39ad47a7d188f3b45416937b7eeb28c3e79b74c
2012-11-07 10:52:45 -08:00
James Zern
984734436d Fix variance (signed integer) overflow
In the variance calculations the difference is summed and later squared.
When the sum exceeds sqrt(2^31) the value is treated as a negative when
it is shifted which gives incorrect results.

To fix this we force the multiplication to be unsigned.

The alternative fix is to shift sum down by 4 before multiplying.
However that will reduce precision.

For 16x16 blocks the maximum sum is 65280 and sqrt(2^31) is 46340 (and
change).

This change is based on:
1698234 Missed some variance casts
fea3556 Fix variance overflow

Change-Id: I2c61856cca9db54b9b81de83b4505ea81a050a0f
2012-11-06 23:06:44 -08:00
Yaowu Xu
a879b4e6d4 fixed function prototype
so they are consistent with actual definitions of the functions

Change-Id: Ie4b4e81b3da3e288fc2edbbd2b393a5c54d2556b
2012-11-06 15:55:11 -08:00
Yaowu Xu
acadcec5c5 group refmv experiment related functions
Change-Id: Iedaa108ddb65f54d768424f9c47ad4d069b656fd
2012-11-06 15:54:47 -08:00
James Zern
182f99f0c6 Merge "fix test builds" into experimental 2012-11-06 12:18:01 -08:00
James Zern
2e3e685799 fix test builds
s/([vV][pP])8/$19/
additionally dct.h was removed; declare the _c functions that are used
in the tests. the TODO for conversion to parameterized tests still
remains.

Change-Id: I73db9425a57075bbb78a92693ba6b320578981cd
2012-11-06 12:12:58 -08:00
John Koleszar
83b1d907da vpx: merge with master
Change-Id: I44b3ad780cef6f448fa17ff8e28fea87ef9cd518
2012-11-06 12:04:53 -08:00
Yunqing Wang
4626faf1e7 Convert 16x16 dct/idct to integer forms
Converted vp9_short_fdct16x16_c and vp9_short_idct16x16_c to
integer versions.

Change-Id: Ie3ec985a890ac0f4f4f5818e6f0122e00c8af69f
2012-11-06 11:25:25 -08:00
James Zern
0078d2f3dc vp9/encoder/bitstream.c: fix unused variable warnings
Change-Id: Ibfac7e000509d2017eac9a108060e534a19fec33
2012-11-06 11:08:34 -08:00
Yaowu Xu
55f2f14f10 Merge "silent a lot of MSVC compiler warnings" into experimental 2012-11-06 09:39:47 -08:00
Yaowu Xu
8a336b0d0d silent a lot of MSVC compiler warnings
there are still a couple type of warning left, which are related to
double constants assigned to float type. As those would be addressed
by the conversion of transforms into integer version. This commit
has left those un-dealt with.

Change-Id: I48fd9b489c0c27ad6b543f4177423419f929f2bb
2012-11-06 09:09:25 -08:00
Jim Bankoski
8ce914f5fd Merge "remove invoke_search macro" into experimental 2012-11-06 06:31:52 -08:00
James Zern
e47d9f1d07 rd_pick_inter_mode: prevent signed integer overflow
calculate the txfm_cache difference first as both values may be
INT64_MAX with the intent that they cancel each other out.

Change-Id: I214d072458e1b24f60289974e6302af1aff7b66c
2012-11-05 17:14:32 -08:00
Jim Bankoski
7849aa20ed remove invoke_search macro
Removed invoke search from encoder

Change-Id: I3d809b795abe6df0e71366edfe94026aaede14fb
2012-11-05 16:58:03 -08:00
James Zern
f2541f8a4a rdopt: fix use of uninitialized value in addition
rd_pick_intra4x4mby_modes / rd_pick_intra8x8mby_modes would both use the
input value of 'rate_y' in the return calculation. In many places this
value is uninitialized. Remove the unneeded sum.

Change-Id: Icbd3df685303000301e69291c0ebc06f74bd548d
2012-11-05 12:50:16 -08:00
Ronald S. Bultje
849c9540d5 Merge "Don't generate residual 3x when doing a macroblock luma RD estimate." into experimental 2012-11-05 06:21:03 -08:00
James Zern
ee38c4184b loopfilter: prevent signed integer overflow
use unsigned ints to extended filter values in
vp9_mbloop_filter_horizontal_edge_c_sse2

Change-Id: I55ec3ac2bcb9baf55626b0384d151b07fc8e087d
2012-11-03 09:45:21 -07:00
Yunqing Wang
28826a913c Merge "Fix eobs data type" into experimental 2012-11-02 16:00:56 -07:00
Yunqing Wang
d41b0e6498 Fix eobs data type
The block sizes for decoding tokens are up to 16x16, which means
eobs is within [0, 256]. Using (signed) char is not enough. Changed
eobs data type to unsigned short to fix the problem.

Change-Id: I88a7d3098e1f1604c336d6adb88ffec971fb03a6
2012-11-02 13:22:29 -07:00
Ronald S. Bultje
6cd2541379 Don't generate residual 3x when doing a macroblock luma RD estimate.
Change-Id: Ia601e96fcb4fc547884b6ab894f9f2ad22a98078
2012-11-02 11:46:57 -07:00
Ronald S. Bultje
3c4f47e843 Place non-static function prototypes in a header file.
Change-Id: I7cd21b9f1e69f4e0b3338bfe27b3c67e4b47de58
2012-11-02 11:22:57 -07:00
John Koleszar
06f3e51da6 vpx_scale: sync from master
Update vpx_scale from current code in master, run style transform, fix
lint warnings.

Change-Id: I47eadeb5b6881d448ea3728537f9b8a5b5aac78e
2012-11-02 08:44:54 -07:00
Ronald S. Bultje
4b2c2b9aa4 Rename vp8/ codec directory to vp9/.
Change-Id: Ic084c475844b24092a433ab88138cf58af3abbe4
2012-11-01 16:31:22 -07:00