Commit Graph

1250 Commits

Author SHA1 Message Date
John Koleszar
e559e14fa6 Make setup_pred_block subsampling-aware.
Code previously set up the pointers by scaling by MI_UV_SIZE, which
is 4:2:0 only.

Change-Id: Ic13a92895cff018ec1345736746ed84cb31e6e31
2013-05-07 11:47:45 -07:00
John Koleszar
64c0b3075e Use common setup_dst_planes function
Previous code assumed 4:2:0 subsampling.

Change-Id: Ie0814e45d099e84d0ded880b62e7bc98f85b7f0c
2013-05-07 10:25:54 -07:00
Jingning Han
c0504a9b24 Merge "Merge SB8X8 into the codebase" into experimental 2013-05-07 09:23:47 -07:00
Jingning Han
776c1482a3 Merge SB8X8 into the codebase
Pull sb8x8 out of experimental list. verified via borg run tests.
Fixed unit test failures.

Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-07 09:08:25 -07:00
Scott LaVarnway
9c7d06e6f3 Merge "Removed vp9_setup_intra_recon()" into experimental 2013-05-07 08:24:26 -07:00
Jingning Han
8c70a0fcab Merge "Fix tile independency issue in sb8x8" into experimental 2013-05-06 22:28:04 -07:00
John Koleszar
aede2a1a00 Merge "Remove heap alloc of FIRSTPASS_STATS" into experimental 2013-05-06 22:20:11 -07:00
Jingning Han
35e8072613 Fix tile independency issue in sb8x8
Resolved the unit test failure on tile independence when sb8x8
enabled.

Change-Id: I23ca86c88170e49ca160e8e897d913905e6080ce
2013-05-06 21:56:45 -07:00
Dmitry Kovalev
e536e17881 Merge "Adding model_rd_for_sb function." into experimental 2013-05-06 17:17:18 -07:00
John Koleszar
6462eebbdd Remove heap alloc of FIRSTPASS_STATS
There is only one instance of these structures, no need for them
to be allocated separately on the heap.

Change-Id: I1333cc92d06bbe21be643c2b2f0e3936f0264cac
2013-05-06 14:36:08 -07:00
Dmitry Kovalev
f7c81ccff0 Removing 3 unused bits from bitstream.
Change-Id: I9cd21c5c9107d6197caab6949700b29c51658af3
2013-05-06 14:00:58 -07:00
Scott LaVarnway
cb7955d83e Removed vp9_setup_intra_recon()
This setup is now handled by vp9_build_intra_predictors()
when left_available and/or up_available is zero.

Change-Id: I59cec0ab95f8be69ce885fd20727510e4deef8a0
2013-05-06 16:13:06 -04:00
Dmitry Kovalev
2e5f0084f3 Adding model_rd_for_sb function.
Iterating over all planes in the loop instead of custom y,uv code inside
handle_inter_mode function.

Change-Id: I301f9276d6d544c2fd7203d84f1318ac80ea625d
2013-05-06 12:42:53 -07:00
Johann
231faf91c3 Merge "Automatically flag intrinsic files" into experimental 2013-05-06 11:04:04 -07:00
Jingning Han
8e1c97cf73 Fix a unit test failure of sb8x8 on scaling ref
Disable the use of scaled reference frame for motion search in
SPLITMV mode. This fixes the unit test failure issue triggered
when merging sb8x8 from experimental list.

Change-Id: I02ac25fd8db8d5762f8fee29513b947189875fa0
2013-05-06 10:28:18 -07:00
Ronald S. Bultje
f7fa367094 Fix first-pass intra4x4 for sb8x8 experiment.
Change-Id: I1df17f45721c690d157800daa6a0b377e3d32bc2
2013-05-04 15:49:41 -07:00
Ronald S. Bultje
842c573e04 Merge "Fix overflow in RD error calculation code." into experimental 2013-05-03 18:03:06 -07:00
John Koleszar
acc9c125dd Remove old_block_idx_4x4
Removes several instances where the old block numbering was
still in use.

Change-Id: Id35130591455a4abe6844613e45c0b70c1220c08
2013-05-03 17:19:13 -07:00
John Koleszar
6c622e2783 Merge "Separate transform and quant from vp9_encode_sb" into experimental 2013-05-03 17:19:01 -07:00
Johann
a62fcbea30 Automatically flag intrinsic files
Change-Id: Iee9894615265d42aa23c43a4183924953aedb0c6
2013-05-03 15:35:13 -07:00
John Koleszar
4529c68b3b Separate transform and quant from vp9_encode_sb
This allows removing a large number of transform size specific functions,
as well as supporting 444/alpha by routing all code through the
subsampling-aware path.

Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03 12:14:50 -07:00
James Zern
15879fb457 Merge "vp9/count_segs: fix out of bounds read" into experimental 2013-05-03 11:27:45 -07:00
Ronald S. Bultje
ee808e52bd Fix overflow in RD error calculation code.
Change-Id: I61ef1f198c876f9f79787ea7d7385a862cfbae19
2013-05-03 10:33:07 -07:00
John Koleszar
f07733010b Merge "Create common vp9_encode_sb{,y}" into experimental 2013-05-03 10:26:53 -07:00
Adrian Grange
7aae782c37 Merge "Extend number of reference buffers to 8." into experimental 2013-05-03 09:59:54 -07:00
Adrian Grange
d7eea782f2 Extend number of reference buffers to 8.
The number of reference buffers is extended to 8 and
a reference sign-bias added for the LAST_FRAME.

Whilst the number of reference buffers used by an
individual frame remains unchanged at 3, these may
now be selected from 8 possible buffers.

Change-Id: I2d247b9c1c2b3a339d6c9fac125e81ba373f75a7
2013-05-03 09:17:18 -07:00
Scott LaVarnway
be7d4f96a2 Merge "Removed is_inter_mode() check" into experimental 2013-05-03 07:34:29 -07:00
Scott LaVarnway
3041cf8c8b Merge "Reduced y_dequant, uv_dequant size" into experimental 2013-05-03 07:30:31 -07:00
Ronald S. Bultje
034928843f Fix use of wrong rate/distortion variables in 16x8 r/d check.
Change-Id: Ib5961b4c8ca84d54c84b2651a4e0317c72fe7da4
2013-05-02 21:03:38 -07:00
Dmitry Kovalev
183aeecd74 Merge "Removing MAXF macro and using MAX instead." into experimental 2013-05-02 20:48:36 -07:00
James Zern
bc3c6d0a15 vp9/count_segs: fix out of bounds read
don't access mi until mi_{row,col} validation has been done

Change-Id: Ie1be052612cfccb2730dff9281c222bca28bf2c6
2013-05-02 19:11:28 -07:00
Ronald S. Bultje
1069c12cf4 Fix 16x16-iteration indexing bug in main encode_sb_row loop.
With this, encoder/decoder appear to match with sb8x8 experiment.
Needs some larger-scale testing.

Change-Id: I44d3cac37b3c98264985ed0a0fc763c30089aa64
2013-05-02 16:41:08 -07:00
Jingning Han
879a2f053d Fix state update in sb8x8 rate-distortion loop
Update mode_info of 8x8 blocks within the scope of current block.

Change-Id: I110c599e60664a5acde6afd919b107cea8419a0d
2013-05-02 14:41:51 -07:00
Jingning Han
1e26cdc178 Merge "Cache splitmv motion vectors appropriately in decoder." into experimental 2013-05-02 14:32:57 -07:00
Dmitry Kovalev
20eef67573 Adding encode_loopfilter function.
Moving code from huge vp9_pack_bitstream to encode_loopfilter function.

Change-Id: Idceb836da5b346cd54fc33402283e7cfb6d9097d
2013-05-02 14:23:56 -07:00
John Koleszar
3f4e80634b Create common vp9_encode_sb{,y}
Creates a common encode (subtract, transform, quantize, optimize,
inverse transform, reconstruct) function for all sb sizes, including
the old 16x16 path.

Change-Id: I964dff1ea7a0a5c378046a069ad83495f54df007
2013-05-02 14:02:03 -07:00
Ronald S. Bultje
c8de931e06 Merge "Fix i4x4 mode reading and writing in sb8x8 bitstream." into experimental 2013-05-02 13:56:06 -07:00
Ronald S. Bultje
168d313888 Cache splitmv motion vectors appropriately in decoder.
This fixes some problems in splitmv-related bitstream parsing in the
sb8x8 experiment.

Change-Id: Id168f9ec9c88626877d8f536e6f394b6f4f782e4
2013-05-02 13:55:35 -07:00
Ronald S. Bultje
e931dac733 Fix i4x4 mode reading and writing in sb8x8 bitstream.
Don't allow i4x4 except for sb8x8 recursion step. Read only 4 (not 16)
i4x4 submodes if we are i4x4.

Change-Id: Iaaaced1a134006b2c96eed66f014300eae41e0ed
2013-05-02 13:01:09 -07:00
Ronald S. Bultje
1cc0e67d85 Merge "Set transform size correctly for non-tx_select splitmv/i4x4_pred." into experimental 2013-05-02 12:35:59 -07:00
Ronald S. Bultje
06df1f82fb Set transform size correctly for non-tx_select splitmv/i4x4_pred.
Fixes mismatch in keyframes with sb8x8 experiment enabled.

Change-Id: Id44c25e1cd32be5c6543ed8cd840a5b0c8e726f1
2013-05-02 12:26:40 -07:00
Dmitry Kovalev
7ab2d7bf55 Removing MAXF macro and using MAX instead.
Change-Id: I51c53692b1150005645bf362c5e5a8275178a8fd
2013-05-02 11:57:16 -07:00
Ronald S. Bultje
96f8171d09 Merge "Fix intra4x4 decoder reconstruction stride for sb8x8 experiment." into experimental 2013-05-02 11:10:07 -07:00
Ronald S. Bultje
6c8071b45c Fix intra4x4 decoder reconstruction stride for sb8x8 experiment.
Change-Id: I5a886a6fa19086629a274f9ada7a862fd0ddf62f
2013-05-02 11:01:02 -07:00
Ronald S. Bultje
ef6007867f Merge "Always initialize values in sb/mb_partitioning." into experimental 2013-05-02 10:53:51 -07:00
Scott LaVarnway
37c3144ed7 Removed is_inter_mode() check
If a reference frame is inter, the only valid modes would
be inter modes.  This check is unnecessary.

Change-Id: Ib8433ab5a3418f94149ee4e3062d48d7740d225a
2013-05-02 13:44:33 -04:00
Dmitry Kovalev
519d9f3e16 Merge "Using treed_read/treed_write functions for segment ids." into experimental 2013-05-02 10:40:58 -07:00
Ronald S. Bultje
4aad30a3cf Always initialize values in sb/mb_partitioning.
This doesn't affect the output, because in previous cases where the
values were uninitialized, this was because the mb_row/col is outside
the codable area, and thus encode_sb will test them for the next
decomposition-level, but return right after that on size-check. All
this does is prevent a warning in valgrind.

Change-Id: I90d8a29e6f8ebb2b0143684e08fe77ae3a0816b1
2013-05-02 10:40:53 -07:00
Ronald S. Bultje
704fb4866e Fix right-edge availability for intra prediction in sb8x8.
Fixes valgrind uninitialized value use warnings.

Change-Id: Ie9314d684e2ad194f8aca5bde1729fb9b7c0221d
2013-05-02 10:16:48 -07:00
Ronald S. Bultje
ec6cf519d1 Merge "Fix some more offset errors in sb8x8." into experimental 2013-05-02 09:08:33 -07:00
Jingning Han
73a4824c34 Merge "Fix bug in sb8x8 partition context" into experimental 2013-05-02 09:04:28 -07:00
Ronald S. Bultje
3e345cd4d8 Fix some more offset errors in sb8x8.
Change-Id: I83677227f7610fdf2db9f15f87fecd4d8e072427
2013-05-02 07:54:18 -07:00
Ronald S. Bultje
dd1e6b8e6f Merge "Fix block reconstruction with sb8x8 enabled." into experimental 2013-05-02 07:11:36 -07:00
Jingning Han
ba24a28f69 Fix bug in sb8x8 partition context
Fix the issue that causes array bound excess in getting partition
context.

Change-Id: I66166f047f0bcaefebb0bcf441c5b1f777d8da44
2013-05-01 22:34:27 -07:00
Jingning Han
a427913864 Merge "Fix bugs in sb8x8 experiment/context prob update" into experimental 2013-05-01 20:52:07 -07:00
Ronald S. Bultje
ff37688a91 Fix block reconstruction with sb8x8 enabled.
The encoder reconstruction is now correct. Decoder to follow shortly.

Change-Id: Iedf98cdaebb4ca1256c7714cad7024a75853ad6a
2013-05-01 19:28:17 -07:00
Ronald S. Bultje
abb66ef0ba Merge "Fix bug in sb8x8 bitstream decoding." into experimental 2013-05-01 19:26:26 -07:00
Ronald S. Bultje
31b842637f Merge "Store splitmv modes in context after 8x8 rd loop." into experimental 2013-05-01 17:26:46 -07:00
Ronald S. Bultje
29492a5e14 Fix bug in sb8x8 bitstream decoding.
Change-Id: I7ac69a369851b09d9422f2e3c62b12b3fcd49a4c
2013-05-01 17:13:27 -07:00
Ronald S. Bultje
f37d8400db Store splitmv modes in context after 8x8 rd loop.
Change-Id: I07aa89a67e0ac5f99ef0c448553dbc46b0ed27f2
2013-05-01 17:13:23 -07:00
Jingning Han
b8decb0313 Fix bugs in sb8x8 experiment/context prob update
Fix bugs occur in contextual partition probability update, when
sb8x8 is enabled.

Change-Id: I19e2cec8a54c2dafd2be2803bbfde7337a2ae45f
2013-05-01 15:16:50 -07:00
Johann
9b94f647cd Merge "Remove unused quantize optimizations." into experimental 2013-05-01 15:13:42 -07:00
Ronald S. Bultje
b6c2d872f0 Fix some crashes in sb8x8 experiment.
Change-Id: I390bb1cedc835f439fd5dd6cda6572b29cbb139c
2013-05-01 14:45:27 -07:00
Jingning Han
650e632400 Merge "Enable bit-stream support to SB8X8" into experimental 2013-05-01 13:48:14 -07:00
Scott LaVarnway
94ed11d89d Reduced y_dequant, uv_dequant size
Currently, only two values are used.  Removed the unused
values.

Change-Id: Idc5b8be354d84ffc68df39ea3e45f9f50d977b35
2013-05-01 16:25:10 -04:00
Jingning Han
2bf1dc2e23 Enable bit-stream support to SB8X8
This commit enables bit-stream writing and reading for recursive
partition down to block 8x8.

Change-Id: I163cd48d191cc94ead49cbb7fc91374f6bf204e2
2013-05-01 13:03:54 -07:00
Dmitry Kovalev
6e4ed2f0fe Merge "Adding vp9_get_qindex function." into experimental 2013-05-01 12:04:21 -07:00
John Koleszar
d139655b14 Merge "Make vp9_optimize_sb* common" into experimental 2013-04-30 21:43:26 -07:00
John Koleszar
1f80a568d2 Make vp9_optimize_sb* common
Unify the various vp9_optimize_sb functions into one that handles all
transform sizes.

Change-Id: I48b642fbfb3e72cc2e0bcf1d0317a80a80547882
2013-04-30 21:34:58 -07:00
Jingning Han
09aaf61e16 Migrating MB decoding into decode_sb/atom
The decode_mb only carries I8X8_PRED decoding, which will be covered
by the regular MB intra modes when SB8X8 is on. To be removed later.

Change-Id: I3b9ee55917a30b42518b81987bc10c22b1a19e7f
2013-04-30 20:48:52 -07:00
Johann
e43662e8e6 Remove unused quantize optimizations.
Files were copied from vp8 and never maintained.

Change-Id: I9659a8755985da73e8c19c3c984423b6666d8871
2013-04-30 18:42:05 -07:00
Dmitry Kovalev
79590f186c Merge "Cleaning up encoder segmentation code." into experimental 2013-04-30 17:49:55 -07:00
Ronald S. Bultje
ff2d69573e Use more restrictive block radius for 8x8 block MV references.
Change-Id: If02e006aa8a89da9de23da92362bd2e7718ea07c
2013-04-30 17:34:02 -07:00
Dmitry Kovalev
aea29cd278 General code cleanup inside treewriter-related files.
Change-Id: Ifaa40612a9c054d96112ba350c6f4adb46b1bd5b
2013-04-30 16:39:07 -07:00
Ronald S. Bultje
d068d869b9 sb8x8 integration in rd loop.
Work-in-progress, not yet ready for review. TODO items:
- bitstream writing (encoder) and reading (decoder)
- decoder reconstruction

Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-04-30 16:13:20 -07:00
Dmitry Kovalev
b5364d4f3b Using treed_read/treed_write functions for segment ids.
Changing the order of probabilities inside mb_segment_tree_probs in order
to use treed_read/treed_write function instead of custom code.

Change-Id: I843487d5057913b9358db73da270893eefecc6c8
2013-04-30 14:06:49 -07:00
Dmitry Kovalev
3f6c6ffc86 Adding vp9_get_qindex function.
Moving common code from encoder and decoder to vp9_get_qindex function.
Also moving quant-related constants from vp9_onyxc_int.h to
vp9_quant_common.h.

Change-Id: I70c5bfbaa1c8bf00fde0bfc459d077f88b6d46c8
2013-04-30 13:39:50 -07:00
Yaowu Xu
ad6890316c Merge "Removed code no longer being used." into experimental 2013-04-30 12:26:09 -07:00
Jingning Han
dc8cd7d9f4 Merge "Migrating SPLITMV_4X4 decoding into decode_atom" into experimental 2013-04-30 12:17:56 -07:00
Yaowu Xu
df6f82c3e8 Removed code no longer being used.
Change-Id: Iab9a88f250614a790b6ad96bf3150a74210910df
2013-04-30 12:09:27 -07:00
Dmitry Kovalev
347ad7fff0 Merge "Adding vp9_update_frame_size function." into experimental 2013-04-30 12:00:35 -07:00
Dmitry Kovalev
15b5e465f2 Adding vp9_update_frame_size function.
Moving common code from encoder and decoder to vp9_update_frame_size.

Change-Id: I6ca758b7d05ffd52821bd3f7ad68089da11e4165
2013-04-30 11:14:27 -07:00
Dmitry Kovalev
be9cf215a3 Merge "General cleanup inside vp9_bitstream.c." into experimental 2013-04-30 11:07:05 -07:00
Jingning Han
ce3a8a6e37 Migrating SPLITMV_4X4 decoding into decode_atom
Separate the decoding process of 4x4 block based coding (both intra
and inter) from decode_mb and move it into decode_atom_. This allows
to further move the rest per 16x16 block decoding of decode_mb into
decode_sb, and hence eventually deprecating decode_mb when SB8X8 is
enabled.

Change-Id: I678cb8007d8a57b792d7a23020edb0c74fbf4237
2013-04-30 10:42:03 -07:00
Dmitry Kovalev
70d12c3a75 Merge "Renaming refresh_entropy_probs to refresh_frame_context." into experimental 2013-04-30 10:21:24 -07:00
Dmitry Kovalev
51a73fbba2 Merge "Consistent names for quant-related functions and variables." into experimental 2013-04-30 10:19:48 -07:00
John Koleszar
e8315aeea0 Merge "Fix incorrect 32x64, 16x32 subpel variance" into experimental 2013-04-30 08:43:08 -07:00
Jingning Han
7492edac93 Merge "Separate I4X4_PRED coding from macroblock modules" into experimental 2013-04-29 21:51:59 -07:00
Jingning Han
94191b5c82 Separate I4X4_PRED coding from macroblock modules
Separate the functionality of I4X4_PRED from decode_mb. Use
decode_atom_intra instead, to enable recursive partition of superblock
down to 8x8.

Change-Id: Ifc89a3be82225398954169d0a839abdbbfd8ca3b
2013-04-29 18:59:36 -07:00
Ronald S. Bultje
ee0f3a9d31 Make sure 32x32 coefficients fit in int16_t.
Fixes mismatches on yt.

Change-Id: I65444137578b29a4984af5c01b2874c5ab086acc
2013-04-29 17:59:00 -07:00
Yaowu Xu
0b48548eeb Merge "fixed new intra code for rectanglar blocks" into experimental 2013-04-29 16:16:27 -07:00
Dmitry Kovalev
ee97da2c03 Cleaning up encoder segmentation code.
Moving code from vp9_pack_bitstream to new function encode_segmentation.

Change-Id: I1f1e59a1f038618ad95162b7db4b6f8164850ea8
2013-04-29 16:07:17 -07:00
Yaowu Xu
caea860a0f Merge "Enabled i4x4 to use right above pixels" into experimental 2013-04-29 16:05:19 -07:00
Yaowu Xu
7ea12f2c5f Merge "Use same intra prediction for all block size" into experimental 2013-04-29 16:02:35 -07:00
Yaowu Xu
4747c6ed90 fixed new intra code for rectanglar blocks
Also fixed two minor subtle boundary conditions in intra prediction
code, and replaced memcpy/memset with vpx_ prefixed version.

Change-Id: I9cddff3be831228b628f1f2f065a61feacbcbee6
2013-04-29 16:02:00 -07:00
Yaowu Xu
e388251d5d Enabled i4x4 to use right above pixels
Change-Id: I7442b4600b6812bed13e655ccf68f9ea56cc83a2
2013-04-29 15:16:59 -07:00
Yaowu Xu
3d655805f2 Use same intra prediction for all block size
The commmit changed to use same intra prediction function for all
block sizes.

Some details on the changes:
1. All directional modes except DC/TM/V/H now have built-in filtering
for all pixels with filter taps either (1, 2, 1)/4 or (1, 1)/2.
2. Above edge get automatic extended to double width (bw*2), which
makes a lot of the prediciton mode computation simpler.
3. Same intra prediction function is called with different size
for i4x4_pred and all other larger size.

Overall, the change helped keyframe only coding for both cif size
and std-hd size test sets by .5% consistently on all encodings.
For normal coding with single/auto key frame, the change now also
is consistently net positive for all encodings. The overall gains
is about .15% on std-hd set.

Change-Id: I01ceb31fbc73d49776262e6bdc06853b03bbd1d1
2013-04-29 15:15:30 -07:00
John Koleszar
907541ee18 Fix incorrect 32x64, 16x32 subpel variance
Incorrect strides caused use of uninitialized stack data.

Change-Id: Ifc12174285311bcf0453987d4283e91141f979ef
2013-04-29 12:59:30 -07:00
John Koleszar
bbac4094e4 Merge "Use foreach_transformed_block with tokenize_b" into experimental 2013-04-29 12:53:04 -07:00
Ronald S. Bultje
767dd469b3 Merge "Merge encode_sb and encode_sb64 in a recursive encode_sb/b function." into experimental 2013-04-29 12:41:55 -07:00