Commit Graph

1154 Commits

Author SHA1 Message Date
Yaowu Xu
f46326c7a2 Simplify rd_pick_intra_sby_mode()
Change-Id: Ifb0915c94c2db48827ddbd446314cb6e3155b99c
2014-09-22 14:58:51 -07:00
hkuang
c70cea97ac Remove mi_grid_* structures.
mi_grid_* are arrays of pointer to pointer. They save the pointers that point
to the MIs in cm->mi. But they are unnecessary and complicated. The original
goal was to remove MODE_INFO_t copy. But with an extra MODE_INFO_t pointer
inside MODE_INFO_t, same goal could be achieved.

This commit totally removes the mi_grid_* structures. But there are still
many dummy MODE_INFO_t inside cm->mi which are a waste of memory. Next commit
will do on-demand MODE_INFO_t allocation in order to save these memories.

Change-Id: I3a05cf1610679fed26e0b2eadd315a9ae91afdd6
2014-09-19 21:27:11 -07:00
Deb Mukherjee
5cd0aab81a Adds high bitdepth quantization functions
Adds various high bitdepth quantization functions.

Change-Id: I36fc0bf75a1bd15128ed271df8723de0ac134b0c
2014-09-16 14:55:37 -07:00
Jingning Han
ffaebfc7b4 Merge "Add ARF validation for compound inter mode check" 2014-09-15 21:26:37 -07:00
Jingning Han
c50256c157 Merge "Remove redundant reference frame check in sub8x8 RD search" 2014-09-15 21:26:11 -07:00
Jingning Han
fe96932c69 Merge "Replace best_ref_index table fetch with best_mbmode" 2014-09-15 21:25:48 -07:00
Yunqing Wang
57eb2a4e83 Merge "Simplify the skip flag cost code" 2014-09-15 18:50:30 -07:00
Yunqing Wang
c60ef810a1 Merge "Set the skip flag to 1 for skippable blocks" 2014-09-15 18:50:19 -07:00
Yunqing Wang
200ec69abb Simplify the skip flag cost code
Code refactoring.

Change-Id: Idad53cb80497d13551a142a642f7529fc305b0bc
2014-09-15 17:11:16 -07:00
Yunqing Wang
46aed7b8d0 Set the skip flag to 1 for skippable blocks
If the partition block is skippable, which means no coefficients
for Y, U, and V planes, its skip flag is set to 1. No quality
change (verified by borg tests), and no noticeable speed change.

Change-Id: I9231f720f8dd6364384cf05aa148ca24d75450f1
2014-09-15 16:50:19 -07:00
Jingning Han
f897dd5f09 Merge "Fix format in vp9_rd_pick_inter_mode_sub8x8" 2014-09-15 15:34:22 -07:00
Jingning Han
f1581b3b2e Add ARF validation for compound inter mode check
This commit enforces ARF validation check for compound inter modes.
It avoids potential access to ARF in the encoding process if it
is not allowed.

Change-Id: I055fec946b5d19d97937dc9001e1e564923e2439
2014-09-15 12:20:57 -07:00
Jingning Han
252822e81c Remove redundant reference frame check in sub8x8 RD search
The valid reference frame check in sub8x8 rate-distortion
optimization search has been included in the ref_frame_skip_mask
scheme. This commit removes the later further validation checks
that are not in effect.

Change-Id: I853b477c44037d3dc0afec6cbfce08a96c597a75
2014-09-15 12:20:04 -07:00
Jingning Han
cc00eea676 Replace best_ref_index table fetch with best_mbmode
This commit replaces the best_ref_index table fetch with the use
of best_mbmode in vp9_rd_pick_inter_mode_sub8x8.

Change-Id: I882ee9ee6a8c0e61befcca1f4dba6d2ea8de8f13
2014-09-15 09:59:20 -07:00
Jingning Han
73805bfa70 Fix format in vp9_rd_pick_inter_mode_sub8x8
Change-Id: I9b6a74bdf003b39235f14f8b5b7f3b861f6bf131
2014-09-15 09:44:09 -07:00
Jingning Han
59dd83a3ea Merge "Refactor reference frame control in sub8x8 block RD search" 2014-09-13 10:43:36 -07:00
Jingning Han
e6d927343e Merge "Format fixes in vp9_rd_pick_inter_mode_sb" 2014-09-13 10:43:24 -07:00
Jingning Han
ad3c92b9b7 Merge "Remove unused best_inter_rd variable" 2014-09-13 10:43:14 -07:00
Jingning Han
f02e0b6cf6 Merge "Remove unused speed feature" 2014-09-13 10:43:03 -07:00
Jingning Han
adb20849b6 Refactor reference frame control in sub8x8 block RD search
This commit unifies the reference frame control in the rate-
distortion optimization search loop of sub8x8 block size to remove
the control dependency on mode search order.

Change-Id: I3a174099f71a7cc176ede9fd60e2374243ae9232
2014-09-12 11:03:03 -07:00
Jingning Han
7f77a1c3c9 Merge "Unify intra mode mask into mode_skip_mask scheme" 2014-09-12 09:06:35 -07:00
Deb Mukherjee
10783d4f3a Adds high bitdepth transform functions and tests
Adds various high bitdepth transform functions and tests.
Much of the changes are related to using typedefs tran_low_t
and tran_high_t for the final transform cofficients and intermediate
stages of the transform computation respectively rather than fixed
types int16_t/int. When vp9_highbitdepth configure flag is off,
these map tp int16_t/int32_t, but when the flag is on, they map
to int32_t/int64_t to make space for needed extra precision.

Change-Id: I3c56de79e15b904d6f655b62ffae170729befdd8
2014-09-11 19:56:33 -07:00
Jingning Han
74ddde01c0 Format fixes in vp9_rd_pick_inter_mode_sb
Change-Id: Ie45687405dcaa34ba465dce2aa14f76017d3a794
2014-09-11 17:15:15 -07:00
Jingning Han
8e3f7a52a1 Remove unused best_inter_rd variable
The variable best_inter_rd is effectively not in use in the rate-
distortion mode search loops of both regular block sizes and sub8x8
block sizes.

Change-Id: I178f909f8c9629772e13adc6257908653b2adf31
2014-09-11 16:16:26 -07:00
Jingning Han
00fe92c22f Remove unused speed feature
The speed feature that skips compound inter prediction modes was
subsumed by other speed features and effectively was not in use.
This commit removes it.

Change-Id: I22b0c71a8ddd15d93b25d86fa63a1dce2ba6a1a9
2014-09-11 15:54:53 -07:00
Jingning Han
bdd8eb6fcc Unify intra mode mask into mode_skip_mask scheme
Integrate intra mode mask speed feature with the mode_skip_mask
scheme. Move it outside the mode search loop in the
vp9_rd_pick_inter_mode_sb function.

Change-Id: I7738fea749bfdc08ad05d7f2524feb8ff67568d9
2014-09-11 10:36:48 -07:00
Jingning Han
8cefed1568 Remove inter_mode_mask from rate-distortion search loop
This speed feature is used in real-time setting only. Remove the
related condition check in the rate-distortion optimization search
loop.

Change-Id: Iaacc1e268214634e6f95c5048c28a60cec6c42fc
2014-09-11 10:18:55 -07:00
Jingning Han
238b2ace86 Move intra block size skip outside mode search loop
Unify this speed feature in the ref_frame_skip_mask scheme.

Change-Id: I7ea5646da02d3ea643680c22d50dabd448d55a27
2014-09-11 09:54:19 -07:00
Jingning Han
8b06a24ce7 Fix format in vp9_rd_pick_inter_mode_sub8x8
Change-Id: I0da29c858c6c1eb5ef07cee8f599329f5a002da9
2014-09-11 09:28:47 -07:00
Jingning Han
8d42fad9c1 Move overlay frame speed feature setting out of mode search loop
Refactor overlay frame speed-up related function. Make it unified
with the ref_frame_skip_mask system and Move it out of the mode
search loop.

Change-Id: I0dde9baf44354f6ba00b4679cba02fa6a30c7316
2014-09-10 19:44:58 -07:00
Jingning Han
f9f0879756 Refactor to remove speed feature dependency on mode search order
This commit refactor the rate-distortion optimization search for
regular block sizes to remove the speed feature dependency on mode
search order.

Change-Id: Ied033ee484c2957e17baa7b6450b720fe7dd0e7d
2014-09-10 17:09:14 -07:00
Jingning Han
68d79146ea Fix a bug in vp9_rd_pick_inter_mode_sb
This commit fixes a bug related to skipping intra mode checking, by
using a separate variable to store the best prediction error from
inter mode. It avoids unintentionally overwriting intra mode
rate-distortion cost, and hence affecting other speed features.

Change-Id: I99e12993339c84c8b4f597996b372012e5858fae
2014-09-09 15:39:54 -07:00
Jingning Han
9a9e2aef09 Remove redundant ref frame pointer assignment
Assigning selected reference frame pointer is done in the
encode_superblock function. No need to do this at the end of
rate-distortion optimization search.

Change-Id: I33fcede0fd304b4a4c4deef2d126d79546a9c070
2014-09-09 15:15:11 -07:00
Jingning Han
33593d1f03 Remove dependency of intra mode search skip check on mode order
This commit refactors the vp9_rd_pick_inter_mode_sb function to
remove the intra mode early termination dependency on the mode
search order.

Change-Id: If6ac49aa7c530c7b9a5bd31b0ab84db83e192bec
2014-09-09 12:30:47 -07:00
Jingning Han
d96228a07c Replace best_mode_index table retrieve with fetching best_mbmode
This commit allows the encoder to find current best prediction mode
state using best_mbmode, instead of fetching from the static mode
search table via best_mode_index.

Change-Id: Ibefeab83aed33a49c2be03e83f09153856ca4271
2014-09-09 11:58:10 -07:00
Jingning Han
a61973bf29 Merge "Enable adaptive motion search for ARF coding" 2014-09-08 08:51:05 -07:00
Yunqing Wang
1dd9a63929 Correct the mode decisions in special cases
The rate costs calculated for inter modes are not precise in some
cases, which causes NEWMV is chosen instead of NEARESTMV, NEARMV,
and ZEROMV. This patch added checks for these cases, and corrected
the mode decisions.

Borg tests at speed 3 showed:
1. stdhd set: 0.102% PSNR gain and 0.088% SSIM gain.
2. derf set:  0.147% PSNR gain and 0.132% SSIM gain.
No speed change.

Change-Id: I35d17684b89ad4734fb610942d707899146426db
2014-09-05 12:01:07 -07:00
Jingning Han
d435148fe6 Enable adaptive motion search for ARF coding
This commit turns on adaptive motion search for ARF coding, in
addition to other normal inter frame coding. It improves the
average compression efficiency:

stdhd 0.1%
derf  0.04%

For the test sequences, the speed 3 runtime is reduced:

pedestrian 1080p 2000 kbps, 149932 ms -> 144580 ms, (3.3% speed-up)
bus CIF 1000 kbps, 8050 ms -> 7895 ms, (1.9%)
highway CIF 100 bkps, 45033 ms -> 44078 ms, (2.2%)

Change-Id: I5228565b609f99e8ae04f6140a2bf2b64a831d21
2014-09-04 16:26:40 -07:00
Jingning Han
3de038f396 Merge "Speed up compound inter prediction mode check" 2014-09-04 16:09:07 -07:00
Jingning Han
d62d804e64 Speed up compound inter prediction mode check
This commit allows the encoder to store outcomes of single reference
frame modes and compares them to decide if the inter prediction
filter, forward transform, and quantization can be skipped.

The compression performance of speed 3 is down
derf  -0.364%
stdhd -0.198%

For test sequences, the speed 3 runtime is reduced
highway CIF 100 kbps, 51976 ms -> 45033 ms, 13% speed-up
stockholm 720p 1000 kbps, 71826 ms -> 67838 ms, 5.5% speed-up
pedestrian 1080p 2000 kbps, 154924 ms -> 150702 ms, 2.6% speed-up

Change-Id: I5aa26f918d2b4b5197a2c0afa2779319f1c88e44
2014-09-03 15:28:01 -07:00
Yaowu Xu
e759d95743 Merge two similar functions into one
intra_super_block_yrd() and inter_super_block_yrd() are largely same,
this commit merges them into one to reduce code duplication.

Change-Id: I64d7042a5b099345627cf55663010c185b25ec37
2014-09-03 11:21:06 -07:00
Jingning Han
801fef26ec Skip comp inter mode test in RD loop with same frame bias signs
This commit allows the encoder to skip check on compound inter
modes in the rate-distortion optimization loop, if the reference
frame bias signs are the same.

Change-Id: Ib753e6bb11cbdd338aee69dbe2b649671f75a6b0
2014-09-02 18:17:33 -07:00
Jingning Han
33176fef87 Skip comp inter mode tests for arf coding
This commit skips the compound inter mode prediction check in the
rate-distortion optimization loop for ARF coding. It reduces the
runtime for certain test clips at speed 3, at no compression
performance change:

bus CIF 1000 kbps, 8260 ms -> 8090 ms, 1.8% speed-up
stockholm 720p 1000 kbps, 74453 ms -> 71826 ms, 2.9% speed-up

No visible speed-up for pedestrian area 1080p at 2000 kbps.

Change-Id: Ic68aa56837159b726563b784e2e3729e846465ad
2014-09-02 11:23:47 -07:00
Jingning Han
bac0268716 Merge "Skip intra mode tests depending on inter residuals" 2014-09-02 08:32:52 -07:00
Jingning Han
6ddf1e152a Fix int64_t to unsigned int conversion warnings
Use unsigned int type to store the sse in the pixel domain. The
precision is sufficient to handle sse of block size up to 64x64.
The transform domain version however needs int64_t, since there is
a transfer gain applied in the forward transformation that might
cause unsigned int overflow.

Change-Id: Ifef97c38597e426262290f35341fbb093cf0a079
2014-08-29 14:29:31 -07:00
Jingning Han
4282955ee1 Skip intra mode tests depending on inter residuals
This commit allows encoder to skip intra coding mode test, when
the known inter residual is less than the source variance. It
reduces the runtime of speed 3 for test clips:
bus cif 1000 kbps: 8587 ms -> 8260 ms, 3.8% speed-up
pedestrian 1080p 2000 kbps: 161381 ms -> 155241 ms, 3.7% speed-up.

The compression performance is down by
derf   -0.36%
stdhd  -0.25%

Change-Id: I75ce1e035b4da2153cb1ac14111d1a07c05a735d
2014-08-29 08:37:35 -07:00
Jingning Han
02e6ecdc4c Extend block level sse to support multiple txfm blocks
This commit extends the sse and forward transform computation flag
to support the case 64x64 blocks where there are 4 32x32 2D-DCT
blocks.

Change-Id: I86a3e805dfaa0f3abd812f590520c71aa0e40473
2014-08-29 08:29:34 -07:00
Yunqing Wang
4d2c376923 Early termination in encoding partition search
In the partition search, the encoder checks all possible
partitionings in the superblock's partition search tree.
This patch proposed a set of criteria for partition search
early termination, which effectively decided whether or
not to terminate the search in current branch based on the
"skippable" result of the quantized transform coefficients.
The "skippable" information was gathered during the
partition mode search, and no overhead calculations were
introduced.

This patch gives significant encoding speed gains without
sacrificing the quality.

Borg test results:
1. At speed 1,
   stdhd set: psnr: +0.074%, ssim: +0.093%;
   derf set:  psnr: -0.024%, ssim: +0.011%;
2. At speed 2,
   stdhd set: psnr: +0.033%, ssim: +0.100%;
   derf set:  psnr: -0.062%, ssim: +0.003%;
3. At speed 3,
   stdhd set: psnr: +0.060%, ssim: +0.190%;
   derf set:  psnr: -0.064%, ssim: -0.002%;
4. At speed 4,
   stdhd set: psnr: +0.070%, ssim: +0.143%;
   derf set:  psnr: -0.104%, ssim: +0.039%;

The speedup ranges from several percent to 60+%.
                 speed1    speed2    speed3    speed4
(1080p, 100f):
old_town_cross:  48.2%     23.9%     20.8%     16.5%
park_joy:        11.4%     17.8%     29.4%     18.2%
pedestrian_area: 10.7%      4.0%      4.2%      2.4%
(720p, 200f):
mobcal:          68.1%     36.3%     34.4%     17.7%
parkrun:         15.8%     24.2%     37.1%     16.8%
shields:         45.1%     32.8%     30.1%      9.6%
(cif, 300f)
bus:              3.7%     10.4%     14.0%      7.9%
deadline:        13.6%     14.8%     12.6%     10.9%
mobile:           5.3%     11.5%     14.7%     10.7%

Change-Id: I246c38fb952ad762ce5e365711235b605f470a66
2014-08-28 11:27:28 -07:00
Deb Mukherjee
bb2a9abb1e Merge "Updates vp9_pattern search to return integer sads" 2014-08-28 09:38:56 -07:00
Deb Mukherjee
04b100b23e Updates vp9_pattern search to return integer sads
Updates the vp9_pattern_search function to return integer one-away
neighbors' sad values, for subsequent use in speeding up the
sub-pel search. Also, removes code for the do_refine option
which is not being used currently.
Updates the integer and subpel functions to pass in a 5-element
sad list for output or input.

A new pruned sub-pel search algorithm is implemented that uses
the sad returned from the integer pel search. But it is not
deployed yet.

Change-Id: Ifa9f5ad024b5b660570366d2bd900343e1891520
2014-08-28 06:49:58 -07:00