Commit Graph

1200 Commits

Author SHA1 Message Date
Jingning Han
94ecfa323f Reset rate cost value in rd mode search
When early termination is triggered, properly reset the rate cost
to invalid value to avoid potential ioc issue.

Change-Id: I3444390be2e49a34bb02cf8a74c33d5dbd96d88d
2014-10-17 09:33:59 -07:00
Jingning Han
ed100c0b00 Fix an ioc issue in super_block_uvrd
This commit fixes an ioc issue that will happen when the cumulative
variables are not in effective use. The fix discards these
redundant additions.

Change-Id: Idbac5bfb989c0cedc5f8a323effce938519b2457
2014-10-16 11:07:39 -07:00
Jingning Han
f3a5de816d Refactor super_block_uvrd function to remove goto statement
Use return value 0/1 as indicator of the validity of the rate-
distortion cost.

Change-Id: I6244126fbf03472cebcba4f177a6cd329fae4743
2014-10-14 09:58:11 -07:00
Jingning Han
69a09a70e9 Use speed feature variable in vp9_rd_pick_inter/intra_mode
Replace repeated fetch cpi->sf with a local sf pointer.

Change-Id: I5a55bba3e1c41fbdbc6ad5f078d2fa49dd95ee67
2014-10-13 16:15:00 -07:00
Jingning Han
3bdb6bfcee Fix vp9_rd_pick_inter/intra function types
The returned value is not used anywhere, hence changing the function
type into void.

Change-Id: I0ece49ed61e7aab6df01140135503ad41d4ef4a4
2014-10-13 16:00:46 -07:00
Jingning Han
811cef97c9 Refactor rate distortion cost structure
This commit makes a struct that contains rate value, distortion
value, and the rate-distortion cost. The goal is to provide a
better interface for rate-distortion related operation. It is
first used in rd_pick_partition and saves a few RDCOST calculations.

Change-Id: I1a6ab7b35282d3c80195af59b6810e577544691f
2014-10-13 14:27:16 -07:00
Jingning Han
a62acf3c0a Fix ActiveMapTest valgrind warning
This fixes a valgrind warning in the ActiveMapTest unit test
reported in issue 870.

Change-Id: Idf172ab0244ebefe630c3577e649bc9ba7c43d10
2014-10-11 22:36:58 -07:00
Deb Mukherjee
9a29fdbae7 Merge "Rename highbitdepth functions to use highbd prefix" 2014-10-09 15:39:56 -07:00
Deb Mukherjee
1929c9b391 Rename highbitdepth functions to use highbd prefix
Uses highbd_ prefix convention consistently.

Change-Id: I58f7f799a7ff8e32701bcd71c955bcf1cdd4581e
2014-10-09 14:40:40 -07:00
Deb Mukherjee
3117830af3 Merge "Subpel search cleanups and enhancements" 2014-10-09 11:14:51 -07:00
Alex Converse
9ffbc31367 Merge "Move the high freq coeff check outside store_coding_context" 2014-10-09 11:12:02 -07:00
Deb Mukherjee
d78dbff09a Subpel search cleanups and enhancements
- Some fixes to surface fit.
- Returns variance function as cost rather than sad in the
  pattern search and diamond search functions. Only
  vp9_pattern_search_sad function used in bigdia search
  uses sad as integer 1-away costs.
- Deploys SUBPEL_TREE_PRUNED_MORE for speed 4+.

Results:
derf [Speed 3]: About +0.036% in coding efficiency without any
discernible speed loss.
derf [Speed 4]: About 2-3% faster at -0.199% loss in coding efficiency.
derf [Speed 5]: About 3-4% faster at -0.149% loss in coding efficiency.

Change-Id: I8462f94f6adb46966ca964f2bd0400977357fd63
2014-10-08 23:59:43 -07:00
Yunqing Wang
189566db58 Merge "Allow mode search breakout at very low prediction errors" 2014-10-08 19:58:18 -07:00
Yunqing Wang
e18edd5eb6 Allow mode search breakout at very low prediction errors
In model_rd_for_sb function, the spatial domain SSE and variance
are checked to see if transform coefficients are quantized to 0.
Besides that, this patch adds another set of thresholds that are
much more strict. These thresholds are used to conduct a partition
block level check to measure if all its TX blocks are skippable
for YUV planes. If it is true, x->skip is set for this partition
block, and thus its mode search is terminated.

This speeds up the encoding at very low prediction error case,
such as screen sharing application. This patch covers what
rd_encode_breakout_test() does, so that function is removed.

Borg test at speed 3 shows:
For stdhd set, psnr: +0.008%, ssim: +0.014%;
For derf set, psnr: +0.018%, ssim: +0.025%.
No noticeable speed change.

Change-Id: I4e5f15cf10016a282a68e35175ff854b28195944
2014-10-08 17:46:22 -07:00
Jingning Han
5fcbcf1b22 Move the high freq coeff check outside store_coding_context
This fixes valgrind message issue 870.

Change-Id: Ibbc2481923a2995029ab05de30c9e8a6e9f0f9a8
2014-10-08 16:10:32 -07:00
Jingning Han
41cea46154 Use local variable in vp9_rd_pick_inter_mode_sb
Change-Id: Ie35a965a6b8de536ccaf61ff61498620d22db205
2014-10-08 16:09:47 -07:00
Jingning Han
3bbec7b422 Merge "Replace mi_width_log2() with mi_width_log2_lookup table" 2014-10-07 15:33:52 -07:00
Jingning Han
27c9577f8e Merge "Take out repeated block width/height lookup functions" 2014-10-07 15:33:45 -07:00
Yunqing Wang
0cac69f594 Merge "Fix skip_txfm issue in rdopt code" 2014-10-07 15:13:44 -07:00
Yunqing Wang
a4aa14020a Fix skip_txfm issue in rdopt code
Fixed an encoder crash. Set skip_txfm to 0 for cases that skip_txfm
isn't calculated. Put memcpy of skip_txfm at right place.

Change-Id: Ib3b6afc1b251a85b2a853c8138fb3393f48cfef6
2014-10-07 12:47:43 -07:00
Jingning Han
7ee58985bd Replace mi_width_log2() with mi_width_log2_lookup table
Change-Id: If0ea98aa139d14d40cd924114e18396aff36b5a5
2014-10-07 12:45:25 -07:00
Jingning Han
b66f7016c1 Take out repeated block width/height lookup functions
The functions b_width_log2 and b_height_log2 only do direct
table fetch. This commit unifies such use cases by using the
table directly and removes these functions.

Change-Id: I3103fc6ba959c1182886a2799d21b8b77c8a7b6b
2014-10-07 12:33:07 -07:00
Deb Mukherjee
cfc337aae8 Merge "Resolves some static analysis / undefined warnings" 2014-10-07 12:15:26 -07:00
Deb Mukherjee
fced63ed30 Resolves some static analysis / undefined warnings
Also fixes a case of distortion becoming negative and messing
up the RDCOST computation.

Change-Id: Id345af9e8dfff31ade622be5756e51f2cdface53
2014-10-07 11:20:56 -07:00
Jingning Han
a75551585b Fix eobs buffer pointer mis-use
This commit fixes a buffer pointer mis-use in store_coding_context.
The compression performance for stdhd set of speed 3 is improved by
0.097%. It fixes issue 869.

Change-Id: Idc59e22035eaf39f7133ca04174894374d647ff7
2014-10-06 15:57:13 -07:00
Jingning Han
1b8c57e915 Merge "Fix an IOC issue in vp9_rd_pick_inter_mode_sb" 2014-10-06 09:29:29 -07:00
Jingning Han
085b97aa5c Fix an IOC issue in vp9_rd_pick_inter_mode_sb
It is possible that the GOLDEN reference frame is not avaiable, in
which setting the predicted mv will be associated with a residual
value of INT_MAX. This commit checks this condition before
left shift and comparison with that of ALTREF frame, to avoid
overflow issue.

Change-Id: Ib98c3149dbdd016f2fe5beaafb13f67d469dd07c
2014-10-05 12:05:14 -07:00
Jingning Han
a1088e0b5f Merge "Rework partition search skip scheme" 2014-10-03 15:23:54 -07:00
Jingning Han
bb260d9076 Rework partition search skip scheme
This commit enables the encoder to skip split partition search if
the bigger block size has all non-zero quantized coefficients in low
frequency area and the total rate cost is below a certain threshold.
It logarithmatically scales the rate threshold according to the
current block size. For speed 3, the compression performance loss:
derf  -0.093%
stdhd -0.066%

Local experiments show 4% - 20% encoding speed-up for speed 3.
blue_sky_1080p, 1500 kbps
51051 b/f, 35.891 dB, 67236 ms ->
50554 b/f, 35.857 dB, 59270 ms (12% speed-up)

old_town_cross_720p, 1500 kbps
14431 b/f, 36.249 dB, 57687 ms ->
14108 b/f, 36.172 dB, 46586 ms (19% speed-up)

pedestrian_area_1080p, 1500 kbps
50812 b/f, 40.124 dB, 100439 ms ->
50755 b/f, 40.118 dB,  96549 ms (4% speed-up)

mobile_calendar_720p, 1000 kbps
10352 b/f, 35.055 dB, 51837 ms ->
10172 b/f, 35.003 dB, 44076 ms (15% speed-up)

Change-Id: I412e34db49060775b3b89ba1738522317c3239c8
2014-10-03 11:54:30 -07:00
Deb Mukherjee
431cdc33ee Prevent negative cost for highbitdepth
Adds proper scaling for highbitdepth in a rdopt cost.

Change-Id: I066694799a7f491b830945ef1c66eb202071c355
2014-10-03 10:22:21 -07:00
Deb Mukherjee
30fbf23fda Merge "High-bitdepth bugfixes" 2014-10-01 16:47:43 -07:00
Yunqing Wang
e350e3fe68 Merge "Modify block transform skipping check" 2014-10-01 16:19:56 -07:00
Deb Mukherjee
a160d72522 High-bitdepth bugfixes
Miscellaneous bug-fixes for high bitdepth functionality.
With this patch, high bit-depth profiles become mostly functional,
except for an intermittent assert failure issue that is being
tracked.

Change-Id: I6a7fcbdcf1e5b09842e88535f8442d2e1230748c
2014-10-01 14:18:11 -07:00
Yunqing Wang
e4aac6bb61 Modify block transform skipping check
Block transform skipping was implemented based on DCT's energy
conservation property. Modified the thresholds using zero bin
parameters. AC and DC coefficients were checked separately to
allow better identifying of skippable blocks.

Borg test at speed 3 showed:
stdhd set: psnr gain: 0.153%, ssim gain: 0.051%;
derf set: psnr gain: 0.023%, ssim gain: 0.036%

For most test clips, the encoding speedup is 1% - 2%.
parkrun(720p): 7.5% speedup, park_joy(1080p): 3.5% speedup.

Change-Id: If28eb81113a077414f5ca7b021c14f9069b373bb
2014-10-01 12:58:09 -07:00
Jingning Han
891793a540 Conditionally skip reference frame check
For regular inter frames, if the distance from GOLDEN_FRAME is larger
than 2 and if the predicted motion vector of LAST_FRAME gives lower
sse than that of GOLDEN_FRAME, skip the GOLDE_FRAME mode checking in
the rate-distortion optimization. It provides about 5% speed-up at
expense of -0.137% and -0.230% performance down for speed 3. Local
experiment results:

pedestrian 1080p 2000 kbps
66712 b/f, 40.908 dB, 113688 ms ->
66768 b/f, 40.911 dB, 108752 ms

blue_sky 1080p 2000 kbps
51054 b/f, 35.894 dB, 70406 ms ->
51051 b/f, 35.891 dB, 67236 ms

old_town_cross 720p 1500 kbps
14412 b/f, 36.252 dB, 60690 ms ->
14431 b/f, 36.249 dB, 57346 ms

Change-Id: Idfcafe7f63da7a4896602fc60bd7093f0f0d82ca
2014-10-01 08:32:15 -07:00
Jingning Han
8b4dd536a5 Merge "Skip certain ALTREF inter modes in ARF coding" 2014-09-29 10:43:45 -07:00
Jingning Han
ccdb518ff8 Skip certain ALTREF inter modes in ARF coding
This commit enables the encoder to skip checking ALTREF inter modes
in ARF coding, if the predicted motion vectors suggest that the
GOLDEN_FRAME provides higher prediction accuracy than ALTREF_FRAME.

It improves the speed 3 encoding speed by about 5%, at the expense
of compression performance loss -0.041% and -0.225% for derf and
stdhd, respectively.

pedestrian_area 1080p 2000 kbps
66705 b/f, 40.909 dB, 118738 ms ->
66732 b/f, 40.908 dB, 113688 ms

old_town_cross 720p 1500 kbps
14427 b/f, 36.256 dB, 62746 ms ->
14412 b/f, 36.252 dB, 60690 ms

blue_sky 1080p 1500 kbps
51026 b/f, 35.897 dB, 73310 ms ->
50921 b/f, 35.893 dB, 70406 ms

bus CIF 1000 kbps
21301 b/f, 34.841 dB, 7326 ms ->
21248 b/f, 34.837 dB, 7196 ms

Change-Id: I76cf88b4d655e1ee3c0cb03c8a5745493040e8d2
2014-09-26 12:53:43 -07:00
Deb Mukherjee
993d10a217 Adds various high bit-depth encode functions
Change-Id: I6f67b171022bbc8199c6d674190b57f6bab1b62f
2014-09-25 01:50:36 -07:00
Jingning Han
6989e81d61 Remove unused variable in handle_inter_mode
Change-Id: Id757d2c940756ce1b0ead2ea24af9ac0a493de05
2014-09-24 18:27:44 -07:00
Yaowu Xu
4a101310e8 Adapt mode based rd_threshold for similar block size
The rd_thresholds are adaptively changed based on best mode tested.
It was only changed for the same block size, this commit makes the
adaptation for similar block sizes too. The commit also made minor
adjustment and code cleanups.

The impact on encoding time for _ped:
118089 ms -> 111927 ms

The impact on compression:
derf:  -0.339%
stdhd: -0.303%

Change-Id: I8817fed1102350497f2ec631849e43f753878e5d
2014-09-23 16:10:59 -07:00
Yaowu Xu
56032b471d Fix an IOC
Change-Id: I0ca6746696d81657c035b0f6523c9af370da3c95
2014-09-23 16:07:22 -07:00
Yaowu Xu
7feede9869 Merge "Remove code duplication" 2014-09-22 17:13:59 -07:00
Yaowu Xu
052bc8ea6a Merge "Simplify rd_pick_intra_sby_mode()" 2014-09-22 17:13:55 -07:00
Yaowu Xu
c7ab18fe56 Remove code duplication
Change-Id: I453b3e0d946951665d5919248445fc4f3222d2ad
2014-09-22 15:22:51 -07:00
Yaowu Xu
f46326c7a2 Simplify rd_pick_intra_sby_mode()
Change-Id: Ifb0915c94c2db48827ddbd446314cb6e3155b99c
2014-09-22 14:58:51 -07:00
Jingning Han
f7023ea014 Remove unnecessary local variable declaration
This commit removes a repetitive local variable declaration in
vp9_rd_pick_inter_mode_sb.

Change-Id: I1b0afa98ff1ecbfb46e17d3d1cee95d32c4309db
2014-09-22 09:29:28 -07:00
Jingning Han
eee904c9b9 Adaptive mode search scheduling
This commit enables an adaptive mode search order scheduling scheme
in the rate-distortion optimization. It changes the compression
performance by -0.433% and -0.420% for derf and stdhd respectively.
It provides speed improvement for speed 3:

bus CIF 1000 kbps
24590 b/f, 35.513 dB, 7864 ms ->
24696 b/f, 35.491 dB, 7408 ms (6% speed-up)

stockholm 720p 1000 kbps
8983 b/f, 35.078 dB, 65698 ms ->
8962 b/f, 35.054 dB, 60298 ms (8%)

old_town_cross 720p 1000 kbps
11804 b/f, 35.666 dB, 62492 ms ->
11778 b/f, 35.609 dB, 56040 ms (10%)

blue_sky 1080p 1500 kbps
57173 b/f, 36.179 dB, 77879 ms ->
57199 b/f, 36.131 dB, 69821 ms (10%)

pedestrian_area 1080p 2000 kbps
74241 b/f, 41.105 dB, 144031 ms ->
74271 b/f, 41.091 dB, 133614 ms (8%)

Change-Id: Iaad28cbc99399030fc5f9951eb5aa7fa633f320e
2014-09-22 09:28:16 -07:00
hkuang
c70cea97ac Remove mi_grid_* structures.
mi_grid_* are arrays of pointer to pointer. They save the pointers that point
to the MIs in cm->mi. But they are unnecessary and complicated. The original
goal was to remove MODE_INFO_t copy. But with an extra MODE_INFO_t pointer
inside MODE_INFO_t, same goal could be achieved.

This commit totally removes the mi_grid_* structures. But there are still
many dummy MODE_INFO_t inside cm->mi which are a waste of memory. Next commit
will do on-demand MODE_INFO_t allocation in order to save these memories.

Change-Id: I3a05cf1610679fed26e0b2eadd315a9ae91afdd6
2014-09-19 21:27:11 -07:00
Deb Mukherjee
5cd0aab81a Adds high bitdepth quantization functions
Adds various high bitdepth quantization functions.

Change-Id: I36fc0bf75a1bd15128ed271df8723de0ac134b0c
2014-09-16 14:55:37 -07:00
Jingning Han
ffaebfc7b4 Merge "Add ARF validation for compound inter mode check" 2014-09-15 21:26:37 -07:00