Commit Graph

172 Commits

Author SHA1 Message Date
Jingning Han
8969f7c892 Chessboard pattern prediction filter type search in non-RD coding
This commit introduces a chessboard pattern search for the prediction
filter type search. It runs extensive search in alternate blocks and
allows the rest blocks to refer coding decisions of their nearby
neighbors.

For pedestrian 1080p at 4000 kbps, the runtime of speed -5 goes down
from 43990 ms to 42200 ms. The overall compression performance for
RTC set is changed by -1.37%.

Change-Id: Icfe220c49451cda796f0ca91d935c9ed01e56c9d
2014-04-23 10:41:07 -07:00
Dmitry Kovalev
e608418899 Renaming MB_PREDICTION_MODE to PREDICTION_MODE.
Actually, it would be great to have two separate enums INTRA_MODES and
INTER_MODES in future.

Change-Id: I6c4147cf0002853da9c1e03fe9514eab876f01c8
2014-04-22 17:48:31 -07:00
Dmitry Kovalev
ef003078e8 Renaming "onyx" to "encoder".
Actual renames:
  vp9_onyx_if.c -> vp9_encoder.c
  vp9_onyx_int.h -> vp9_encoder.h

Change-Id: I80532a80b118d0060518e6c6a0d640e3f411783c
2014-04-22 14:57:05 -07:00
Dmitry Kovalev
1ff4aa33de Adding RD_OPT struct.
Change-Id: I2d1b5c1481aefe226082e1f096ca9edd340d4d31
2014-04-11 10:58:37 -07:00
Jingning Han
06ec873f2c Select prediction filter type in non-RD mode decision
This commit allows the non-RD mode decision flow to  select
prediction filter type in NEWMV mode. It provides 8.14% compression
performance gains in both settings of AQ=0 and 3. The current speed
impact is about 5% to 10% slower.

Change-Id: Id66ecebf77abd8f90fb3f6a066c0e8dfb4bf1c42
2014-04-08 16:47:42 -07:00
Jingning Han
d8f68c2454 Merge "Set up early RD cost check for NEWMV in non-RD mode decision flow" 2014-04-06 11:13:32 -07:00
Jingning Han
516d0b1180 Set up early RD cost check for NEWMV in non-RD mode decision flow
This commit estimates the motion vector rate cost right after full
pixel motion search. It combines this and the mode cost and compares
the corresponding rate-distortion cost. If it is already above the
current best one, skip the rest sub-pixel motion search and modeling
process. For pedestrian_area 1080p at 4000 kpbs, the speed -5 runtime
goes down from 39425 ms -> 38399 ms.

Change-Id: If4cd7119fd6c266798d5cf1d19d19ab425e52a26
2014-04-04 16:08:51 -07:00
Dmitry Kovalev
a9f324fa7f Removing interp_kernel from MACROBLOCKD.
Now interp_kernel is obtained when it is really required (based on
mbmi->interp_filter value).

Change-Id: I4c7a93c179d1045eba16e7526c293d02c9b8b47e
2014-04-03 15:28:42 -07:00
Alex Converse
825908218a Remove full scale duplicates of bsize.
Change-Id: I28e45870a0e3b0de3b9e40fb044789ae1209c0f7
2014-04-03 11:51:21 -07:00
Jingning Han
751d002f2f Adjust rate allocation in non-RD coding mode
This commit reduces the frequency of frames using finer quantizer
in non-RD coding flow, and slightly tune up the quantizer resolution
when used. It provides 1.7% compression gains in speed -5 at no speed
difference.

Change-Id: I430249a51260a841a0402666e5ec1566e4f7d5a6
2014-04-02 09:51:22 -07:00
Dmitry Kovalev
86f44a91f4 Renaming two members in MACROBLOCKD struct.
Renames:
  mi_8x8 -> mi
  mode_info_stride -> mi_stride

Change-Id: I66f3e5fd1e7b7f46f108af5bb711c5fd9493c1be
2014-04-01 17:46:40 -07:00
Jingning Han
4a42047b4e Fix segment_id assignment in update_state_rt for AQ mode
Properly obtain and buffer the segment_id in non-RD mode information
update process.

Change-Id: I43230001872f6a986d12c0d94e98fdec0db3be2c
2014-03-27 12:42:04 -07:00
Jingning Han
6b6d3886fc Enable recursive partition selection for non-RD coding mode
This commit enables a recursive partition type search for non-RD
mode decisions. It allows the encoder to choose variable block
sizes in a 64x64 block based on rate-distortion modeling.

It improves speed -6 coding efficiency for rtc set by 2.4%. Most
of the gains come from 32-40dB range, where many sequences gain
about 5% to 20%. Local tests suggest there is no speed change.

Change-Id: I06300016e500a21652812b7b3b081db39a783d66
2014-03-24 14:40:29 -07:00
Yunqing Wang
3967435f65 Merge "Add prediction mode skipping in non-rd situation" 2014-03-21 13:40:27 -07:00
Yaowu Xu
0e53d52088 changed to use correct pred_mv
The third pred_mv is stored in x->pred_mv[ref_frame]. This commit make
sure the correct mv is read.

Change-Id: Ibed24daf36703a63f0394c87b2381ee1d2eb7910
2014-03-21 09:16:06 -07:00
Yunqing Wang
49b2330d8b Add prediction mode skipping in non-rd situation
In non-rd pick_mode code, added mode skipping according to
thresholds. Used rd thresholds now, but we can modified them
later for real-time case.

RTC set borg test showed a 0.095% PSNR gain. For different rtc
clips, the real-time(speed 7) encoder speedup is 2% - 10%.

Change-Id: Ic72535c96b891092c662453be32d3168f7e34dcc
2014-03-21 09:15:36 -07:00
Yunqing Wang
cf07d3e332 Remove unused mode_sad
Removed mode_sad.

Change-Id: I230b42ac9b617ae2c375e297057aa0756bd355fe
2014-03-20 09:28:16 -07:00
Jingning Han
60f9ebc3f1 Enable variable block size test in non-RD mode decision
This is an initial attempt to allow variable block size partition
in non-RD coding flow. It tests 8x8, 16x16 and 32x32 block size per
64x64 block, all using non-RD mode decision and the associated rate
distortion costs from modeling, then selects the best block size to
encode the entire 64x64 block. Such operations are triggered every
other 3 frames. The blocks of intermediate frames will reuse the
collocated block's partition type.

It improves the compression performance by 13.2%. Note that the gains
are not evenly distributed. For many hard clips, the compression
performance is improved by 20% to 28%. Local speed test shows that
it will also increase runtime by 50%, as compared to speed -7. It is
now enabled in speed -6 setting.

Change-Id: Ib4fb8659d21621c9075b3c369ddaa9ecb0a4b204
2014-03-19 16:10:29 -07:00
Yunqing Wang
587cca4505 Calculate rate_mv after sub-pixel search in non-rd mode
Modified the non-rd rate_mv calculation and moved it after sub-pixel
motion search is done.

Change-Id: I6a5cbd3d67ff0a86b3693e25ecf1fd77b16d708c
2014-03-14 15:55:02 -07:00
Jingning Han
ca0625ff31 Return rate and distortion cost values from non-RD mode decision
This commit allows the non-RD mode decision process to return the
rate and distortion costs associated with the selected mode.

Change-Id: Ibe0f67d323f65839fd9cb0a726c1219bf7b55da9
2014-03-13 12:52:59 -07:00
Jingning Han
1ff4a5b3a2 Merge "Tune rate-distortion modeling for non-RD mode decision" 2014-03-13 09:14:13 -07:00
Jingning Han
b25e4061e8 Merge "Explicitly assign interp_kernel value in non-RD coding mode" 2014-03-13 09:14:00 -07:00
Deb Mukherjee
e39ecfaa98 Preliminary code for variance based paritioning
Brings back most of Jim's previous patch for choosing
partitioning based on variance while making it compatible
with the current state of the code. Also adds a
nonrd_use_partition() function to recursively encode  for any
arbitrary sb_type decisions within a 64x64 block; and
includes some refactoring.

Currently, when the VAR_BASED_PARTITIONING mode is turned on
for speed 7, there is a 10+% speed-up observed.

Experiments/improvements with this new partitioning method
will be conducted subsequently.

Change-Id: Ie6f43bfbde30583e941f450bf07c3b48828c9571
2014-03-12 15:06:20 -07:00
Jingning Han
7686c950b5 Tune rate-distortion modeling for non-RD mode decision
This commit adjusts the rate-distortion modeling for non-RD mode
decision. It puts more weights on energy from AC coefficients when
estimating the cost. The coding performance for rtc testset is
improved by 0.72%.

Change-Id: Ifa6ff11311a513ec2b10586589e82a9a21f6c61c
2014-03-12 14:31:59 -07:00
Jingning Han
4e9b0d6cef Explicitly assign interp_kernel value in non-RD coding mode
Assign interp_kernel value in MACROBLOCKD. This will be used to
select prediction filter coefficient sets and generate motion
compensated prediction.

Change-Id: I28c8dfb2dae6566f6939bb328aca5875c94bee65
2014-03-12 14:30:54 -07:00
Alex Converse
29a487c77f Add a conservative RD based active map in vp9.
Change-Id: I47b3c38aadfd8f3ea08515a18a5948aa1375c650
2014-03-10 15:48:43 -07:00
Deb Mukherjee
44a203f5cd Merge "Support for a fast diamond search" 2014-03-10 09:53:21 -07:00
Jim Bankoski
622f06eb59 Merge "vp9_reconinter.h static functions in header converted to global" 2014-03-10 07:36:05 -07:00
Deb Mukherjee
bead2e8fb4 Support for a fast diamond search
Adds a fast diamond search which is about 5% faster than FAST_HEX
with only a 0.1% drop in psnr when turned on for both speeds 5 and 7.
This search is turned on for speed 7.

Change-Id: I497630aa88a5148926086bb3038e7975e5f4eb98
2014-03-07 17:15:09 -08:00
Jingning Han
d2059b0d2d Skip mode check when mv has been tested
This commit allows the non-RD mode decision to skip mode RD modelling
check, if the motion vector associated with the current mode is
same as that of NEARESTMV mode. This makes speed -7 about 2% faster.
Previous change that converts cost metric from SAD to model based RD
value makes the codec 6% slower at speed -7.

Change-Id: I30cfec5452f606a671b8432a2f7f0c94fbb49fc8
2014-03-07 14:06:20 -08:00
Jingning Han
0250902475 Clearn up model_rd in non-RD coding mode
The rate-distortion model in non-RD coding mode is only applied to
luma component. This commit removed a few redundant addition steps.

Change-Id: Id8edc0a47c2dbef8deba43debe2c95db39454de3
2014-03-07 10:34:34 -08:00
Jingning Han
6849cde893 Merge "Use modeled rate distortion costs for non-RD mode" 2014-03-07 09:06:24 -08:00
Jingning Han
539c961ed4 Use modeled rate distortion costs for non-RD mode
This commit replaces SAD cost with modeled rate-distortion cost
for non-RD mode decision. It translates the prediction residual
SSE into estimate rate and reconstruction distorion costs, hence
capturing the quantization setting effect. The compression
performance of speed -7 for rtc set is improved by 14.79%.

Change-Id: Ifda014eb0501d13109fe7f92680bf1410b463632
2014-03-06 23:00:48 -08:00
Deb Mukherjee
26fa8cad01 Fixes static analysis issues
Removes some unused variables and assignments

Change-Id: I228f43d658ce50237d65ce8dc8ab2ccf0a0b21ca
2014-03-06 11:07:06 -08:00
Dmitry Kovalev
191e2e8455 Merge "Replacing int_mv with MV in full_pixel_motion_search()." 2014-03-05 21:44:54 -08:00
Deb Mukherjee
b80020d4db Refactoring motion search libs
The core motion estimation fucntions all return sad now consistently.
The only exception is vp9_full_pixel_diamond(), however the core diamond
and refining search routines called from vp9_full_pixel_diamond() also
return SAD. If variance of pred error + mv cost is desired it must be
calculated explicitly outside these functions. For very fast encoding,
hopefully this will eliminate some redundant computations.

Also suggests reimplementing FAST_HEX with the vp9_pattern_search
framework. It is not exactly the same as the existing FAST_HEX, but
performance is slightly better and speed is very similar. Enables
removing a lot of duplicate code.

Change-Id: I152736393438c25bdf7e96b37cbb8ce330f4f94a
2014-03-03 15:13:59 -08:00
Jim Bankoski
e5e9b05d68 vp9_reconinter.h static functions in header converted to global
Change-Id: I916944950deb22f4c2301d83a803b732bf3ecd77
2014-03-03 14:58:43 -08:00
Dmitry Kovalev
686b480ad1 Replacing int_mv with MV in full_pixel_motion_search().
Change-Id: I16dd4d4aaae8ce6a482da3c9d142f41fe9155e82
2014-03-03 14:43:06 -08:00
Paul Wilkins
74558bc582 Clear implied conversion to int64_t warning.
Clear warning caused by implied conversion from int32 to int64.

Change-Id: I473b37a54984cbfb22702eb3f712465881ff86e8
2014-03-03 15:41:21 +00:00
Dmitry Kovalev
4aacfb0f85 Merge "Replacing int_mv with MV." 2014-02-28 16:09:17 -08:00
Deb Mukherjee
dc8a2c07b3 Merge "Adds speed 8 to vp9 as reference" 2014-02-28 14:17:24 -08:00
Dmitry Kovalev
dac630bb22 Merge "Sorting and eliminating includes in vp9_pickmode.c." 2014-02-28 13:57:31 -08:00
Deb Mukherjee
f872a98b1b Adds speed 8 to vp9 as reference
Adds a speed 8 to VP9 where only the nearestmv (0 mv) is searched.
This seems to be about the same speed as vp8 speed 5.

Adds a new speed feature to disable inter modes based on a mask for
each blocksize.

Adds code for having lower complexity motion search methods
in nonrd pick mode function, even though speed 7 still uses DIAMOND
search for now.

Also uses HEX search for speed 6 rather than FAST_HEX which improves
psnr by 0.56% without any noticeable speed drop (tested on gipsmotion).

Change-Id: Ic13176572dbd3aed5884a26786940a4b1bbd8a75
2014-02-28 12:29:23 -08:00
Dmitry Kovalev
ce404e898c Replacing int_mv with MV.
Change-Id: Ib68d4a2de838254a66272324b909292048cbe60d
2014-02-28 12:11:46 -08:00
Dmitry Kovalev
b68e0a0c38 Sorting and eliminating includes in vp9_pickmode.c.
Change-Id: Icc057b819d80c608a6003fd7ef58aefed7a15824
2014-02-28 11:28:34 -08:00
Jingning Han
24c7ee78c5 Skip some mode SAD calculation in non-RD mode
This commit checks if the motion vector associated with the current
mode has been computed in previous mode tests. If possible, skip the
redundant reference block generation and SAD calculation in the
non-RD mode decision process.

For test sequence pedestrian_area 1080p, the runtime goes from
24261 ms to 23770 ms. This does not change compression performance.
The speed-up is mostly around places with consistent motion.

Change-Id: I97be63c6a2d07c57be26b3c600fbda3803adddda
2014-02-28 09:44:53 -08:00
Yunqing Wang
17b1e92d6c Merge "Enable using fast HEX search in non-rd pick mode" 2014-02-28 08:29:02 -08:00
Yunqing Wang
e6fd071243 Enable using fast HEX search in non-rd pick mode
Added fast HEX search while doing non-rd partition picking to
speed up the encoder.

Borg test(speed 7) on rtc set showed 1.8% overall PSNR loss.
Encoder speedup was 5% - 15% for different rtc clips.

Change-Id: I9c83026eabc70b69fcc747c90369ec60bfa3ca24
2014-02-27 11:48:07 -08:00
Jingning Han
08053edb80 Assign motion vector invalid value in intra mode
In non-RD mode decision, assign motion vector INVALID_MV when intra
prediction mode is selected.

Change-Id: I8f6ec39b71e755758f0f698074a2c17d934622ea
2014-02-27 11:26:39 -08:00
Jingning Han
c30d5703e5 Run sub-pixel motion search for NEWMV mode test
As Yunqing suggested, this commit makes non-RD mode decision always run
sub-pixel motion search in NEWMV mode. The compression performance
gains becomes fairly significant after we enabled sub-pixel accuracy
motion compensated prediction to calculate SAD cost.

For test sequences pedestrian_area at 1080p and vidyo1 at 720p, the
runtime goes slower by 5%. For rtc test set, the compression performance
is improved by 21.20%.

Change-Id: I38cbfdd5c53d79423e1fafb3154f8ddeed63bbf0
2014-02-26 11:17:51 -08:00
Jingning Han
c0e99f84f9 Use sub-pixel accuracy prediction non-RD mode
This commit builds the actual prediction block in sub-pixel accuracy
and uses which to calculate SAD for non-RD mode decision. In the trail
run on pedestrian_area at 1080p, rtc speed -7 runtime goes from
23495 ms -> 25107 ms (7% slower). The compression performance is
improved by 20.57% for rtc test set.

Change-Id: I438589cd103fe99f1b50c2d1939ac6ca43fa0157
2014-02-25 16:29:20 -08:00
Jingning Han
fd49e9678e Use dedicated variables to store the best mode
Use a set of dedicated variables to buffer the current best mode
in non-RD mode decision. This allows to use mode_info for more
complicated test in the non-RD process.

Change-Id: I6024c9feb0662afd3eb29f7017f7b5a5446f303f
2014-02-25 13:56:12 -08:00
Yaowu Xu
5a7ac03b9e Changed a constant array to static const type
Change-Id: I5b3918441f63ceb7889dcf546e9a2d781be4a3cb
2014-02-21 22:44:13 -08:00
Jingning Han
1fc0bde675 Refactor selected partition size coding for rtc
This commit makes a refactoring of the rtc_use_partition. It allows
the encoder to take a preferred block size for non-RD mode decision.
The boundary blocks are handled such that smaller block sizes that
fit in the boundary size will be used instread.

In rtc mode, the coding performance of speed -6 for pedestrian_1080p
goes from
158980 b/f, 38.934 dB, 22721 ms to
159008 b/f, 40.064 dB, 23721 ms.

For rtc set, the speed -6 compression performance is improved by
26%. Still about 2dB behind speed -5 at this point.

Change-Id: If0944f0880eaf1ad340bc325d97cea8d0f9dd53f
2014-02-21 14:49:57 -08:00
Jingning Han
d66a63f02b Enable reduced set of intra modes in rtc coding
This commit enables the use of DC, vertical, and horizontal intra
prediction mode in rtc non-RD mode decision. When the best cost value
of inter modes is above a given threshold, the encoder runs the
above three intra modes and selects the one that has minimum
prediction residual in terms of SAD.

This together with recent changes on non-RD mode decision and coding
control improves compression performance of speed -6 by
derf  91%
yt    61%
hd    46%
stdhd 52%

In terms of encoding speed, it is about 3 times faster than speed -5.

Change-Id: I6b483bfd0307e6482bb22a6676ae4e25a52b1310
2014-02-20 18:03:23 -08:00
Jingning Han
b99ed1ec5a Rework non-RD mode decision coding flow
In the first coding run of a 64x64 block, check the coding mode
for each 8x8 block. Will need a second annealing stage to decide
the partition size to be encoded.

Change-Id: Ida9417805ff3358979b0c0429d4099c023c88866
2014-02-19 11:13:05 -08:00
Jingning Han
e3178d0c1c Merge "Enable sub-pixel motion search for rtc mode" 2014-02-18 14:52:13 -08:00
Jingning Han
6478673933 Enable sub-pixel motion search for rtc mode
Run sub-pixel motion search when NEWMV gives lower rate-distortion
cost. This improves coding performance of derf set by 8%, std-hd by
2.2%.

Change-Id: Ife50f7fda8463927784fe59a41cc439c833e941a
2014-02-18 12:07:55 -08:00
Jim Bankoski
524bac612a vp9_pickmode.c resolve vs issue
Change-Id: Ibe4276fa9ffeb2ef30871d6db8ac75c4648f1e58
2014-02-14 11:19:31 -08:00
Dmitry Kovalev
fde1b6d7f2 Using MV instead of int_mv inside vp9_full_pixel_diamond().
Change-Id: If33a5a12c4025d9b5ec863dfccea7ee70f800665
2014-02-13 11:23:05 -08:00
Dmitry Kovalev
df6c523fed Merge "Renaming skip_coeff to skip for consistency." 2014-02-13 11:04:34 -08:00
Jim Bankoski
6a6797f5c9 Merge "vp9_pickmode.c static analysis issues resolved" 2014-02-13 08:31:49 -08:00
Jim Bankoski
c6eaf5f9f9 vp9_pickmode.c static analysis issues resolved
Change-Id: I716ea2967c6771d7adea571406f315a3197eab93
2014-02-13 06:20:57 -08:00
Dmitry Kovalev
004c8c636e Renaming skip_coeff to skip for consistency.
Change-Id: I036e815ca63d00cba71202ae09ba0f6ef745dcb8
2014-02-12 17:44:12 -08:00
Jingning Han
e8b7610e8f Use INTER_OFFSET in vp9_pick_inter_mode
Cosmetic change to use pre-defined macros.

Change-Id: I93e9fa90113d0242599048940b39694660385a6f
2014-02-12 09:14:29 -08:00
Jingning Han
f34a7eb253 Rework mode_info update in vp9_pick_inter_mode
This commit moves unnecessary mode_info update steps out of inter
mode test loop.

Change-Id: I3705450c44cda060021f4fd76c34f1b20d11658b
2014-02-10 15:20:11 -08:00
Yaowu Xu
f99eb936a4 Set proper rtc interp_filter at block level
Based on frame level settings.

Change-Id: I922c09c195ab055585eaa4f544c51aceb1203a7a
2014-01-31 16:32:19 -08:00
Yaowu Xu
dc9af3b509 fix IOCs
Left shift of negative values caused IOC warnings.

Change-Id: I6f41b020ca0ff51f7861719d41393e9460b75d4e
2014-01-31 15:07:51 -08:00
Jim Bankoski
ea8aaf15b5 create super fast rtc mode
This patch only works if the video is a width and height that are both
a multiple of 32..   It sets every partition to 16x16, and does INTRADC
only on the first frame and ZEROMV on every other frame.   It always does
does the largest possible transform, and loop filter level is set to 4.

Was ~20% faster than speed -5 of vp8

Now 20% slower but adds motion search ( every block ), nearest, near
and zeromv

The SVC test was changed because - while this realtime mode produces
bad quality albeit quickly, it isn't obeying all the rules it should
about which frames are available.

Change-Id: I235c0b22573957986d41497dfb84568ec1dec8c7
2014-01-29 08:39:39 -08:00
Dmitry Kovalev
e4755db40b Removing get_scale_ref_frame_idx() function + cleanup.
Change-Id: Ic9472f361f8749a165aa464c2fa859681b66381b
2014-01-22 17:10:37 -08:00
Dmitry Kovalev
ef9f7446be Cleaning up vp9_setup_buffer_inter() function.
Change-Id: Ia63e2d05fa5dca3fb67788b0f1f28e9802ef27dd
2014-01-21 19:46:07 -08:00
Jingning Han
2f52decd22 Inter-frame non-RD mode decision
This commit setups a test framework for real-time coding. It enables
a light motion search for non-RD mode decision purpose.

Change-Id: I8bec656331539e963c2b685a70e43e0ae32a6e9d
2014-01-16 12:35:04 -08:00