1258 Commits

Author SHA1 Message Date
Yunqing Wang
4d2c376923 Early termination in encoding partition search
In the partition search, the encoder checks all possible
partitionings in the superblock's partition search tree.
This patch proposed a set of criteria for partition search
early termination, which effectively decided whether or
not to terminate the search in current branch based on the
"skippable" result of the quantized transform coefficients.
The "skippable" information was gathered during the
partition mode search, and no overhead calculations were
introduced.

This patch gives significant encoding speed gains without
sacrificing the quality.

Borg test results:
1. At speed 1,
   stdhd set: psnr: +0.074%, ssim: +0.093%;
   derf set:  psnr: -0.024%, ssim: +0.011%;
2. At speed 2,
   stdhd set: psnr: +0.033%, ssim: +0.100%;
   derf set:  psnr: -0.062%, ssim: +0.003%;
3. At speed 3,
   stdhd set: psnr: +0.060%, ssim: +0.190%;
   derf set:  psnr: -0.064%, ssim: -0.002%;
4. At speed 4,
   stdhd set: psnr: +0.070%, ssim: +0.143%;
   derf set:  psnr: -0.104%, ssim: +0.039%;

The speedup ranges from several percent to 60+%.
                 speed1    speed2    speed3    speed4
(1080p, 100f):
old_town_cross:  48.2%     23.9%     20.8%     16.5%
park_joy:        11.4%     17.8%     29.4%     18.2%
pedestrian_area: 10.7%      4.0%      4.2%      2.4%
(720p, 200f):
mobcal:          68.1%     36.3%     34.4%     17.7%
parkrun:         15.8%     24.2%     37.1%     16.8%
shields:         45.1%     32.8%     30.1%      9.6%
(cif, 300f)
bus:              3.7%     10.4%     14.0%      7.9%
deadline:        13.6%     14.8%     12.6%     10.9%
mobile:           5.3%     11.5%     14.7%     10.7%

Change-Id: I246c38fb952ad762ce5e365711235b605f470a66
2014-08-28 11:27:28 -07:00
Deb Mukherjee
bb2a9abb1e Merge "Updates vp9_pattern search to return integer sads" 2014-08-28 09:38:56 -07:00
Deb Mukherjee
04b100b23e Updates vp9_pattern search to return integer sads
Updates the vp9_pattern_search function to return integer one-away
neighbors' sad values, for subsequent use in speeding up the
sub-pel search. Also, removes code for the do_refine option
which is not being used currently.
Updates the integer and subpel functions to pass in a 5-element
sad list for output or input.

A new pruned sub-pel search algorithm is implemented that uses
the sad returned from the integer pel search. But it is not
deployed yet.

Change-Id: Ifa9f5ad024b5b660570366d2bd900343e1891520
2014-08-28 06:49:58 -07:00
Jingning Han
143be253b6 Merge "Re-work RD modeling based on inter frame prediction residual" 2014-08-27 18:48:49 -07:00
Jingning Han
34675e6631 Merge "Re-use switchable rate value in handle_inter_mode" 2014-08-27 18:48:41 -07:00
Jingning Han
4e4f4ba868 Merge "Add an early termination check in handle_inter_mode" 2014-08-27 18:48:32 -07:00
Jingning Han
6924fddb08 Merge "Use max txfm size unit in rate-distortion cost modeling" 2014-08-27 18:48:24 -07:00
Jingning Han
993ef8bd4c Re-work RD modeling based on inter frame prediction residual
This commit re-work the operation flow related to prediction
residual generation and the rate-distortion modeling. It saves one
call for model_rd_for_sb.

Change-Id: Icaf96c0ff09c903637ed5283448afe01d798195f
2014-08-27 15:03:32 -07:00
Jingning Han
4db022c368 Re-use switchable rate value in handle_inter_mode
The value of switchable rate has been stored in a local variable.
This change skips the second call to vp9_get_switchable_rate() by
reusing the local variable.

Change-Id: Ib7d3fef7621cc4bde94c6d6e6b3a71f1fd4559f2
2014-08-27 15:03:16 -07:00
Jingning Han
cd228fcdb8 Add an early termination check in handle_inter_mode
Check the mode and motion vector cost. If it is already above
the existing best rate-distortion cost, skip the rest check process
on this mode.

Change-Id: Ie065cebdfda2a3be3be18b8e8b43dc29aaa8c179
2014-08-27 14:59:52 -07:00
Jingning Han
ec7ce316d2 Use max txfm size unit in rate-distortion cost modeling
This commit makes the rate distortion modeling run in the unit of
maximum transform block size. No compression/speed change observed.
It is for the use of later fast forward transform purpose.

Change-Id: Ibaaedb69c765e8d0c5d5012f0ec07f36fd9f68fd
2014-08-27 14:59:02 -07:00
Yaowu Xu
bcfb1ffb9d Merge "add a new interp filter search strategy." 2014-08-26 17:30:42 -07:00
Yaowu Xu
1144fee3d5 add a new interp filter search strategy.
This commit addes a new strategy to reduce the search for optimal
interpolation filter type. The encoder counts and store how many each
filter type is selected and used for each of the reference frames.
A filter type that is rarely used for all three reference frames is
masked out to avoid computation.

The impact on compression is neglectible:
-0.02% on derf
+0.02% on stdhd

Encoding time is seen to reduce by 2~3%.

Change-Id: Ibafa92291b51185de40da513716222db4b230383
2014-08-26 09:05:04 -07:00
Dmitry Kovalev
4478553efc Removing tx_stepdown_count from VP9_COMP.
The variable is never read.

Change-Id: I94141c1667fa5d10604cd6f83c5f64df107dee94
2014-08-25 14:42:05 -07:00
Dmitry Kovalev
45425f8c1e Removing is_best_mode() function.
Change-Id: Iccd7cec885e8aeb0e54613d888f9960c393cee0b
2014-08-21 11:32:33 -07:00
Jim Bankoski
f274ac7df5 vp9_rdopt.c : signed unsigned mismatch in choose_tx_size
Change-Id: Id68736ffa71843b02ee3634edc525219d6e54d4f
2014-08-19 06:40:29 -07:00
Yunqing Wang
4d98b50be5 Merge "Add early termination in transform size search" 2014-08-18 19:00:24 -07:00
Yunqing Wang
ba70f16011 Add early termination in transform size search
In the full-rd transform size search, we go through all transform
sizes to choose the one with best rd score. In this patch, an
early termination is added to stop the search once we see that the
smaller size won't give better rd score than the larger size. Also,
the search starts from largest transform size, then goes down to
smallest size.

A speed feature tx_size_search_breakout is added, which is turned off
at speed 0, and on for other speeds. The transform size search is
turned on at speed 1.

Borg test results:
1. At speed 1,
   derf set: psnr gain: 0.618%, ssim gain: 0.377%;
   stdhd set: psnr gain: 0.594%, ssim gain: 0.162%;
   No noticeable speed change.
3. At speed 2,
   derf set: psnr loss: 0.157%, ssim loss: 0.175%;
   stdhd set: psnr loss: 0.090%, ssim loss: 0.101%;
   speed gain: ~4%.

Change-Id: I22535cd2017b5e54f2a62bb6a38231aea4268b3f
2014-08-18 16:27:04 -07:00
Jingning Han
6a464eca05 Speed up mode search depending on relative ref frame position
This commit enables the encoder to record the location of the
center frame to generate alter reference frame. It then allows to
skip checking prediction modes of other reference frame types when
it comes to encode this frame.

The speed 3 runtime is reduced for the test sequences:
bus at CIF 1000 kbps, 9791 ms -> 9446 ms, i.e., 3.5% speed-up,
pedestrian at 1080p 2000 kbps, 184043 ms -> 175730 ms, i.e., 4.5%
speed-up.

No compression performance change observed.

Change-Id: Iacfde3bcc1445964e7a241f239bd6ea11cb94bd1
2014-08-18 16:06:54 -07:00
Jingning Han
5386df41ba Remove unused variables in vp9_rd_pick_inter_mode_sb
Change-Id: Ib52e1ce5aa4a22d70e124c3c29a92e00ffac50e4
2014-08-15 15:28:25 -07:00
Jingning Han
5d453e39cf Remove unused code in vp9_rd_pick_inter_mode_sb
Change-Id: I4f0bac3367b1a7c5e7f746b97ca619b81c8dfc49
2014-08-15 15:08:46 -07:00
Jingning Han
1e30547984 Skip mode search based on reference frame consistency
This commit enables the encoder to skip NEARMV and ZEROMV if the
above and left blocks have identical reference frame, and the
current reference is different from that. It reduces the runtime
of speed 3 for test sequences:
bus cif at 1000 kbps 10064 ms -> 9823 ms
pedestrian 1080p at 2000 kbps 193078 ms -> 189559 ms

The compression performance is changed by
derf  -0.085%
stdhd -0.103%

Change-Id: If304f26d42e6412152a84c3dd7b02635c38444f4
2014-08-13 14:16:18 -07:00
Jingning Han
0daadeb60c Enable motion field based mode seach skip
This commit allows the encoder to check the above and left neighbor
blocks' reference frames and motion vectors. If they are all
consistent, skip checking the NEARMV and ZEROMV modes. This is
enabled in speed 3. The coding performance is improved:

pedestrian area 1080p at 2000 kbps,
from  74773 b/f, 41.101 dB, 198064 ms
to    74795 b/f, 41.099 dB, 193078 ms

park joy 1080p at 15000 kbps,
from 290727 b/f, 30.640 dB, 609113 ms
to   290558 b/f, 30.630 dB, 592815 ms

Overall compression performance of speed 3 is changed
derf  -0.171%
stdhd -0.168%

Change-Id: I8d47dd543a5f90d7a1c583f74035b926b6704b95
2014-08-13 12:15:13 -07:00
Jingning Han
245e57c78e Merge "Enable fast forward txfm and quant for rate-distortion search" 2014-08-11 17:56:48 -07:00
Jingning Han
b4b09c9796 Enable fast forward txfm and quant for rate-distortion search
This commit enables encoder to select fast forward transform and
quantization path according to the prediction residual sse/variance,
in the rate-distortion optimization scheme.

Change-Id: Ief9fc3844fd4107166d401970e800c6e5ce2b5fe
2014-08-08 16:16:51 -07:00
Jingning Han
ef1023b786 Merge "Cosmetic indent fix in vp9_rd_pick_inter_mode_sb" 2014-08-04 15:39:51 -07:00
Jingning Han
395c934720 Cosmetic indent fix in vp9_rd_pick_inter_mode_sb
Change-Id: I303213fbb0ad5fbf28f0ca17271624b75e9d6319
2014-08-04 15:00:47 -07:00
Jim Bankoski
64126c3aa4 set but unused var in vp9_rdopt
Change-Id: Ie813af24d23c7d72b38749b3382b315daf516be5
2014-08-04 08:56:05 -07:00
Jim Bankoski
7f63dabfe9 break at the end of clauses with assert(0) to avoid gcc warning
Change-Id: I1b3c5337f018dde27dc819ab18bd081d169a91e8
2014-08-04 08:52:53 -07:00
Yunqing Wang
d5b8a1b324 Merge "Code cleanup in rdopt.c" 2014-07-31 12:20:06 -07:00
Yunqing Wang
678d747259 Code cleanup in rdopt.c
Moved encode_breakout code out of handle_inter_mode().

Change-Id: Icd661136b05fdf163768c406f91e0c98a8df89eb
2014-07-31 09:55:40 -07:00
Jingning Han
c36f78b054 Use frame index directly in get_chessboard_index
The get_chessboard_index() used to call the entire VP9_COMMON
struct pointer to retrieve the chessboard pattern index. This cl
makes it call the frame index directly.

Change-Id: I3cad9d209ea2e77a358085a04fe1ff0ddec5ba03
2014-07-29 10:55:56 -07:00
Jingning Han
3d5f17311c Merge "Remove unnecessary conditional assignment" 2014-07-25 15:56:31 -07:00
Jingning Han
7112d70f24 Remove unnecessary conditional assignment
The assignment of the variable mode_excluded in
vp9_rd_pick_inter_mode_sub8x8 takes redundant conditional jump.
This commit removes it.

Change-Id: Ie195fbe6e54ec2ade7093d562c456a2e93143704
2014-07-24 15:34:11 -07:00
Alex Converse
6eae35c07f Refactor inter/intra_suberblock_yrd.
Move txfm_rd_in_plane into choose_tx_size_from_rd and cleanup callers.

Change-Id: I1df2d7dc984802bd5e204cbe881ada0d75fbb3f7
2014-07-24 11:21:51 -07:00
Jingning Han
374c885919 Merge "Remove redundant argument entry in handle_inter_mode" 2014-07-23 15:07:01 -07:00
Jingning Han
787e8240d5 Merge "Use the chessboard pattern pred search in newmv mode" 2014-07-23 15:06:52 -07:00
Jingning Han
e945c56d4a Remove redundant argument entry in handle_inter_mode
The value of mode_excluded has been properly set in
vp9_rd_pick_inter_mode_sb(). It is redundant to send it in
handle_inter_mode() and re-set the value again.

Change-Id: I408d4731f2f42e0bcf3ae62e85757717bb410471
2014-07-23 12:04:45 -07:00
Jingning Han
4f2f86725b Use the chessboard pattern pred search in newmv mode
This commit extends the chessboard pattern prediction filter search.
If the above and left blocks have the same prediction filter type,
the encoder will skip the prediction filter type search and use the
reference one.

The overall chessboard pattern prediction filter type search reduces
speed 3 runtime for hard clips. Experiments on park joy at 1080p
and 15000 kbps show that the runtime goes from 723265 ms to 65832 ms,
i.e., about 10% speed-up. Compression performance wise, it affects
the coding quality by

Change-Id: I880975497c7ad166532e9eea9bf46684d77ff327
derf:    -0.326%
yt:      -0.257%
hd:      -0.241%
stdhd:   -0.417%
2014-07-23 11:59:52 -07:00
Jingning Han
66d5757695 Merge "Remove redundant num_refs definition" 2014-07-23 10:34:54 -07:00
Jingning Han
353819103e Remove redundant num_refs definition
Use is_comp_pred to replace the use case of num_refs.

Change-Id: I4d0c1e14d5f728428a2ae3d293cd2b4a8b2f31d8
2014-07-23 09:29:51 -07:00
Jingning Han
0e5edf4eae Merge "Enable chessboard inter prediction filter type search" 2014-07-23 09:12:56 -07:00
Jingning Han
54ad09586c Enable chessboard inter prediction filter type search
This commit enables a chessboard pattern prediction filter type
search scheme for rate-distortion optimization speed-up. For the
inferred motion vector modes, the encoder can re-use its above/left
neighbor blocks' prediction filter type and skip a full test on
all possible filter types. Such operation is turned on/off
alternatively in a chessboard manner.

It is turned on in speed 3. For test clip pedestrian 1080p, the
runtime is reduced from 231500 ms -> 221700 ms. The compression
performance is changed:
derf:  -0.147%
yt:    -0.134%
hd:    -0.079%
stdhd: -0.220%

Change-Id: I1912f278e7576c2dc632688e3ad7a257410c605a
2014-07-22 16:49:03 -07:00
Jingning Han
f0f428e9ba Merge "USE local best_filter variable in handle_inter_mode" 2014-07-22 14:21:34 -07:00
Jingning Han
5de6114e8f USE local best_filter variable in handle_inter_mode
This should be a local variable. Move the definition from
vp9_rd_pick_inter_mode_sb to handle_inter_mode.

Change-Id: I14f4168bb1c896ed04e8f6d4cd89fbf4c9839944
2014-07-22 11:35:59 -07:00
Yunqing Wang
765485cab2 Add -DNDEBUG when config option debug is disabled
For gcc, when libvpx config option debug is disabled, added the
flag -DNDEBUG to disable the assertions in libvpx for some speedup.

Change-Id: Ifcb7b9e8ef5cbe5d07a24407b53b9a2923f596ee
2014-07-21 09:20:03 -07:00
Yunqing Wang
6298e232e1 Merge "Code refactoring: use defined inline functions" 2014-07-11 13:46:46 -07:00
Yaowu Xu
6673d2f309 Remove an unused parameter in vp9_init_search_range()
Change-Id: I3d9130e726a1299fd258f6dfe93315e2d12f76da
2014-07-11 10:32:39 -07:00
Yunqing Wang
1b5e9871f7 Code refactoring: use defined inline functions
Changed to use defined inline functions consistently through
the code.

Change-Id: I7644d24fa7a837378564a6e0790416d3725dd200
2014-07-11 10:30:25 -07:00
Yunqing Wang
a581da218e Remove repetitive code in mcomp.c
Deleted vp9_find_best_sub_pixel_comp_tree(), and combined it in
vp9_find_best_sub_pixel_tree().

Change-Id: Ifb25763c8b19822df5537cc1daa76ce88dc3b056
2014-07-09 14:50:50 -07:00