Commit Graph

1259 Commits

Author SHA1 Message Date
Jingning Han
66f440f1ee Refactoring hybrid transform coding
The forward and inverse hybrid transforms are now performed using
single function modules, where the dimension is sent as argument.

Added an inline function clip8b to clip the reconstruction pixels
into range of 0-255.

Change-Id: Id7d870b3e1aefc092721c80c0af6f641eb5f3747
2012-08-06 17:26:21 -07:00
Daniel Kang
106815f830 Remove code to set transform size.
It's reset in calling functions.

Change-Id: Ia5df086de615aac6feb3c911fe298626a002b850
2012-08-06 16:39:05 -07:00
Daniel Kang
6630895daf Combine cost_coeffs functions.
Change-Id: I3421d07fdcf2148d75c8dbfc8c44bc2d96a3a8f6
2012-08-06 16:22:15 -07:00
Daniel Kang
9921b55b22 Merge token_costs into a single table.
Change-Id: Id1ea2d543f12d4589df7038d5fb7bac1fee4da11
2012-08-06 15:55:54 -07:00
Daniel Kang
680af9acd5 More refactoring of RD cost to make later changes easier.
Change-Id: Ia6a290a6a53fdb88c971bea2de47ac73dcb7a364
2012-08-06 15:55:54 -07:00
Daniel Kang
02f3aaf76c Minor refactoring of RD cost functions.
Change-Id: I876955b1d577321b5cd3f71e3fb2c4083ad5111b
2012-08-06 15:55:54 -07:00
Christian Duvivier
82edabce75 Add x86_64-darwin11-gcc target.
This allows building on MountainLion as the 10.6 SDK has been
removed from the latest Xcode version (4.4 4F250). Also fix
all warnings for that build.

Change-Id: Ib70bca4a25295f13595f0d10ea9f0229631de5a4
2012-08-06 15:26:58 -07:00
Deb Mukherjee
2af5473a90 Merging in high_precision_mv experiment
Merged in the high_precision_mv experiment to make it easier
to work on new mv encoding strategies. Also removed
coef_update_probs3().

Change-Id: I82d3b0bb642419fe05dba82528bc9ba010e90924
2012-08-03 13:38:49 -07:00
Jingning Han
fcbff9ee04 Replacing the 8x8 DCT with 8x8 ADST/DCT for I8x8
Fixed the code review comments.

Under the htrans8x8 experiment the 8X8 DCT in the
I8X8 mode is replaced with a combination of 8X8 ADST and
DCT.

Overall coding gains with the htrans8x8 experiment are:
derf:   0.486
std-hd: 1.040
hd:     1.063
yt:     0.506

Note that part of the gain comes from bigger transforms
(8x8 instead of 4x4) and part comes from replacing the DCT
wth the ADST.

Change-Id: I92ca6bbfce11b4165d612b81d9adfad4d010c775
2012-08-03 12:02:07 -07:00
Daniel Kang
e6de9c2e5d Merge "16x16 DCT blocks." into experimental 2012-08-03 10:22:04 -07:00
Deb Mukherjee
4aabfaa5d0 Merge "Merging and bug-fix in enhanced_interp experiment" into experimental 2012-08-03 10:18:25 -07:00
Daniel Kang
fed8a1837f 16x16 DCT blocks.
Set on all 16x16 intra/inter modes

Features:
- Butterfly fDCT/iDCT
- Loop filter does not filter internal edges with 16x16
- Optimize coefficient function
- Update coefficient probability function
- RD
- Entropy stats
- 16x16 is a config option

Have not tested with experiments.

hd:     2.60%
std-hd: 2.43%
yt:     1.32%
derf:   0.60%

Change-Id: I96fb090517c30c5da84bad4fae602c3ec0c58b1c
2012-08-02 17:33:10 -07:00
Ronald S. Bultje
d15b07d8c6 A few more conversions from 32-bit to 64-bit RD calculations.
Change-Id: Ida01bb8f4b35fde24b0f096329d326833f1fdc7e
2012-08-02 11:01:48 -07:00
Ronald S. Bultje
a4ad3809e2 Make RDCOST() results 64-bit.
This prevents various integer overflows that several of us have been
seeing in the past few days.

Change-Id: If0020613a608e13d311b7838796c7ad53cb97255
2012-08-02 10:26:11 -07:00
Jingning Han
c7846ebc34 Use 8x8 DCT transform for I8X8 prediction mode
Apply 2D-DCT transform of dimension 8x8 to encode prediction
residuals of I8X8 mode.
Brought back block type 3 probability context model for 8x8 tokens,
which is used for the coefficients of Y blocks in I8x8 modes. The
coefficient costs estimate of I8X8 mode in rate-distortion is also
changed appropriately.
Performance results:
derf:   0.246
yt:     0.114
std-hd: 0.730
hd:     0.670

Change-Id: If1d970eeb4e1827c9f0d2c5b27d33089b347ea27
2012-08-02 09:09:17 -07:00
Deb Mukherjee
0ebf548c75 Merging and bug-fix in enhanced_interp experiment
Merged the enhanced_interp experiment.
Found and fixed a bug in the include files framework, whereby
certain encoder files were still using the old INTERP_EXTEND
value of 3 instead of 4. The thresholds for mv range mcomp.c
need a small adjustment to prevent crashes.

The results are more or less unchanged.

Change-Id: Iac5008390f1efc97ce1102fbb5f8989c847fb579
2012-07-31 11:45:31 -07:00
Deb Mukherjee
5259744145 Adds support for switchable interpolation filters.
Allows for swtiching/setting interpolation filters at the MB
level. A frame level flag indicates whether to use a specifc
filter for the entire frame or to signal the interpolation
filter for each MB. When switchable filters are used, the
encoder chooses between 8-tap and 8-tap sharp filters. The
code currently has options to explore other variations as well,
which will be cleaned up subsequently.

One issue with the framework is that encoding is slow. I
tried to do some tricks to speed things up but it is still slow.
Decoding speed should not be affected since the number of
filter taps remain unchanged.

With the current version, we are up 0.5% on derf on average but
some videos city/mobile improve by close to 4 and 2% respectively.
If we did a full-search by turning the SEARCH_BEST_FILTER flag
on, the results are somewhat better.

The framework can be combined with filtered prediction, and I
seek feedback regarding that.

Rebased.

Change-Id: I8f632cb2c111e76284140a2bd480945d6d42b77a
2012-07-30 11:33:43 -07:00
Deb Mukherjee
9984a155d6 Merges several experiments
The following five experiments are merged:

newentropy
newupdate
adaptive_entropy (also includes a couple of parameter changes
                  that improves results a little
                  in common/entropymode.c and encoder/modecosts.c
                  that were not merged from the internal branch)
newintramodes
expanded_coef_context

Change-Id: I8a142a831786ee9dc936f22be1d42a8bced7d270
2012-07-27 12:12:39 -07:00
Daniel Kang
bd3a9f1346 Update 8x8 tokenizer to not encode an EOB if seg_eob is active.
Change-Id: Id2bac0597b6dd1ce41bd78cbd9e9d876b71cd8a2
2012-07-23 16:46:44 -07:00
Yaowu Xu
6f6856839b fixed a compiling issue
Change-Id: I9ccbea76eaeb81ce7b833dd92216734725796e84
2012-07-19 14:52:04 -07:00
Jingning Han
9824230fe3 Adds hybrid transform
Adds ADST/DCT hybrid transform coding for Intra4x4 mode.
The ADST is applied to directions in which the boundary
pixels are used for prediction, while DCT applied to
directions without corresponding boundary prediction.

Adds enum TX_TYPE in b_mode_infor to indicate the transform
type used.

Make coding style consistent with google style.
Fixed the commented issues.

Experimental results in terms of bit-rate reduction:
derf:   0.731%
yt:     0.982%
std-hd: 0.459%
hd:     0.725%

Will be looking at 8x8 transforms next.

Change-Id: I46dbd7b80dbb3e8856e9c34fbc58cb3764a12fcf
2012-07-19 13:02:57 -07:00
Yaowu Xu
6fd0929fb5 Merge "removed floating point version 8x8 fdct" into experimental 2012-07-18 10:53:57 -07:00
Yaowu Xu
df35f6b3f0 Merge "cleanup experiments in configure" into experimental 2012-07-18 10:53:46 -07:00
Ronald S. Bultje
84d2298828 Don't auto-default to comp-pred unless content is static.
Although it can be useful, in some cases it causes artifacts in motion-
based content.

Change-Id: I27bb80901020c40636e2018a8217759175a23f5b
2012-07-18 09:39:52 -07:00
Yaowu Xu
d632bf8cf5 removed floating point version 8x8 fdct
the integer version has very good precision, the float version is no
longer useful. this commit also removes the experiment option from
configure script.

Change-Id: Ibb92e63c9f5083357cdf89c559d584a7deb3353f
2012-07-17 22:50:47 -07:00
Yaowu Xu
11e23e673d cleanup experiments in configure
this commit removes a number of experiment options from configure
script. the associated experiments are already fully merged, the
options in configure script have no effect at all.

Change-Id: I8054ccaee0a04610162ed76ac9e59c4538217113
2012-07-17 22:34:07 -07:00
John Koleszar
c6b9039fd9 Restyle code
Approximate the Google style guide[1] so that that there's a written
document to follow and tools to check compliance[2].

[1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml
[2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py

Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-07-17 11:46:03 -07:00
Ronald S. Bultje
cb1e60fb28 Fix bug in reference frame counting.
vp8_encode_inter_macroblock() is called in both pick_mb_modes() as
well as encode_sb(), thus the number of macroblocks in the counter
were twice as big as actual numbers. This doesn't affect output.

Change-Id: I6de8a996ee44d2f7f2080d8d2177dd7bc6207c93
2012-07-16 17:49:59 -07:00
Ronald S. Bultje
da8159a479 Add missing mv_{col,row}_{min,max}_sb variables to MACROBLOCK struct.
This allows CONFIG_SUPERBLOCKS experiment to almost compile succesfully,
except for the missing pick_sb_modes() function.

Change-Id: Ib2322f2aacdc371e8066f2eb4a8d761c40490b4d
2012-07-16 15:25:59 -07:00
Ronald S. Bultje
6c0b21c075 Use 8x8 transform for all 16x16 intra prediction modes in keyframes.
Also use the 8x8 transform in the RD loop.

Change-Id: If1a25aa15d3f64e05c9a31875e309d8cd5bc812d
2012-07-16 12:22:36 -07:00
Ronald S. Bultje
1f889ad63f Keep statistics for chosen intra modes in keyframes also.
Change-Id: Ibfecf1fb0fcf51fb349746fec015b1a3e44a4530
2012-07-16 11:32:44 -07:00
Christian Duvivier
5a5ced3e79 Get rid of some cast.
Change-Id: Iff5ccf2400bcb7b78c7b70f289ea65c982efddd5
2012-06-29 15:15:00 -07:00
Christian Duvivier
c5b70370dc Fix warnings.
Change-Id: I6dd27a0ea661e6f10a55d7841021b66ae4fc72ae
2012-06-29 14:46:11 -07:00
Hui Su
e44ee38aef Add lossless compression mode.
This commit adds lossless compression capability to the experimental
branch. The lossless experiment can be enabled using --enable-lossless
in configure. When the experiment is enabled, the encoder will use
lossless compression mode by command line option --lossless, and the
decoder automatically recognizes a losslessly encoded clip and decodes
accordingly.

To achieve the lossless coding, this commit has changed the following:
    1. To encode at lossless mode, encoder forces the use of unit
quantizer, i.e, Q 0, where effective quantization is 1. Encoder also
disables the usage of 8x8 transform and allows only 4x4 transform;
    2. At Q 0, the first order 4x4  DCT/IDCT have been switched over
to a pair of forward and inverse Walsh-Hadamard Transform
(http://goo.gl/EIsfy),  with proper scaling applied to match the range
of the original 4x4 DCT/IDCT pair;
    3. At Q 0, the second order remains to use the previous
walsh-hadamard transform pair. However, to maintain the reversibility
in second order transform at Q 0, scaling down is applied to first
order DC coefficients prior to forward transform, and scaling up is
applied to the second order output prior to quantization. Symmetric
upscaling and downscaling are added around inverse second order
transform;
    4. At lossless mode, encoder also disables a number of minor
features to ensure no loss is introduced, these features includes:
        a. Trellis quantization optimization
        b. Loop filtering
        c. Aggressive zero-binning, rounding and zero-bin boosting
        d. Mode based zero-bin boosting

Lossless coding test was performed on all clips within the derf set,
to verify that the commit has achieved lossless compression for all
clips. The average compression ratio is around 2.57 to 1.
(http://goo.gl/dEShs)

Change-Id: Ia3aba7dd09df40dd590f93b9aba134defbc64e34
2012-06-28 17:09:47 -07:00
Adrian Grange
bbc926dca2 Added Prediction Filter to Mode Selection
Added the ability to optionally filter the prediction data
when inter modes are selected (excludes SPLITMV, for now).

The mode selection loop considers both the filtered and
non-filtered prediction data when choosing mode. The filter
can be turned on/off at the frame-level, or signaled for
each MB.

Change-Id: I1b783c71d95a361ab36c761b07e8a6b06bc36822
2012-06-27 14:51:41 -07:00
Deb Mukherjee
f3dff40255 Continued adaptive entropy coding
Incorporates mv_ref, mbsplit and second_mv into the adaptive
entropy framework. The mv_ref framework has been modified from
before.

Adds some clean-ups and fixes.

Results with the adaptive entropy experiment are currently up by
+1.93% on derf; +2.33% std-hd and +1.87% yt-hd.

Fixed a nasty intermittent bug.

Change-Id: I4b1ac9f9483b48432597595195bfec05f31d1e39
2012-06-25 18:09:54 -07:00
Christian Duvivier
e0a338c428 Fix a bunch of warnings.
Change-Id: I883b4f63559c1cf66a25016e946b1b1ae98a20da
2012-06-18 17:30:36 -07:00
Deb Mukherjee
1fe85a35e0 Adaptive entropy coding of coefficients, modes, mv.
This patch incorporates adaptive entropy coding of coefficient tokens,
and mode/mv information based on distributions encountered in a frame.
Specifically, there is an initial forward update to the probabilities
in the bitstream as before for coding the symbols in the frame, however
at the end of decoding each frame, the forward update to the
probabilities is reverted and instead the probabilities are updated
towards the actual distributions encountered within the frame.
The amount of update is weighted by the number of hits within each
context.

Results on derf/hd/std-hd are all up by 1.6%.

On derf, the most of the gains come from coefficients, however for the
hd and std-hd sets, the most of the gains come from the mode/mv
information updates.

Change-Id: I708c0e11fdacafee04940fe7ae159ba6844005fd
2012-06-15 10:35:23 -07:00
Yaowu Xu
074a38b1fd changed to use a fixed number for update probabity
This commit is to remove two arrays, which contain the probabilities
of how likely each probability in coef_probs table is updated. The
commit changed to use a fixed number "252".

Surprisedly, the overall impact on quality is close to zero, which
basically says the two big static arrays are not helpful at all.
derf: -0.016%, -0.020%
std-hd: 0.000%, -0.013%
yt: -0.022%, +0.007%
yt-hd: -0.038%, +0.034%

Change-Id: Ifee94d28a37dcab4f1d2b994bd5b07575be42b72
2012-06-12 14:57:54 -07:00
Yaowu Xu
bb25083d65 Added the ability to accumulate coef stats across encodings
This commit added the ability to accumulate the coef stats across
different encodings using an intermediate binary stats files. The
accumulation happens only the binary stats file exists in current
directory. The encoder needs to be built with "ENTROPY_STATS" to
allow the output. The commit also fixed a few formating issues in
output stats file.

Change-Id: Ib1a41180aa554845cf51e4421a230b128a3a82b4
2012-06-12 14:41:51 -07:00
Paul Wilkins
36fe9735a9 Merge "Merge of further two pass rc clean up and adjustments." into experimental 2012-06-12 16:29:46 +00:00
Paul Wilkins
9b2c63e982 Merge "Key frame and GF sizing changes." into experimental 2012-06-12 16:19:13 +00:00
Paul Wilkins
7eb66d82ca Merge of further two pass rc clean up and adjustments.
Changes to calculation of  sr_coded_error to include 0,0 case.

Experimental use of sr_coded_error in calculating correction factor
for estimating the allowable Q range.

Reinstated some code needed for calculating section_intra_rating.

Add flash detection in calculation of KF boost

Increased tolerance in testing candidate key frames (needed with
longer motion search as this tends to slightly increase inter %.

Zbin changes for 8x8.

Other minor adjustments, refactoring and bug fixes.

Reinstated some motion break out clauses in boost loop
as their removal hurt a few 50fps  clips badly in the std set.
It may be possible to remove them again later if a better way
can be found of preventing overly long gf intervals.

Change-Id: Iee686d0c31072828bb1ccd2bc63f5f1c7c548ea2
2012-06-12 17:15:29 +01:00
Paul Wilkins
0cb4985b51 Key frame and GF sizing changes.
Changes to the equations for kf and gf minQ and to
the boost calculations for kg and gf

Change-Id: I312031c910e6a575334f49075c32f49a8dfff239
2012-06-08 10:49:22 +01:00
Adrian Grange
04fa2ecdf2 Fixed bug where invalid pointer is dereferenced
Variables m & mi were being dereferenced when they might
hold invalid values.

The fix is simply to move these dereferences to after the
point at which mb_row and mb_col are tested for validity.

Change-Id: Ib16561efa9792dc469759936189ea379d374ad20
2012-05-31 11:07:17 -07:00
Paul Wilkins
b8c369053b Rate control fix.
This fix addresses some problems with very complex clips like
handling of flashes on clips like crew (which was made worse
by an earlier patch (derf and std-hd)).

Most clips a small effect but some between 1 & 2%

Derf +0.039, +0.211%
YT +0.042, +0.083%

Change-Id: I65fc7c13afc31482040068544dd65b8808f5cb4a
2012-05-30 09:53:52 -07:00
Paul Wilkins
09a9701272 Merge "fixed one more compiling issue with VC" into experimental 2012-05-29 10:06:49 +00:00
Paul Wilkins
f66747fe38 Remove "est_max_qcorrection_factor"
Removed the local scaling factor est_max_qcorrection_factor
and related code to simplify estimateq calculation (little effect
anyway)

Cap range of total correction factor.

Slight change to break out case to turn off arf.

Change-Id: I748187737ba93cfadf016f3dfdf8d2741934067f
2012-05-29 09:55:04 +01:00
Yaowu Xu
2722cfb4b8 fixed one more compiling issue with VC
One more compiling issue, only happened in Visual C under a specific
combination of experiments.

Change-Id: Iba0775dc0ea7342b556348a9cdf60a05a59e0241
2012-05-25 21:52:09 -07:00
Yaowu Xu
ea0e4f1743 fixed a number of compiling issues
the commit fixed a number of compiling issues when some epxeriments
are turned on at the same time.

Change-Id: Idb15b215e2d2a7d25f2707f99ef55a34e7301ce7
2012-05-25 10:55:47 -07:00