Commit Graph

92 Commits

Author SHA1 Message Date
John Koleszar
109b69a706 RTCD: add arnr functions
This commit continues the process of converting to the new RTCD
system. It removes the last of the VP8_ENCODER_RTCD struct references.

Change-Id: I2a44f52d7cccf5177e1ca98a028ead570d045395
2012-01-30 12:10:48 -08:00
John Koleszar
83a91e789c RTCD: add variance functions
This commit continues the process of converting to the new RTCD
system.

Change-Id: Ie5c1aa480637e98dc3918fb562ff45c37a66c538
2012-01-30 12:08:30 -08:00
John Koleszar
fdb61a4531 RTCD: add recon functions
This commit continues the process of converting to the new RTCD
system.

Change-Id: I9bfcf9bef65c3d4ba0fb9a3e1532bad1463a10d6
2012-01-30 12:08:28 -08:00
John Koleszar
f56918ba9c Remove legacy integer types
Remove BOOL, INTn, UINTn, etc, in favor of C99-style fixed width
types.

Change-Id: I396636212fb5edd6b347d43cc940186d8cd1e7b5
2011-12-22 09:58:40 -08:00
John Koleszar
26c6a44c66 Avoid heap allocation of firstpass stats
The total_stats, this_frame_stats, and total_left_stats structures
were previously create by a heap allocation, despite being of fixed
size. These structures were allocated and deallocated during
{de,}allocate_compressor_data, which is reinvoked whenever the frame
size changes. Unfortunately, this clobbers the total_stats and
total_left_stats data.

Historically, these were variable size at one time, due to the first
pass motion map, which necessitated their being created by a unique
heap allocation. However, this bug with the total_stats being
clobbered has probably been present since that initial implementation.

These structures are instead moved to be stored within the struct
twopass_rc directly, rather than being heap allocated separately.

Change-Id: I7f9e519e25c58b92969071f0e99fa80307e0682b
2011-12-16 11:40:23 -08:00
John Koleszar
bdd35c13cc avoid resetting framerate during vpx_codec_enc_config_set()
The calculated frame_rate is a state variable in the codec, and
shouldn't be maintained in the configuration struct. Move it to the
main part of cpi so that it isn't clobbered when the configuration
struct is updated. The initial framerate estimate is moved from the
vp8_cx_iface.c wrapper into the body of init_config() in onyx_if.c, so
that it is only called once and not reset on every call to
vp8_change_config().

Change-Id: I8d9a3d1283330d1ee297d07e9d78d1f2875f2465
2011-11-11 14:45:58 -08:00
John Koleszar
86712c50f2 Fix uninitialized new_mv_count in first pass file
Uninitialized data could be written to the first pass file when no
motion vectors are present in the frame.

Also fix a number of compiler warnings.

Change-Id: Icc9f53b6d33da9de4563d86d9fd591910473ea90
2011-10-04 09:50:52 -04:00
Paul Wilkins
b6e27d5f0b CQ and two pass rate control.
Changes to the selection of Q limits for two pass
and two pass CQ mode.

Allowance made for Mode and motion vector costs.
Some refactoring of common code.

For Derf and YT sets CQ mode average improvement
circa 1% (SSIM and Global PSNR).

Some increased tendency to undershoot even when
user CQ not reached.

Patch2: Removed some test code accidentally merged.

Change-Id: Icf74d13af77437c08602571dc7a97e747cce5066
2011-09-30 10:55:52 +01:00
Fritz Koenig
4797a97215 Quiet warning by removing unused variable.
fwd_boost_score was not being computed or
referenced, so remove declaration.

Change-Id: Iece36cde1ec113e3c6afaff1407d24cdf12bd0a8
2011-08-24 15:47:09 -07:00
Paul Wilkins
744f482350 Small boost to every other frame.
Instead of a single mid GF boost apply a few extra bits to
every other frame. This gives a very small average metrics
improvement on both derf and YT sets.

Also use min GF interval as min KF interval.

Change-Id: Iee238b8cae0ffaed850a5a944ac825cee18da485
2011-08-17 14:14:23 +01:00
John Koleszar
e96131705a Revert "Improved 1-pass CBR rate control"
This reverts commit b5ea2fbc2c. Further
testing showed noticable keyframe popping in some cases, reverting this
for now to give time for a proper fix.

Conflicts:

	vp8/encoder/onyx_if.c
	vp8/encoder/ratectrl.c

Change-Id: I159f53d1bf0e24c035754ab3ded8ccfd58fd04af
2011-08-12 14:51:36 -04:00
James Zern
b45065d38b cosmetics: consistently use [u]int64_t
Removes mixed usage of (unsigned) long long and INT64.
Fixes Issue #208.

Change-Id: I220d3ed5ce4bb1280cd38bb3715f208ce23cf83a
2011-07-26 11:34:36 -07:00
John Koleszar
b5ea2fbc2c Improved 1-pass CBR rate control
This patch attempts to improve the handling of CBR streams with
respect to the short term buffering requirements. The "buffer level"
is changed to be an average over the rc buffer, rather than a long
running average. Overshoot is also tracked over the same interval
and the golden frame targets suppressed accordingly to correct for
overly aggressive boosting.

Testing shows that this is fairly consistently positive in one
metric or another -- some clips that show significant decreases
in quality have better buffering characteristics, others show
improvenents in both.

Change-Id: I924c89aa9bdb210271f2e03311e63de3f1f8f920
2011-07-18 11:48:05 -04:00
Yunqing Wang
40991faeae Adjust full-pixel clamping and motion vector limit calculation
Do mvp clamping in full-pixel precision instead of 1/8-pixel
precision to avoid error caused by right shifting operation.
Also, further fixed the motion vector limit calculation in change:
b748045470

Change-Id: Ied88a4f7ddfb0476eb9f7afc6ceeddbf209fffd7
2011-07-08 11:34:28 -04:00
Yunqing Wang
ae8aa836d5 Merge "Copy macroblock data to a buffer before encoding it" 2011-06-30 11:14:24 -07:00
Paul Wilkins
11694aab66 Change to arf boost calculation.
In this commit I have added an experimental function
that tests prediction quality either side of a central position
to calculate a suggested boost number for an ARF frame.

The function is passed an offset from the current position and
a number of frames to search forwards and backwards.
It returns a forward, backward and compound boost number.

The new code can be deactivated using #define NEW_BOOST 0

In its current default state the code searches forwards and backwards
from the proposed  position of the next alt ref.

The the old code used a boost number calculated by scanning forward
from the previous GF up to the proposed alt ref frame position.

I have also added some code to try and prevent placement of a gf/arf
where there is a brief flash.

Change-Id: I98af789a5181148659f10dd5dd2ff2d4250cd51c
2011-06-29 18:01:25 +01:00
Yunqing Wang
0d87098e08 Copy macroblock data to a buffer before encoding it
I got this idea from Pascal (Thanks). Before encoding a macroblock,
copy it to a 16x16 buffer, and then read source data from there
instead. This will help keep the source data in cache, and help
with the performance.

Change-Id: Id05f4cb601299150511d59dcba0ae62c49b5b757
2011-06-23 13:54:02 -04:00
John Koleszar
db67dcba6a Revert "Reduce overshoot in 1 pass rate control"
This reverts commit 212f618373.

Further testing shows that the overshoot accumulation/damping is too
aggressive on some clips. Allowing the accumulated overshoot to
decay and limiting to damping to golden frames shows some promise.
But some clips show significant overshoot in the buffer window, so
I think this still needs work.

Change-Id: Ic02a9ca34f55229f9cc04786f4fab54cdc1a3ef5
2011-06-23 11:52:12 -04:00
Ronald S. Bultje
87fd66bb0e Assign boost to GF bit allocation if past frame had no ARF.
Modify the second-pass code to provide a full golden-frame (GF) bit
allocation boost if the past GF group (GFG) had no alt-ref frame (ARF),
even if the current GFG does contain and ARF.

This mostly has no effect on clips, since switching ARFs on/off between
GFGs is not very common. Has a positive effect on e.g. cheer (+0.45 SSIM
at 600kbps) and football (+0.25 SSIM at 600kbps), particularly at high
bitrates. Has a negative effect (-0.04 SSIM at 300kbps) at pamphlet,
which appears only marginally related to this patch, and crew (-0.1 SSIM
at 700kbps).

Change-Id: I2e32899638b59f857e26efeac18a82e0c0b77089
2011-06-16 13:01:27 -04:00
John Koleszar
eb645abeac Merge "Disable specialcase for last frames if the sequence contains ARFs." 2011-06-16 09:56:05 -07:00
Ronald S. Bultje
299193dd1c Disable specialcase for last frames if the sequence contains ARFs.
firstpass.c contains some rate adjustment code that assures that the
last few frames in a sequence abide by rate limits. If the second-to-
last group of frames contains an alt-ref frame (ARF), the last golden
frame (GF) is zero bytes, and we will thus spend a ridiculously high
number of bits on regular P-frames trying to hit the target rate. This
does slightly enhance the quality of these last few frames, but has
no perceptual value (other than hitting the target rate).

Disabling this code means we consistently (slightly) undershoot the
target rate and consequently do worse on the last few frames of a
clip, which is particularly noticeable for small clips. The quality-
per-bitrate is generally better, ~0.2% better overall on derf-set,
especially on clips such as garden, tennis, foreman at low bitrates.
Has a negative effect on hallmonitor at high bitrates.

Change-Id: I1d63452fef5fee4a0ad2fb2e9af4c9f2e0d86d23
2011-06-15 09:47:00 -04:00
Tero Rintaluoma
9909047461 Fix RT only build
Moved encode_intra function from firstpass.c to encodeintra.c to
prevent linking problem in real-time only build. Also changed name
of the function to vp8_encode_intra because it is not a static.

Change-Id: Ibf3c6c1de3152567347e5fbef47d1d39564620a5
2011-06-14 13:39:06 +03:00
Paul Wilkins
4e81a68af7 Further activity masking changes:
Some further re-structuring of activity masking code.
Still has various experimental switches.
Supports a metric based on intra encode.
Experimental comparison against a fixed activity target  rather
than a frame average, for altering rd and zbin.

Overall the SSIM performance is similar  to TT's original
code but there is a much smaller PSNR hit of circa
0.5% instead of 3.2%

Change-Id: I0fd53b2dfb60620b3f74d7415e0b81c1ac58c39a
2011-06-08 16:03:37 +01:00
John Koleszar
5c166470a5 Merge "Reduce overshoot in 1 pass rate control" 2011-06-07 12:30:37 -07:00
John Koleszar
824e9410c6 Merge "Don't allow very short GF groups even when the GF is predicted from an ARF." 2011-06-06 07:02:29 -07:00
John Koleszar
212f618373 Reduce overshoot in 1 pass rate control
This patch attempts to reduce the peak bitrate hit by the encoder
when using small buffer windows.

Tested on the CIF set over 200-500kbps using these settings:

  --buf-sz=500 --buf-initial-sz=250 --buf-optimal-sz=250 \
  --undershoot-pct=100

Two pass encodes were tested at best quality. One pass encodes were
tested only at realtime speed 4:

  --rt --cpu-used=-4

The peak datarate (over the specified 500ms window) was measured
for each encode, and averaged together to get metric for
"average peak," computed as SUM(peak)/SUM(target). This patch
reduces the average peak datarate as follows:

  One pass:
    baseline:   1.29715
    this patch: 1.23664

  Two pass:
    baseline:   1.32702
    this patch: 1.37824

This change had a positive effect on our quality metrics as well:

  One pass CBR:
                    Min  / Mean / Max (pct)
    Average PSNR    -0.42 / 2.86 / 27.32
    Overall PSNR    -0.90 / 2.00 / 17.27
    SSIM            -0.05 / 3.95 / 37.46

  Two pass CBR:
                    Min  / Mean / Max (pct)
    Average PSNR    -4.47 / 4.35 / 35.99
    Overall PSNR    -3.40 / 4.18 / 36.46
    SSIM            -4.56 / 6.98 / 53.67

  One pass VBR:
                    Min  / Mean / Max (pct)
    Average PSNR    -5.21 /  0.01 / 3.30
    Overall PSNR    -8.10 / -0.38 / 1.21
    SSIM            -7.38 / -0.11 / 3.17
    (note: most values here were close to the mean, there were a few
     outliers on files that were very sensitive to golden frame size)

  Two pass VBR:
                    Min  / Mean / Max (pct)
    Average PSNR    0.00 / 0.00 / 0.00
    Overall PSNR    0.00 / 0.00 / 0.00
    SSIM            0.00 / 0.00 / 0.00

Neither one pass or two pass CBR mode adheres particularly strictly
to the short term buffer constraints, and two pass is less
consistent, even in the baseline commit. This should be addressed
in a later commit. This likely will hurt the quality numbers, as it
will have to reduce the burstiness of golden frames.

Aside: My work on this commit makes it clear that we need to make
rate control modes "pluggable", where you can easily write a new
one or work on one in isolation.

Change-Id: I1ea9a48f2beedd59891f1288aabf7064956b4716
2011-06-03 16:38:11 -04:00
Scott LaVarnway
8c5b73de2a Merge "Removed B_MODE_INFO" 2011-06-03 08:32:30 -07:00
Scott LaVarnway
773768ae27 Removed B_MODE_INFO
Declared the bmi in BLOCKD as a union instead of B_MODE_INFO.
Then removed B_MODE_INFO completely.

Change-Id: Ieb7469899e265892c66f7aeac87b7f2bf38e7a67
2011-06-02 13:46:41 -04:00
Ronald S. Bultje
9f002bee53 Don't allow very short GF groups even when the GF is predicted from an ARF.
This is basically a slightly modified version of the previous patch,
and it has a moderately positive effect (SSIM/PSNR both +0.08% avg
on derf-set). Most clips show no change, except waterfall/coastguard,
each ~ +0.8% SSIM/PSNR. You can see similar effects in other clips
by shortening their length to terminate at a very short last group
of frames.

Change-Id: I7a70de99ca1f9fe6a8b6ca7a6e30e8a4b64383e4
2011-06-02 09:14:51 -07:00
Yaowu Xu
5b2fb32961 further clean up of errorperbit and sadperbit
this commit makes the usage errorperbit and sadperbit consistent for
encoding modes and passes. Removed all different magic weight factors
associated with errorperbit. Now 1/2 is used for both sadperbit16 and
sadperbit4, the /2 operation is merged into initializations of the 2
variables.

Tests on cif set show .23%, 0.18% and 0.19% gain by avg psnr, overall
psnr and ssim respectively.

Change-Id: Ifa285c3e065ce0a5a77addfc9f95aabf54ee270d
2011-06-01 14:44:06 -07:00
John Koleszar
8196cc85f8 Merge "cleanup: collect twopass variables" 2011-05-20 11:20:44 -07:00
Yaowu Xu
1fbc81a970 Merge "revise two function definitions with less parameters" 2011-05-20 07:45:42 -07:00
Yaowu Xu
a4c69e9a0f revise two function definitions with less parameters
Change-Id: Ia96e5bf915e4d3c0ac9c1795114bd9e5dd07327a
2011-05-19 19:06:03 -07:00
John Koleszar
63cb1a7ce0 cleanup: collect twopass variables
This patch collects the twopass specific memebers of VP8_COMP into a
dedicated struct. This is a first step towards isolating the two pass
rate control and aids readability by decorating these variables with
the 'twopass.' namespace. This makes it clear to the reader in what
contexts the variable will be valid, and is a hint that a section of
code might be a good candidate to move to firstpass.c in later
refactoring. There likely will be other rate control modes that need
their own specific data as well.

This notation is probably overly verbose in firstpass.c, so an
alternative would be to access this struct through a pointer like
'rc->' instead of 'cpi->firstpass.' in that file. Feel free to make
a review comment to that effect if you prefer.

Change-Id: I0ab8254647cb4b493a77c16b5d236d0d4a94ca4d
2011-05-19 17:26:09 -04:00
John Koleszar
048497720c Remove unused members of VP8_COMP
Various members that were either completely unreferenced or written
and not read.

Change-Id: Ie41ebac0ff0364a76f287586e4fe09a68907806e
2011-05-19 15:49:09 -04:00
Scott LaVarnway
6b25501bf1 Using int_mv instead of MV
The compiler produces better assembly when using int_mv
for assignments.  The compiler shifts and ors the two 16bit
values when assigning MV.

Change-Id: I52ce4bc2bfbfaf3f1151204b2f21e1e0654f960f
2011-05-12 11:08:16 -04:00
John Koleszar
88841f1059 Refactor lookahead ring buffer
This patch cleans up the source buffer storage and copy mechanism to
allow access through a standard push/pop/peek interface. This approach
also avoids an extra copy in the case where the source is not a
multiple of 16, fixing issue #102.

Change-Id: I05808c39f5743625cb4c7af54cc841b9b10fdbd9
2011-04-13 14:26:45 -04:00
Scott LaVarnway
f212a98ee7 Fixed unused variable warnings for firstpass.c
Change-Id: I8378a9a541ade2f098359a7b20fa08e6c1596d80
2011-04-04 14:18:31 -04:00
Yunqing Wang
3d6815817c Use full-pixel MV in mvsadcost calculation
MV sad cost error is only used in full-pixel motion search,
which only need full-pixel resolution instead of quarter-pixel
resolution. This change reduced mvsadcost table size, and
removed unneccessary pamameter passing since this table is
constant once it is generated.

Change-Id: I9f931e55f6abc3c99011321f1dfb2f3562e6f6b0
2011-04-01 16:41:58 -04:00
John Koleszar
429dc676b1 Increase static linkage, remove unused functions
A large number of functions were defined with external linkage, even
though they were only used from within one file. This patch changes
their linkage to static and removes the vp8_ prefix from their names,
which should make it more obvious to the reader that the function is
contained within the current translation unit. Functions that were
not referenced were removed.

These symbols were identified by:

  $ nm -A libvpx.a | sort -k3 | uniq -c -f2 | grep ' [A-Z] ' \
    | sort | grep '^ *1 '

Change-Id: I59609f58ab65312012c047036ae1e0634f795779
2011-03-17 20:53:47 -04:00
John Koleszar
e34e417d94 Merge "Fix incorrect macroblock counts in twopass rate control" 2011-03-11 06:06:04 -08:00
Adrian Grange
6daacdb785 Added missing format specifier in print statement
Printout of firstpass stats for frame had one fewer
format specifiers than arguments.

Change-Id: I5a42c85aa79c471e1a70afd75e24a91546b7a1cd
2011-03-10 12:43:49 -08:00
Adrian Grange
ed40ff9e2d Removed firstpass motion map
The firstpass motion map consists of an 8-bit flag for
each MB indicating how strongly the firstpass code
believes it should be filtered during the second pass
ARNR filtering.

For long or large format material the motion map can
become extremely large and hamper the operation of
the encoding process.

This change removes the motion map altogether, leaving
the second pass to rely on the magnitude of the motion
compensated error to determine the filter weight to
use for the MB during ARNR filtering.

Tests on the derf set indicate that the effect of this
change is neutral, with some small wins and losses. The
motion map has therefore been removed based on
a cost/benefit evaluation.

Change-Id: I53e07d236f5ce09a6f0c54e7c4ffbb490fb870f6
2011-03-10 11:32:48 -08:00
James Berry
f3e9e2a0f8 Fix incorrect macroblock counts in twopass rate control
The previous calculation of macroblock count (w*h)/256
is not correct when the width/height are not multiples of
16. Use the precalculated macroblock count from
cpi->common instead. This manifested itself as a divide
by zero when the number of pixels was less than 256.
num_mbs updated in estimate_max_q, estimate_q,
 estimate_kf_group_q, and estimate_cq

Change-Id: I92ff98587864c801b1ee5485cfead964673a9973
2011-03-10 13:33:06 -05:00
Ralph Giles
e6948bf0f9 Fix a multi-line format-string warning.
GCC 4.5 and 4.6 both issue a warning about the multi-line format
string introduced in bc9c30a0, which also changed the whitespace
in the associated stt file by line-wrapping the long format string.

Instead, use multiple string constants, which the compiler will
concatenate. This maintains the original formatting, but remains
legible within the standard line length.

Change-Id: I27c9f92d46be82d408105a3a4091f145f677e00e
2011-03-08 07:14:12 -08:00
Paul Wilkins
de87c420ef Corrected minor typos.
Change-Id: Icc9f12bd1e1bdaf51256dc8a90d08aa9be89ef34
2011-03-08 14:46:22 +00:00
Paul Wilkins
bc9c30a003 Improved key frame detection.
In some cases where clips have been encoded with
borders (eg. some wide-screen content where there is a
border top and bottom and slide shows containing portrait
format photographs (border left and right)) key frames were
not being correctly detected.

The new code looks to measure cases where a portion of
the image can be coded equally easily using intra or inter
modes and where the resulting error score is also very low.
These "neutral" areas are then discounted in the key frame
detection code.

Change-Id: I00c3e8230772b8213cdc08020e1990cf83b780d8
2011-03-07 15:58:07 +00:00
Paul Wilkins
9fc8cb39aa Improved KF insertion after fades to still.
This code extends what was previously done for GFs, to pick
cases where insertion of a key frame after a fade (or other
transition or complex motion)  followed by a still section, will
be beneficial and will reduce the number of forced key frames.

Change-Id: If8bca00457f0d5f83dc3318a587f61c17d90f135
2011-03-07 15:11:09 +00:00
John Koleszar
36be4f7f06 Fix drastic undershoot in long form content
When the modified_error_left accumulator exceeds INT_MAX, an incorrect
cast to int resulted in a negative value, causing the rate control to
allocate no bits to that keyframe group, leading to severe undershoot
and subsequent poor quality.

This error was exposed by the recent change to the rolling target and
actual spend accumulators in commit 305be4e4 which fixed them to
actually calculate the average value rather than be re-initialized
on every frame to the average per-frame bitrate. When this bug was
triggered, the target bitrate could be 0, so the rolling target
becomes small, which causes the undershoot. The code prior to 305be4e4
did not exhibit this behavior because the rolling target was always
set to a reasonable value and was independent of the actual target
bitrate. With this patch, the actual target bitrate is calculated
correctly, and the rate control tracks as expected.

This cast was likely added to silence a compiler warning on a comparison
between a double (modified_error_left) and an int (0). Instead, this
patch removes the cast and changes the comparison to be against 0.0,
which should prevent the warning from reoccuring.

This fixes issue #289. Special thanks to gnafu for his efforts in
reporting and debugging this fix.

Change-Id: Ie5cc1a7b516c578a76c3a50c892a6f04a11621fe
2011-03-02 22:52:27 -05:00
Paul Wilkins
b862c108dd Overflow of frame error accumulators.
This fixes an overflow problem in the frame error accumulators.

The overflow condition is extreme but did trigger when Frank B.
coded some high motion interlaced HD content.

The observed effect was a catastrophic  breakdown of the rate
control leading to massive undershoot and poor bit allocation.

All the error values should really be unsigned but I will look at this
separately.

Change-Id: I9745f5c5ca2783620426b66b568b2088b579151f
2011-02-24 15:49:41 +00:00