Fixes out of array reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Fixes: out of array read
Fixes: 76c515fc3779d1b838667c61ea13ce92/asan_heap-oob_1fc0d07_8913_794a4629a264ebdb25b58d3a94ed1785.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The DC VLC table used is too small, fixing this requires a sample,
thus request a sample.
Some samples are said to work even though the table has the wrong size, thus
this is left enabled if the user enables experimental features.
Fixes: 2abd25478c62a675f335fac00b467023/asan_static-oob_10aff98_1227_8811480c6ef1e970a7977ceb7e5a9958.mxf
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Approved-by: kurosu
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
As noted in a comment, pe.min in the reference encoder
is centered around current pe. The bit reservoir algo
needs pe.min to be a local minimum, because it can only
account for local PE variations. If it's set to a global
minimum as was being done, bit reservoir logic doesn't
work as efficiently.
This patch tries to forget old minimums and converge to
a local minimum without losing the stability of the
previous solution. Listening tests until now suggest this
solves numerous RC issues.
* commit '7831fb90503142e32cc3c9be43bc3f9d342ded6b':
textureencdsp: cosmetics: Use normal static const for tables
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
* commit '4a0918cae6394e503b17c71f8f171b4a795eb849':
sgienc: Support encoding high bit depth images with RLE
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
* commit 'c12c085be7e86880924249e5cb3f898e45dee134':
dcadec: Do not check for overreads in auxiliary data
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
Fixes out of array read
Fixes: 59bb925e90201fa0f87f0a31945d43b5/asan_heap-oob_4a52e5_3388_66027f11e3d072f1e02401ecc6193361.jvt
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access
Fixes: 482d8f2fd17c9f532b586458a33f267c/asan_heap-oob_4a52b6_7417_1d08d477736d66cdadd833d146bb8bae.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access
Fixes: 08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
ff_aac_tableinit is a macro in the case of hardcoded tables, so wrap
that up in a function (similar to how the decoder template does it) and
use that as the argument for ff_thread_once().
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Fixes out of array access
Fixes: 01859c9a9ac6cd60a008274123275574/asan_heap-oob_1dff571_8250_50d3d1611e294c3519fd1fa82198b69b.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
AAC-Fixed decoder segfaulted. This commit makes the aac encoder
and decoder init the table twice in case of transcoding again.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Since the ff_aac_tableinit() can be called by both the encoder and
the decoder (in case of transcoding) this commit shares the AVOnce
variable to prevent this.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This speeds up aac_tablegen to a ludicruous degree (~97%), i.e to the point
where it can be argued that runtime initialization can always be done instead of
hard-coded tables. The only cost is essentially a trivial increase in
the stack size.
Even if one does not care about this, the patch also improves accuracy
as detailed below.
Performance:
Benchmark obtained by looping 10^4 times over ff_aac_tableinit.
Sample benchmark (x86-64, Haswell, GNU/Linux):
old:
1295292 decicycles in ff_aac_tableinit, 512 runs, 0 skips
1275981 decicycles in ff_aac_tableinit, 1024 runs, 0 skips
1272932 decicycles in ff_aac_tableinit, 2048 runs, 0 skips
1262164 decicycles in ff_aac_tableinit, 4096 runs, 0 skips
1256720 decicycles in ff_aac_tableinit, 8192 runs, 0 skips
new:
21112 decicycles in ff_aac_tableinit, 511 runs, 1 skips
21269 decicycles in ff_aac_tableinit, 1023 runs, 1 skips
21352 decicycles in ff_aac_tableinit, 2043 runs, 5 skips
21386 decicycles in ff_aac_tableinit, 4080 runs, 16 skips
21299 decicycles in ff_aac_tableinit, 8173 runs, 19 skips
Accuracy:
The previous code was resulting in needless loss of
accuracy due to the pow being called in succession. As an illustration
of this:
ff_aac_pow34sf_tab[3]
old : 0.000000000007598092294225
new : 0.000000000007598091426864
real: 0.000000000007598091778545
truncated to float
old : 0.000000000007598092294225
new : 0.000000000007598091426864
real: 0.000000000007598091426864
showing that the old value was not correctly rounded. This affects a
large number of elements of the array.
Patch tested with FATE.
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
This hugely reduces the echo which was introduced with the previous
commit (though likely because previously everything was broken).
Makes LTP actually worthwhile now.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Should fix issues with ppc, tested by bug reporter.
Reported-by: John Warburton <john@johnwarburton.net>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Copy pointers to AVPicture after memory has been allocated.
Fixes NULL pointers in AVPicture after a17a766190.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
In some conditions, where the first band was being zeroed
mainly, the wrong global gain scalefactor would be written
to the stream since it's always taken from the first band
regardless of whether it's been marked as zero or not.
So, always make sure it contians something useful.
When both M/S coding and PNS are enabled, scalefactors
and coding books would be mistakenly clobbered when setting
the M/S flag on PNS'd bands. The flag needs to be set to
signal the generation of correlated noise, but the scalefactors,
coefficients and the coding books need to be kept intact.
Fixes out of array access
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2049_f2192b6829ab6e0eefcb035329c03c60.264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
SGI RLE encoding is slighlty different than the one provided by rle
module (especially at high bit depth). The pixel count function however
does not change, so it is simply made library-public.
This is never mentioned in the specifications, and decoders work
just as fine without it. Update the fate references since the compressed
file is smaller.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
There is no such thing as a slice structured mode in the original version 1 H.263,
that mode was added in H.263+ in 1998. Also the headers for slice structured mode
are not part of the older version 1 and this would result in unplayable files
An alternative to this patch would be to merge the H263 and H263P AVCodecs and use
other means to distinguish the older and newer versions.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
VP8E_UPD_ENTROPY, VP8E_UPD_REFERENCE, VP8E_USE_REFERENCE were removed
from libvpx and the remaining values were never used here
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Zern <jzern@google.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
The auxiliary data length field is not reliable,
and incorrect overread errors could be returned
for valid, real-world bitstreams.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This actually fixes an incorrect float literal. It is believed by
examining the precision that the literals were all pre-computed as
floats, resulting in this needless loss of precision. There is no
benefit to keeping such reduced precision:
1. These constants are used for static array computation, hence
compile-time.
2. They will be treated as doubles anyway, since f specifier was not
present.
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
This uses M_SQRT1_2, M_SQRT2 instead of the actual literals. This yields
greater precision in some places in avcodec/ac3, while fixed point
values remain unchanged.
Reviewed-by: Clément Bœsch <u@pkh.me>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
* commit '4d8f536b535487063a08609636e712ad86d2ad54':
qsvenc: print the actual video parameters used by MSDK
Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
* commit 'f6c94457b44f41d900cd0991857f54e1f0ccedd6':
mpegvideo_enc: enable rtp_mode when multiple slices are used
Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
* commit '79ae1e630b476889c251fc905687a3831b43ab5e':
avcodec: Define side data type for fallback track
Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Doing that doesn't make sense, because the only purpose of sbr_dequant
is to process the data from read_sbr_data.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This function returns the encoded data of a frame, one slice at a time
directly when that slice is encoded, instead of waiting for the full
frame to be done. However this field has a debatable usefulness, since
it looks like it is just a convoluted way to get data at lowest
possible latency, or a somewhat hacky way to store h263 in RFC-2190
rtp encapsulation.
Moreover when multi-threading is enabled (which is by default) the order
of returned slices is not deterministic at all, making the use of this
function not reliable at all (or at the very least, more complicated
than it should be).
So, for the reasons stated above, and being used by only a single encoder
family (mpegvideo), this field is deemed unnecessary, overcomplicated,
and not really belonging to libavcodec. Libavformat features a complete
implementation of RFC-2190, for any other case.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
If videotoolbox_common_end_frame failed, then the AVFrame was returned
to the API user with the dummy buffer (in AVFrame.buf[0]) still set, and
the decode call indicating success.
These "half-set" AVFrames with dummy buffer are a videotoolbox specific
hack, because the decoder requires an allocated AVFrame for its internal
logic. Videotoolbox on the other hand allocates its frame itself
internally, and outputs it only on end_frame. At this point, the dummy
buffer is replaced with the real frame (unless decoding fails).
Currently, multiple slices with just one thread produce corrupted
output.
Additionally, enable slice structured mode for h263(+)
Bug-Id: 912
CC: libav-stabl@libav.org
The replaced check should have become redundant
Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
It is used as size argument of ff_canopus_parse_info_tag, which uses it
as size argument to bytestream2_init, which only supports sizes up to
INT_MAX.
Changing it's type to unsigned simplifies the check.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This allows removing a special case for the fixed point decoder and will
make error checks simpler
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
In case of bitstream errors the deblock filter and slices can access uninitialized
top_borders from previous slices which did not fill them as they stoped halfway due
to error or where entirely missing.
This also makes code using these tables deterministic in case of missing or damaged
slices
Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This side data type is meant to be added to AVStream side data.
A fallback track indicates an alternate track to use when the
current track can not be decoded for some reason. e.g. no
decoder available for codec.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
FFv1 uses two types of coders, golomb and range with two different
tables. This is exposed this in a rather convoluted way, for example
mentioning to set coder type 1 while initializing the variable 'ac' to 2,
because encoder does not use range coder with default table.
Appropriate internal coder type values have been added and used in any
check rather than using raw numbers.
Initialization of avctx.coder_type in ffv1dec is removed because this
field is encoder only. An unneeded validation check in the encoder
is dropped too.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
It is used as size argument of ff_canopus_parse_info_tag, which uses it
as size argument to bytestream2_init, which only supports sizes up to
INT_MAX.
Changing it's type to unsigned simplifies the check.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes potential integer overflows
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi
This fix is choosen to be simple to backport, better solution
for master is planed
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes assertion failure
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi
This fix is choosen to be simple to backport, better solution
for master is planed
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This guarantees a "best effort precision".
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Fixes out of array access
Fixes: ce19e41f0ef1e52a23edc488faecdb58/asan_heap-oob_2504e97_4202_ffa0df1baed14022b9bfd4f8ac23d0cb.smk
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Otherwise the h263p decoder can try to copy overread bytes, even though
buffer is NULL.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes out of array read
Fixes: cb3f38b08b4541523974667c7d1eee9e/asan_heap-oob_2659e18_9838_021fd5cd635bf76cede6398cd9ecbcdd.tak
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: c50c4aa6cefda71b19a31ea12302980c/asan_heap-oob_12be5fd_7011_33ebd015a74976215934add72b9c8352.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The height of tables can be rounded up for MBAFF but this does not imply that is also true
for the previous frames
Fixes out of array reads
Fixes: c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Having both is not valid and can cause a NULL pointer dereference of
frame->data[1] later.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes out of array read
Fixes: 7fcd09eadd046e326d8ea0af66f166c8/asan_heap-oob_4a52e5_2273_fa6078a10dd575df266fb1e0b4114cd5.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
M_SQRT2 is defined in math.h, or in avutil/mathematics.h for
compatibility hacks. This uses this value instead of a floating literal.
Fixed point values produced by C_FIX(), R_FIX() remain identical.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
M_SQRT2 is defined in math.h, or in avutil/mathematics.h for
compatibility hacks. This uses this value instead of an ad-hoc define.
Fixed point values produced by FIX() remain identical.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
M_PI is defined in math.h, or in avutil/mathematics.h for compatibility
hacks. This uses this value instead of an ad-hoc floating literal.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
M_E is defined in math.h, or in avutil/mathematics.h for compatibility
hacks. This uses this value instead of an ad-hoc define.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Fixes out of array read
Fixes: 61cf123c081ee2bb774d307c75bdb99e/asan_heap-oob_1224f76_5546_bee833ffae73f752b489b9eeaac52db7.dpx
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 3c857d4d90365731524716e6d051e43a/signal_sigsegv_7f4f59bcc29e_1386_20abd2c8e655cb9c75b24368e65fe3b1.flv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 20dd01398dee0f6d83d7e5410a2ae8eb/signal_sigsegv_39eeb1f_4001_62efbdf1c60748dabf1ec310b59525fd.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Having both is not valid and can cause a NULL pointer dereference of
frame->data[1] later.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Just realized my previous patch doesn't work quite right. I uploaded a better
sample file that actually has visible captions to /incoming/hevc_cc.ts. I
tested with that file doing hevc->x264 and it works.
This is basically an exact copy of the existing h264 logic.
Signed-off-by: Will Kelleher <wkelleher@gogoair.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
