Fixes out of array access
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2049_f2192b6829ab6e0eefcb035329c03c60.264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
SGI RLE encoding is slighlty different than the one provided by rle
module (especially at high bit depth). The pixel count function however
does not change, so it is simply made library-public.
This is never mentioned in the specifications, and decoders work
just as fine without it. Update the fate references since the compressed
file is smaller.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
There is no such thing as a slice structured mode in the original version 1 H.263,
that mode was added in H.263+ in 1998. Also the headers for slice structured mode
are not part of the older version 1 and this would result in unplayable files
An alternative to this patch would be to merge the H263 and H263P AVCodecs and use
other means to distinguish the older and newer versions.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
VP8E_UPD_ENTROPY, VP8E_UPD_REFERENCE, VP8E_USE_REFERENCE were removed
from libvpx and the remaining values were never used here
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Zern <jzern@google.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
The auxiliary data length field is not reliable,
and incorrect overread errors could be returned
for valid, real-world bitstreams.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This actually fixes an incorrect float literal. It is believed by
examining the precision that the literals were all pre-computed as
floats, resulting in this needless loss of precision. There is no
benefit to keeping such reduced precision:
1. These constants are used for static array computation, hence
compile-time.
2. They will be treated as doubles anyway, since f specifier was not
present.
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
This uses M_SQRT1_2, M_SQRT2 instead of the actual literals. This yields
greater precision in some places in avcodec/ac3, while fixed point
values remain unchanged.
Reviewed-by: Clément Bœsch <u@pkh.me>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
* commit '4d8f536b535487063a08609636e712ad86d2ad54':
qsvenc: print the actual video parameters used by MSDK
Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
* commit 'f6c94457b44f41d900cd0991857f54e1f0ccedd6':
mpegvideo_enc: enable rtp_mode when multiple slices are used
Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
* commit '79ae1e630b476889c251fc905687a3831b43ab5e':
avcodec: Define side data type for fallback track
Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Doing that doesn't make sense, because the only purpose of sbr_dequant
is to process the data from read_sbr_data.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This function returns the encoded data of a frame, one slice at a time
directly when that slice is encoded, instead of waiting for the full
frame to be done. However this field has a debatable usefulness, since
it looks like it is just a convoluted way to get data at lowest
possible latency, or a somewhat hacky way to store h263 in RFC-2190
rtp encapsulation.
Moreover when multi-threading is enabled (which is by default) the order
of returned slices is not deterministic at all, making the use of this
function not reliable at all (or at the very least, more complicated
than it should be).
So, for the reasons stated above, and being used by only a single encoder
family (mpegvideo), this field is deemed unnecessary, overcomplicated,
and not really belonging to libavcodec. Libavformat features a complete
implementation of RFC-2190, for any other case.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
If videotoolbox_common_end_frame failed, then the AVFrame was returned
to the API user with the dummy buffer (in AVFrame.buf[0]) still set, and
the decode call indicating success.
These "half-set" AVFrames with dummy buffer are a videotoolbox specific
hack, because the decoder requires an allocated AVFrame for its internal
logic. Videotoolbox on the other hand allocates its frame itself
internally, and outputs it only on end_frame. At this point, the dummy
buffer is replaced with the real frame (unless decoding fails).
Currently, multiple slices with just one thread produce corrupted
output.
Additionally, enable slice structured mode for h263(+)
Bug-Id: 912
CC: libav-stabl@libav.org
The replaced check should have become redundant
Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
It is used as size argument of ff_canopus_parse_info_tag, which uses it
as size argument to bytestream2_init, which only supports sizes up to
INT_MAX.
Changing it's type to unsigned simplifies the check.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This allows removing a special case for the fixed point decoder and will
make error checks simpler
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
In case of bitstream errors the deblock filter and slices can access uninitialized
top_borders from previous slices which did not fill them as they stoped halfway due
to error or where entirely missing.
This also makes code using these tables deterministic in case of missing or damaged
slices
Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This side data type is meant to be added to AVStream side data.
A fallback track indicates an alternate track to use when the
current track can not be decoded for some reason. e.g. no
decoder available for codec.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
FFv1 uses two types of coders, golomb and range with two different
tables. This is exposed this in a rather convoluted way, for example
mentioning to set coder type 1 while initializing the variable 'ac' to 2,
because encoder does not use range coder with default table.
Appropriate internal coder type values have been added and used in any
check rather than using raw numbers.
Initialization of avctx.coder_type in ffv1dec is removed because this
field is encoder only. An unneeded validation check in the encoder
is dropped too.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
It is used as size argument of ff_canopus_parse_info_tag, which uses it
as size argument to bytestream2_init, which only supports sizes up to
INT_MAX.
Changing it's type to unsigned simplifies the check.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes potential integer overflows
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi
This fix is choosen to be simple to backport, better solution
for master is planed
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes assertion failure
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi
This fix is choosen to be simple to backport, better solution
for master is planed
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This guarantees a "best effort precision".
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Fixes out of array access
Fixes: ce19e41f0ef1e52a23edc488faecdb58/asan_heap-oob_2504e97_4202_ffa0df1baed14022b9bfd4f8ac23d0cb.smk
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Otherwise the h263p decoder can try to copy overread bytes, even though
buffer is NULL.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes out of array read
Fixes: cb3f38b08b4541523974667c7d1eee9e/asan_heap-oob_2659e18_9838_021fd5cd635bf76cede6398cd9ecbcdd.tak
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: c50c4aa6cefda71b19a31ea12302980c/asan_heap-oob_12be5fd_7011_33ebd015a74976215934add72b9c8352.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The height of tables can be rounded up for MBAFF but this does not imply that is also true
for the previous frames
Fixes out of array reads
Fixes: c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Having both is not valid and can cause a NULL pointer dereference of
frame->data[1] later.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes out of array read
Fixes: 7fcd09eadd046e326d8ea0af66f166c8/asan_heap-oob_4a52e5_2273_fa6078a10dd575df266fb1e0b4114cd5.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
M_SQRT2 is defined in math.h, or in avutil/mathematics.h for
compatibility hacks. This uses this value instead of a floating literal.
Fixed point values produced by C_FIX(), R_FIX() remain identical.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
M_SQRT2 is defined in math.h, or in avutil/mathematics.h for
compatibility hacks. This uses this value instead of an ad-hoc define.
Fixed point values produced by FIX() remain identical.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
M_PI is defined in math.h, or in avutil/mathematics.h for compatibility
hacks. This uses this value instead of an ad-hoc floating literal.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
M_E is defined in math.h, or in avutil/mathematics.h for compatibility
hacks. This uses this value instead of an ad-hoc define.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Fixes out of array read
Fixes: 61cf123c081ee2bb774d307c75bdb99e/asan_heap-oob_1224f76_5546_bee833ffae73f752b489b9eeaac52db7.dpx
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 3c857d4d90365731524716e6d051e43a/signal_sigsegv_7f4f59bcc29e_1386_20abd2c8e655cb9c75b24368e65fe3b1.flv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 20dd01398dee0f6d83d7e5410a2ae8eb/signal_sigsegv_39eeb1f_4001_62efbdf1c60748dabf1ec310b59525fd.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Having both is not valid and can cause a NULL pointer dereference of
frame->data[1] later.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Just realized my previous patch doesn't work quite right. I uploaded a better
sample file that actually has visible captions to /incoming/hevc_cc.ts. I
tested with that file doing hevc->x264 and it works.
This is basically an exact copy of the existing h264 logic.
Signed-off-by: Will Kelleher <wkelleher@gogoair.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
A too small buffer will cause segfaults somewhere below
decompress_texture_thread.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If it is too small av_image_copy_plane segfaults.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This prevents various values from getting an insanely huge exponent.
If someone knows a cleaner solution, thats welcome!
This is similar to commit 8978c74 for aacsbr.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If it is negative, it causes segmentation faults in decode_rle.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
this fixes the return code of avcodec_decode_video2 for gif decoding
and the gif frame data buffer is skipped properly
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit '60f50374f1955442dc987abc4a6c61c2109620c2':
rpza: Check the blocks left before processing one
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
Fixes apparent mmal_port_disable() freezes in ffmmal_stop_decoder() when
calling ffmmal_decode() with flush semantics a large number of times in
a row.
This is similar to commit ec38a1b for aac_decode_frame_int.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
This fixes a SIGFPE crash in the aac_fixed decoder.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
The assert in ffmmal_stop_decoder() could trigger sometimes. The
packets_buffered counter was indeed not correctly maintained, and
packets were not subtracted from it if they were still in the waiting
queue.
For some reason, this happened especially with VC-1.
The buffer needs s->bpp bytes, at maximum currently 10.
Assert that s->bpp is not larger.
This fixes a stack buffer overflow.
Reviewed-by: wm4 <nfxjfg@googlemail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes out of array access
Fixes: b877a6b788a25c70e8b1d014f8628549/asan_heap-oob_1da2c3f_2324_5a1b329b0b3c4bb6b1d775660ac56717.r3d
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: a9502b60f4cecc19475382aee255f73c/asan_heap-oob_1e87fba_2548_a8ad47f6dde36644fe9cdc444d4632d0.sub
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes libavcodec/vdpau.c:282:5: warning:
"CONFIG_H263_VDPAU_HWACCEL" is not defined [-Wundef]
Removed in d35d0c723e.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access
Fixes: 54e488b9da4abbceaf405d6492515697/asan_heap-oob_32769b0_160_a8755eb08ee8f9579348501945a33955.TIF
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array access
Fixes: 24d05e8b84676799c735c9e27d97895e/asan_heap-oob_1b70f6a_2955_7c3652a7f370f9f3ef40642bc2c99bb2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: d92114d8c2a019b8a6e50cd2a7301b54/asan_heap-oob_26bf563_60_1d3420277533de9dbf8aba3f93af346f.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 04715144ba237443010554be0d05343f/asan_heap-oob_1eafc76_1737_c685b48041a563461839e4e7ab97abb8.jpg
Fixes out of array access
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
When sbr->reset is set in encode_frame, a bunch of qsort calls might get made.
Thus, there is the potential of calling qsort whenever the spectral
contents change.
AV_QSORT is substantially faster due to the inlining of the comparison callback.
Thus, the increase in performance should be worth the increase in binary size.
Tested with FATE.
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Fixes: libavcodec/rawenc.c:64:40: warning: passing argument 3 of av_image_copy_to_buffer from incompatible pointer type [enabled by default]
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: libavcodec/vdpau.c:320:5: warning: "CONFIG_H263_VDPAU_HWACCEL" is not defined [-Wundef]
It was removed in d15adeacf3
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit '8487987b87a252963b27b6adfd82887ea5933918':
h264: Print user data SEI under normal debug verbosity
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
* commit '1720791e36f9cc24c05efea5bb275ab52156ce50':
png: read and write stereo3d frame side data information
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
The size of decoding map can differ from one calculated
internally, producing artifacts while decoding video.
Signed-off-by: Paul B Mahol <onemda@gmail.com>
ISO C restricts enumerator values to the range of int. Thus (for instance) 0x80000000
unfortunately does not work, and throws a warning with -Wpedantic on
clang 3.7.
This fixes it by using alternative expressions that result in identical
values but do not have this issue.
Tested with FATE.
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Previously the message was cut off at 256th byte.
This will allow dumping the complete x264 encode info when needed.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Drop the need of setting -debug bugs since it's not a bug, and the
message is already under a AV_LOG_DEBUG log level. Instead only print
it when there is an actual string in it.
Use optional sTER chunk defining side-by-side stereo pair
within "Extensions to the PNG 1.2 Specification", version 1.3.0.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
This is likely more precise and conveys the intent better.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
This is likely more precise and conveys the intent better.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
This is likely more precise and conveys the intent better.
The expression has also been accordingly simplified.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
* commit '8edaf625f3c38c695c33745822182e94e17d6e1b':
libopenh264enc: Count and warn about the number of skipped frames
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
* commit '9e14a992409348630ea96521411127418a92e8a3':
libopenh264enc: Add an option for controlling the frame skipping
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
* commit 'a9a60106370f862e191dea58e748626da6a8fe97':
avpacket: Provide an alloc and a free function for the struct
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
This code is in a performance critical section. AV_QSORT is
substantially faster due to the inlining of the comparison callback.
Thus, the increase in performance is worth the increase in binary size.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
This allows enabling the frame skipping, which is required for the
encoder to properly hit the target bitrate.
Signed-off-by: Martin Storsjö <martin@martin.st>
When not all the opus stream have the same amount of decoded samples
process the least amount and store what is left from the other streams.
Bug-Id: 909
CC: libav-stable@libav.org
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Use optional sTER chunk defining side-by-side stereo pair
within "Extensions to the PNG 1.2 Specification", version 1.3.0.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The commit 932ff70 introducing this header mentions it should be public.
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
