Doing that doesn't make sense, because the only purpose of sbr_dequant
is to process the data from read_sbr_data.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If videotoolbox_common_end_frame failed, then the AVFrame was returned
to the API user with the dummy buffer (in AVFrame.buf[0]) still set, and
the decode call indicating success.
These "half-set" AVFrames with dummy buffer are a videotoolbox specific
hack, because the decoder requires an allocated AVFrame for its internal
logic. Videotoolbox on the other hand allocates its frame itself
internally, and outputs it only on end_frame. At this point, the dummy
buffer is replaced with the real frame (unless decoding fails).
Currently, multiple slices with just one thread produce corrupted
output.
Additionally, enable slice structured mode for h263(+)
Bug-Id: 912
CC: libav-stabl@libav.org
The replaced check should have become redundant
Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
It is used as size argument of ff_canopus_parse_info_tag, which uses it
as size argument to bytestream2_init, which only supports sizes up to
INT_MAX.
Changing it's type to unsigned simplifies the check.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This allows removing a special case for the fixed point decoder and will
make error checks simpler
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
In case of bitstream errors the deblock filter and slices can access uninitialized
top_borders from previous slices which did not fill them as they stoped halfway due
to error or where entirely missing.
This also makes code using these tables deterministic in case of missing or damaged
slices
Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This side data type is meant to be added to AVStream side data.
A fallback track indicates an alternate track to use when the
current track can not be decoded for some reason. e.g. no
decoder available for codec.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
FFv1 uses two types of coders, golomb and range with two different
tables. This is exposed this in a rather convoluted way, for example
mentioning to set coder type 1 while initializing the variable 'ac' to 2,
because encoder does not use range coder with default table.
Appropriate internal coder type values have been added and used in any
check rather than using raw numbers.
Initialization of avctx.coder_type in ffv1dec is removed because this
field is encoder only. An unneeded validation check in the encoder
is dropped too.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
It is used as size argument of ff_canopus_parse_info_tag, which uses it
as size argument to bytestream2_init, which only supports sizes up to
INT_MAX.
Changing it's type to unsigned simplifies the check.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes potential integer overflows
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi
This fix is choosen to be simple to backport, better solution
for master is planed
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes assertion failure
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi
This fix is choosen to be simple to backport, better solution
for master is planed
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This guarantees a "best effort precision".
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Fixes out of array access
Fixes: ce19e41f0ef1e52a23edc488faecdb58/asan_heap-oob_2504e97_4202_ffa0df1baed14022b9bfd4f8ac23d0cb.smk
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Otherwise the h263p decoder can try to copy overread bytes, even though
buffer is NULL.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes out of array read
Fixes: cb3f38b08b4541523974667c7d1eee9e/asan_heap-oob_2659e18_9838_021fd5cd635bf76cede6398cd9ecbcdd.tak
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: c50c4aa6cefda71b19a31ea12302980c/asan_heap-oob_12be5fd_7011_33ebd015a74976215934add72b9c8352.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The height of tables can be rounded up for MBAFF but this does not imply that is also true
for the previous frames
Fixes out of array reads
Fixes: c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Having both is not valid and can cause a NULL pointer dereference of
frame->data[1] later.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes out of array read
Fixes: 7fcd09eadd046e326d8ea0af66f166c8/asan_heap-oob_4a52e5_2273_fa6078a10dd575df266fb1e0b4114cd5.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
M_SQRT2 is defined in math.h, or in avutil/mathematics.h for
compatibility hacks. This uses this value instead of a floating literal.
Fixed point values produced by C_FIX(), R_FIX() remain identical.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
M_SQRT2 is defined in math.h, or in avutil/mathematics.h for
compatibility hacks. This uses this value instead of an ad-hoc define.
Fixed point values produced by FIX() remain identical.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
M_PI is defined in math.h, or in avutil/mathematics.h for compatibility
hacks. This uses this value instead of an ad-hoc floating literal.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
M_E is defined in math.h, or in avutil/mathematics.h for compatibility
hacks. This uses this value instead of an ad-hoc define.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Fixes out of array read
Fixes: 61cf123c081ee2bb774d307c75bdb99e/asan_heap-oob_1224f76_5546_bee833ffae73f752b489b9eeaac52db7.dpx
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 3c857d4d90365731524716e6d051e43a/signal_sigsegv_7f4f59bcc29e_1386_20abd2c8e655cb9c75b24368e65fe3b1.flv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 20dd01398dee0f6d83d7e5410a2ae8eb/signal_sigsegv_39eeb1f_4001_62efbdf1c60748dabf1ec310b59525fd.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Having both is not valid and can cause a NULL pointer dereference of
frame->data[1] later.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Just realized my previous patch doesn't work quite right. I uploaded a better
sample file that actually has visible captions to /incoming/hevc_cc.ts. I
tested with that file doing hevc->x264 and it works.
This is basically an exact copy of the existing h264 logic.
Signed-off-by: Will Kelleher <wkelleher@gogoair.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
A too small buffer will cause segfaults somewhere below
decompress_texture_thread.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If it is too small av_image_copy_plane segfaults.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This prevents various values from getting an insanely huge exponent.
If someone knows a cleaner solution, thats welcome!
This is similar to commit 8978c74 for aacsbr.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If it is negative, it causes segmentation faults in decode_rle.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
this fixes the return code of avcodec_decode_video2 for gif decoding
and the gif frame data buffer is skipped properly
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit '60f50374f1955442dc987abc4a6c61c2109620c2':
rpza: Check the blocks left before processing one
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
Fixes apparent mmal_port_disable() freezes in ffmmal_stop_decoder() when
calling ffmmal_decode() with flush semantics a large number of times in
a row.
This is similar to commit ec38a1b for aac_decode_frame_int.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
This fixes a SIGFPE crash in the aac_fixed decoder.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
The assert in ffmmal_stop_decoder() could trigger sometimes. The
packets_buffered counter was indeed not correctly maintained, and
packets were not subtracted from it if they were still in the waiting
queue.
For some reason, this happened especially with VC-1.
The buffer needs s->bpp bytes, at maximum currently 10.
Assert that s->bpp is not larger.
This fixes a stack buffer overflow.
Reviewed-by: wm4 <nfxjfg@googlemail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes out of array access
Fixes: b877a6b788a25c70e8b1d014f8628549/asan_heap-oob_1da2c3f_2324_5a1b329b0b3c4bb6b1d775660ac56717.r3d
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: a9502b60f4cecc19475382aee255f73c/asan_heap-oob_1e87fba_2548_a8ad47f6dde36644fe9cdc444d4632d0.sub
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
