s->decoded_buffer is allocated with a min_size of:
2 * FFALIGN(blockstodecode, 8) * sizeof(*s->decoded_buffer)
Then it is assigned to s->decoded[0] (and s->decoded_buffer + FFALIGN(blockstodecode, 8)
to s->decoded[1]) and passed as out buffer to decode_array_0000.
In this function 64 elements of the out buffer are written
unconditionally and outside the array if blockstodecode is too small.
This causes memory corruption, leading to segmentation faults or other
crashes.
Thus change decode_array_0000 to write at most blockstodecode elements
of the out buffer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 699341d647f7af785fb8ceed67604467b0b9ab12)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Otherwise range_start_decoding is not necessarily run and thus
ctx->rc.range still 0 in range_dec_normalize leading to an infinite
loop.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 464c49155ce7ffc88ed39eb2511e7a75565c24be)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If rice_limit is 0, k can be 0 in decode_scalar, which calls show_bits(gb, k).
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b657a1b1eedcf38bcf36e89a2f4be6f76b5ce09)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
For max_order = 0 the clipping range is invalid. (amin = 2, amax = 1)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 58d605ee9b3277289278dc40e022311f8e083833)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b3408ae4c64cb674b1d5f0f30171759113ce722a)
Conflicts:
libavcodec/mpeg4videodec.c
Also change the type of begin, end and smp to ptrdiff_t to make the
comparison well-defined.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Thilo Borgmann <thilo.borgmann@mail.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit afc7748d1f6abc4b3b1cc957b0fa6941837db3d0)
Conflicts:
libavcodec/alsdec.c
If begin is smaller than t, the subtraction 'begin -= t' wraps around,
because begin is unsigned. The same applies for end < t.
This causes segmentation faults.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit faf9fe2c224ea81a98afd53e2f0be0a2e13aeca9)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If the same idx is used for more than one i, at least one entry in
sconf->chan_pos remains uninitialized.
This can cause segmentation faults.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ef16501aebed43e34a3721336e8bee732eca2877)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes an invalid read if end is 0:
band_end = ff_ac3_bin_to_band_tab[end-1] + 1;
Depending on what is before the array, this can cause stack smashing,
when band_end becomes too large.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bc4fee7f2a51635fa3c0f61d1e5164da1efeded3)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If band->thr is 0.0f, the division is undefined, making norm_fac not a
number or infinity, which causes psy_band->threshold to become NaN.
This is passed on to other variables until it finally reaches
sce->sf_idx and is converted to an integer (-2147483648).
This causes a segmentation fault when it is used as array index.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e224aa41917454e7b5c23d9f2541425743ce595a)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0be54ad280cf114c02306b7063147e8379f8ed1e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
ac may be NULL and then accessing ac->avctx results in a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5b75689b987e4c4dd4f34d5c8be389547e9cc701)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes segmentation faults, when pic->linesize[0] is negative.
In that case 'line * pic->linesize[0] + pixel_ptr' is treated as
unsigned and wraps around.
This reverts commit 7d78a964.
The problem was introduced in commit f7e1367f, which should obsolete
that commit.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ae6fd7300b4e9f81d3b5ba201096ffe7cccf26fb)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes Ticket4460
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e88b3852aefaa39b2170ef185ad03dda18732821)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array access
Fixes: da14e86d8462be6493eab16bc2d40f88/asan_heap-oob_204cfd2_528_cov_340150052_COMPRESS.BMP
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f7e1367f58263593e6cee3c282f7277d7ee9d553)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06c70d45373dedc600f28e345685b130b60203c1)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: signal_sigsegv_35bcf26_471_cov_2806540268_CAINIT_A_SHARP_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57e5812198aada016e9ba4149123c541f8c8a7ec)
Conflicts:
libavcodec/hevc_ps.c
Fixes Ticket4227
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2c660e34cf3c2b77cd2bef6f292920334dfd9192)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 31816eae32019ff0e2243533f618efa2a4da9c33)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array accesses
Fixes: ffmpeg_012v_crash.ts
Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Reviewed-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 48df30d36c3ca360c407d84f96749888d1fbe853)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes division by 0
Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d3b25383daffac154846daeb4e4fb46569e728db)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
So far it is only set in roq_encode_frame, but it is used in
roq_encode_end to free the coded_frame. This currently segfaults if
roq_encode_frame is not called between roq_encode_init and
roq_encode_end.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cf82c426fadf90105e1fb9d5ecd267cc3aa2b288)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
allowing access to the size but not the extradata itself is not useful
and could lead to potential problems if writing happens through this field
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Reviewed-by: Lukasz Marek <lukasz.m.luki2@gmail.com>
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1f4088b28540080ce1d42345c5614be3e1a6a197)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
According to the WebP Lossless Bitstream Specification the highest
allowed value for a prefix code is 39.
If prefix_code is too large, the calculated extra_bits has an invalid
value and triggers an assertion in get_bits.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5de2dab12b951b2fe121eb18503accfc91cd1565)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If it doesn't fit into 12 bits it triggers an assertion.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2578a546183da09d49d5bba8ab5e982dece1dede)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Normally the aic decoder finds the proper slice combination (multiple of
some number less than 32) but in case of odd width, it resorts to the
default values, which were actually swapped.
The number of slices is modified to account for such odd width cases.
CC: libav-stable@libav.org
(cherry picked from commit e878ec0d47cd6228c367b2f3128b76d7523f7255)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This ensures that all current and future code-pathes get bpp checked
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d5e9fc782150d4596c72440a0aa02b7f4f1254b1)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash2.avi
Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Tested-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e3201c38d53d2b8b24d0bc95d726b2cb1752dc12)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
According to the WebP Lossless Bitstream Specification
"each transform is allowed to be used only once".
If a transform is more than once this can lead to memory
corruption.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c089e720c1b753790c746a13053636d7facf6bf0)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash.avi
Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08509c8f86626815a3e9e68d600d1aacbb8df4bf)
Conflicts:
libavcodec/mjpegdec.c
(cherry picked from commit 0861bb66d7c281c92a0746167bb141735a660856)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Several encoders were multiplying the buffer size by 8, in order to get
a bit size. However, the buffer_size argument is for the byte size of
the buffer. We had experienced crashes encoding prores (Anatoliy) at
size 4096x4096.
(cherry picked from commit 50833c9f7b4e1922197a8955669f8ab3589c8cef)
Conflicts:
libavcodec/proresenc_kostya.c
Change register constraint on the v variable from = to +. This was causing GCC
to think that the v variable was never read and therefore not initialize it.
This fixes about 20 fate failures on mips64el.
Signed-off-by: James Cowgill <james410@cowgill.org.uk>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b9de1303a6414174ab2f3bccefa801bfabcf0f88)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
log2 values which imply numeric overflow are not supported
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 205b2ba3d677330e023aac2f4bd3f624039256b9)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: asan_heap-oob_4d4eb0_3994_cov_3169972261_zmbv_15bit.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1f5c7781e63d6519192ada59c1e36bcecc92791d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes integer overflow and out of array read.
Fixes: signal_sigsegv_24169e6_3445_cov_3778346427_snow_chroma_bug.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f4cbf940212079a34753c7f4d6c6b5a43586d30)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Averaging over 2 pixels doesn't work correctly for the last pixel, because the
rest of the buffer is not initialized.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 87513d654546a99f8ddb045ca4fa5d33778a617e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes invalid reads of the packet buffer in av_dup_packet
Based on patch by Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d96142e9af92ded84f2580620c571ab96c4bb657)
Conflicts:
libavcodec/a64multienc.c
This fixes freeing the frame buffer twice on cleanup leading to a crash.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39e4ed7c1d8d840be47f6d604704d47a59a9ae5d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Based on patch by Francisco Blas Izquierdo Riera
Commit message partly taken from carl
fixes a compilation
error in mlpdsp_init.c with -fstack-check and some gcc compilers (I
reproduced the issue with gcc 4.7.3) by simplifying the code.
See also https://bugs.gentoo.org/show_bug.cgi?id=471756
$ make libavcodec/x86/mlpdsp_init.o
libavcodec/x86/mlpdsp_init.c: In function ‘mlp_filter_channel_x86’:
libavcodec/x86/mlpdsp_init.c:142:5: error: can’t find a register in
class ‘GENERAL_REGS’ while reloading ‘asm’
libavcodec/x86/mlpdsp_init.c:142:5: error: ‘asm’ operand has impossible
constraints
4551 -> 4509 dezicycles
Reviewed-by: Ramiro Polla <ramiro.polla@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03f39fbb2a558153a3c464edec1378d637a755fe)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This prevents using a wrong (first thread's) AVCodecContext if decoding
a frame in the first pass over all threads fails.
(cherry picked from commit a06b0b1295c51d100101e0ca0434e199ad6de6b5)
Conflicts:
libavcodec/h264.c
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This might fix a hypothetical race condition
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f111831ed61103f9fa8fdda41473a23da016bdaa)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Conflicts:
libavcodec/h264_slice.c
Fixes out of array read
Fixes: asan_static-oob_30328b6_719_cov_3325483287_H264_artifacts_motion.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69aa79365c1e8e1cb597d33e77bf1062c2ef47d4)
Conflicts:
libavcodec/h264_ps.c
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
