* commit '964fef3f3ced60e67831549df223bc177e1537c9':
h264: Make sure reinit failures mark the context as not initialized
See: e8714f6f93d1a32f4e4655209960afcf4c185214
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'f29cf578923d2d4d2cf891dfc0c7ac45c641a5b7':
png: Set the color range as full range
Conflicts:
libavcodec/pngdec.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '1dcb68c02a55648facb82c0b2b6a1eb3dc18f988':
alac: Reject rice_limit 0 if compression is used
Conflicts:
libavcodec/alac.c
See: 4b657a1b1eedcf38bcf36e89a2f4be6f76b5ce09
Merged-by: Michael Niedermayer <michaelni@gmx.at>
If rice_limit is 0, k can be 0 in decode_scalar, which calls show_bits(gb, k).
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b657a1b1eedcf38bcf36e89a2f4be6f76b5ce09)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
s->decoded_buffer is allocated with a min_size of:
2 * FFALIGN(blockstodecode, 8) * sizeof(*s->decoded_buffer)
Then it is assigned to s->decoded[0] (and s->decoded_buffer + FFALIGN(blockstodecode, 8)
to s->decoded[1]) and passed as out buffer to decode_array_0000.
In this function 64 elements of the out buffer are written
unconditionally and outside the array if blockstodecode is too small.
This causes memory corruption, leading to segmentation faults or other
crashes.
Thus change decode_array_0000 to write at most blockstodecode elements
of the out buffer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 699341d647f7af785fb8ceed67604467b0b9ab12)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Can be tested with:
$ ./configure --cc='cc -m32' --disable-optimizations --enable-pic
(cherry picked from commit b38910c9790253b362839042a17e13252c1d4b90)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '891ed1184e49bc7944311302d9ece01e87c188a8':
libvpx: Fix mixed use of av_malloc() and av_reallocp()
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Simplifies the code and makes it build on certain compilers
running out of registers on x86.
CC: libav-stable@libav.org
Reported-By: mudler
(cherry picked from commit e4610300de6869bd6b3b00e76cfeabb6d7653dcd)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 4dc0fbb13c33b4e5bdb766652f4daf900ccc952f)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
The format uses full range for the gray formats.
CC: libav-stable@libav.org
(cherry picked from commit 0f50c53cfb959162f2bccc1a2c2e066d35723595)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
If in compression mode rice_limit = 0 leads to call
`show_bits(gb, k)` in `decode_scalar` with k = 0.
Request a sample in case it is valid and it should be accepted.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
CC: libav-stable@libav.org
(cherry picked from commit cb5324200ccdc693dd5b28dcd7d4b722fad83ea2)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
The decode_array_0000 assumed that 64 is the minimal block size
while it is not.
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit ac1660509ecfbeca7b63eb5ab8360011180e705b)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
It uses 6 registers, unbreaks building on hardened x86 system.
Bug-Id: gentoo/541930
CC: libav-stable@libav.org
(cherry picked from commit 2af720fe5f0418612a8fc26b0147a0e10414fcbe)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
So far it is only set in roq_encode_frame, but it is used in
roq_encode_end to free the coded_frame. This currently segfaults if
roq_encode_frame is not called between roq_encode_init and
roq_encode_end.
CC:libav-stable@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 9f6c36d961d27283808310e3ca1d8390b55fce9b)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
This buffer is resized when vpx_codec_get_cx_data() returns a
VPX_CODEC_STATS_PKT packet.
CC: libav-stable@libav.org
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
(cherry picked from commit 7244cefd6e6ba7258cb022dfd7a284099d88a3e8)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
For max_order = 0 the clipping range is invalid. (amin = 2, amax = 1)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 58d605ee9b3277289278dc40e022311f8e083833)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Also change the type of begin, end and smp to ptrdiff_t to make the
comparison well-defined.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Thilo Borgmann <thilo.borgmann@mail.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit afc7748d1f6abc4b3b1cc957b0fa6941837db3d0)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If begin is smaller than t, the subtraction 'begin -= t' wraps around,
because begin is unsigned. The same applies for end < t.
This causes segmentation faults.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit faf9fe2c224ea81a98afd53e2f0be0a2e13aeca9)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If the same idx is used for more than one i, at least one entry in
sconf->chan_pos remains uninitialized.
This can cause segmentation faults.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ef16501aebed43e34a3721336e8bee732eca2877)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'b0b92ee04d1ee49a5d8b1aca718ea7d7d44932aa':
aacpsy: correct calculation of minath in psy_3gpp_init
Merged-by: Michael Niedermayer <michaelni@gmx.at>
avctx->bits_per_raw_sample is used in get_sbits_long, which only
supports up to 32 bits.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4c2b88678b436f59132386d9be2fc143e3ee480d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
For max_order = 0 the clipping range is invalid. (amin = 2, amax = 1)
CC: libav-stable@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 60f1cc4a1ffcbf24acbb543988ceeaec76b70818)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 378ee3bad5b99e8f90864af9bc851590e0f64825)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Also change the type of begin, end and smp to ptrdiff_t to make the
comparison well-defined.
CC: libav-stable@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 94bb1ce882a12b6d7a1fa32715a68121b39ee838)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 41a89cba6086de2bd24f9ec7e21200fa162505e9)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The minimum of the ath(x, ATH_ADD) function depends on ATH_ADD.
This patch uses the first order approximation to determine it.
For ATH_ADD = 4 this results in the value at 3407.06812 (-5.24241638)
not the one at 3410 (-5.24237967).
CC: libav-stabl@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 110f7f35fb615b97d983b1c6c6a714fddd28bcbe)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 7b66cf5ce7fdb8b3fa13459aab3f4d6ab559f1ea)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
avctx->bits_per_raw_sample is used in get_sbits_long, which only
supports up to 32 bits.
CC: libav-stable@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit e191aaca44b986816695e3b7ecfae64697fd6631)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 97010c74cbff177b58daf9a092b4e37a7da26f85)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
CC: libav-stable@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 8fc8024ea56e814cd257d5fe27b21a865080782f)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 0d3a7dd26490156b607541dd2e1faeaa0fc61a88)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
This fixes an invalid read if end is 0:
band_end = ff_ac3_bin_to_band_tab[end-1] + 1;
Depending on what is before the array, this can cause stack smashing,
when band_end becomes too large.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bc4fee7f2a51635fa3c0f61d1e5164da1efeded3)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If band->thr is 0.0f, the division is undefined, making norm_fac not a
number or infinity, which causes psy_band->threshold to become NaN.
This is passed on to other variables until it finally reaches
sce->sf_idx and is converted to an integer (-2147483648).
This causes a segmentation fault when it is used as array index.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e224aa41917454e7b5c23d9f2541425743ce595a)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0be54ad280cf114c02306b7063147e8379f8ed1e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
ac may be NULL and then accessing ac->avctx results in a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5b75689b987e4c4dd4f34d5c8be389547e9cc701)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes segmentation faults, when pic->linesize[0] is negative.
In that case 'line * pic->linesize[0] + pixel_ptr' is treated as
unsigned and wraps around.
This reverts commit 7d78a964.
The problem was introduced in commit f7e1367f, which should obsolete
that commit.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ae6fd7300b4e9f81d3b5ba201096ffe7cccf26fb)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes Ticket4460
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e88b3852aefaa39b2170ef185ad03dda18732821)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes non deterministic crash in ticket4408/fuzz2.264
Likely fixes other samples as well
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 43b434210e597d484aef57c4139c3126d22b7e2b)
Conflicts:
libavcodec/h264.h
libavcodec/h264_slice.c
(cherry picked from commit dbbc42858e87cdd04e6c3b7694f8b394d4bfcdc6)
Conflicts:
libavcodec/h264_slice.c
Fixes Ticket4408
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc58d5c43b4c7396fc69081eb0dfe5b6a21cb10d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array access
Fixes: da14e86d8462be6493eab16bc2d40f88/asan_heap-oob_204cfd2_528_cov_340150052_COMPRESS.BMP
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f7e1367f58263593e6cee3c282f7277d7ee9d553)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06c70d45373dedc600f28e345685b130b60203c1)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: signal_sigsegv_35bcf26_471_cov_2806540268_CAINIT_A_SHARP_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57e5812198aada016e9ba4149123c541f8c8a7ec)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
