Commit Graph

57999 Commits

Author SHA1 Message Date
Michael Niedermayer
3f30c49f7e avcodec/utils: Align YUV411 by as much as the other YUV variants
Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash2.avi

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Tested-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e3201c38d5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-22 12:27:23 +02:00
Andreas Cadhalpun
dc2d15360c webp: ensure that each transform is only used once
According to the WebP Lossless Bitstream Specification
"each transform is allowed to be used only once".

If a transform is more than once this can lead to memory
corruption.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c089e720c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-22 12:27:23 +02:00
Andreas Cadhalpun
86c024ea9e avformat/flvenc: check that the codec_tag fits in the available bits
flags is later written with avio_w8 and if it doesn't fit in one byte it
triggers an av_assert2.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e8565d21c2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-28 23:00:55 +01:00
Michael Niedermayer
0b0a8cd133 swscale/utils: clear formatConvBuffer on allocation
Fixes use of uninitialized memory
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 007498fc1a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-27 03:34:27 +01:00
Michael Niedermayer
dfc3cd73ff avcodec/mjpegdec: Skip blocks which are outside the visible area
Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash.avi

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08509c8f86)

Conflicts:

	libavcodec/mjpegdec.c
(cherry picked from commit 0861bb66d7c281c92a0746167bb141735a660856)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-26 23:35:26 +01:00
Andreas Cadhalpun
8a9d5a8140 avformat/bit: only accept the g729 codec and 1 channel
Other codecs/channel numbers are not supported by this muxer.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d0b8640f75)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-26 23:28:01 +01:00
Andreas Cadhalpun
81196b6d60 avformat/bit: check that pkt->size is 10 in write_packet
Ohter packet sizes are not supported by this muxer.

This avoids a null pointer dereference of pkt->data.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit eeda2c3de8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-26 23:27:12 +01:00
Andreas Cadhalpun
cd74b344c2 avformat/adxdec: check avctx->channels for invalid values
This avoids a null pointer dereference of pkt->data.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7faa40af98)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-26 21:02:13 +01:00
Dyami Caliri
b49f8de4ec Fix buffer_size argument to init_put_bits() in multiple encoders.
Several encoders were multiplying the buffer size by 8, in order to get
a bit size. However, the buffer_size argument is for the byte size of
the buffer. We had experienced crashes encoding prores (Anatoliy) at
size 4096x4096.
(cherry picked from commit 50833c9f7b)

Conflicts:

	libavcodec/proresenc_kostya.c
2015-02-26 20:49:50 +01:00
James Cowgill
11141fb9ff mips/acelp_filters: fix incorrect register constraint
Change register constraint on the v variable from = to +. This was causing GCC
to think that the v variable was never read and therefore not initialize it.

This fixes about 20 fate failures on mips64el.

Signed-off-by: James Cowgill <james410@cowgill.org.uk>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b9de1303a6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-26 19:48:30 +01:00
Michael Niedermayer
c417b7b566 avcodec/hevc_ps: Sanity checks for some log2_* values
log2 values which imply numeric overflow are not supported

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 205b2ba3d6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-25 17:47:22 +01:00
Michael Niedermayer
41ca0489ba avcodec/zmbv: Check len before reading in decode_frame()
Fixes out of array read
Fixes: asan_heap-oob_4d4eb0_3994_cov_3169972261_zmbv_15bit.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1f5c7781e6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-25 12:42:50 +01:00
Michael Niedermayer
232b1f83de avcodec/h264: Only reinit quant tables if a new PPS is allowed
Fixes null pointer dereference
Fixes: signal_sigsegv_3042097_3007_cov_1741463594_non_monotone_timestamps1.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c23a0e77dd)

Conflicts:

	libavcodec/h264_slice.c
2015-02-25 03:42:31 +01:00
Michael Niedermayer
7afb16c9a1 avcodec/snowdec: Fix ref value check
Fixes integer overflow and out of array read.
Fixes: signal_sigsegv_24169e6_3445_cov_3778346427_snow_chroma_bug.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f4cbf9402)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-25 03:37:42 +01:00
Michael Niedermayer
a79e5c994c swscale/utils: More carefully merge and clear coefficients outside the input
Fixes out of array read
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1895d414aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-25 03:35:20 +01:00
Andreas Cadhalpun
1a8f9f0e58 avcodec/a64multienc: fix use of uninitialized values in to_meta_with_crop
Averaging over 2 pixels doesn't work correctly for the last pixel, because the
rest of the buffer is not initialized.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 87513d6545)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-23 13:47:55 +01:00
Michael Niedermayer
91ced1608c avcodec/a64multienc: don't set incorrect packet size
This fixes invalid reads of the packet buffer in av_dup_packet

Based on patch by Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d96142e9af)

Conflicts:

	libavcodec/a64multienc.c
2015-02-23 13:46:51 +01:00
Andreas Cadhalpun
d0da72ebb0 avcodec/a64multienc: use av_frame_ref instead of copying the frame
This fixes freeing the frame buffer twice on cleanup leading to a crash.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39e4ed7c1d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-23 13:35:08 +01:00
Michael Niedermayer
b62a3478e9 avcodec/x86/mlpdsp_init: Simplify mlp_filter_channel_x86()
Based on patch by Francisco Blas Izquierdo Riera
Commit message partly taken from carl

fixes a compilation
error in mlpdsp_init.c with -fstack-check and some gcc compilers (I
reproduced the issue with gcc 4.7.3) by simplifying the code.

See also https://bugs.gentoo.org/show_bug.cgi?id=471756

$ make libavcodec/x86/mlpdsp_init.o
libavcodec/x86/mlpdsp_init.c: In function ‘mlp_filter_channel_x86’:
libavcodec/x86/mlpdsp_init.c:142:5: error: can’t find a register in
class ‘GENERAL_REGS’ while reloading ‘asm’
libavcodec/x86/mlpdsp_init.c:142:5: error: ‘asm’ operand has impossible
constraints

4551 -> 4509 dezicycles

Reviewed-by: Ramiro Polla <ramiro.polla@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03f39fbb2a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-21 16:13:43 +01:00
Anton Khirnov
d005972995 h264: initialize H264Context.avctx in init_thread_copy
This prevents using a wrong (first thread's) AVCodecContext if decoding
a frame in the first pass over all threads fails.
(cherry picked from commit a06b0b1295)

Conflicts:

	libavcodec/h264.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-21 12:01:01 +01:00
Michael Niedermayer
94ebfef2fb avformat/omadec: Use 64bit for ret to avoid overflow
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0f55bc29d4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-20 21:14:32 +01:00
Michael Niedermayer
f8b6ba5574 avformat/vqf: Use 64bit for ret to avoid overflow
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb08687180)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-20 21:14:15 +01:00
Michael Niedermayer
3b17f32ec3 avformat/mvdec: Use 64bit for ret to avoid overflow
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 26c0cc154e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-20 20:49:51 +01:00
Michael Niedermayer
1480d9b3d3 avformat/gxf: Use 64bit for res to avoid overflow
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 12987f8900)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-20 20:30:26 +01:00
Michael Niedermayer
1aea2213a6 avformat/idcin: Use 64bit for ret to avoid overflow
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d1923d15a3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-20 20:30:11 +01:00
Michael Niedermayer
6ab48096f5 avcodec/h264_slice: ignore SAR changes in slices after the first
Fixes race condition and null pointer dereference
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 38d5241b7f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Conflicts:

	libavcodec/h264_slice.c
2015-02-07 14:17:54 +01:00
Michael Niedermayer
6c67cc2ad1 avcodec/h264_slice: Check picture structure before setting the related fields
This might fix a hypothetical race condition

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f111831ed6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Conflicts:

	libavcodec/h264_slice.c
2015-02-07 14:17:53 +01:00
Michael Niedermayer
cab9306040 avcodec/h264_slice: Do not change frame_num after the first slice
Fixes potential race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f906982c94)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Conflicts:

	libavcodec/h264_slice.c
2015-02-07 14:17:52 +01:00
Michael Niedermayer
01e2e746ba avcodec/h264: Be more strict on rejecting pps/sps changes
Fixes race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6fafc62b0b)

Conflicts:

	libavcodec/h264.c
2015-02-06 15:50:12 +01:00
Michael Niedermayer
17ecc8a771 avcodec/h264_ps: More completely check the bit depths
Fixes out of array read
Fixes: asan_static-oob_30328b6_719_cov_3325483287_H264_artifacts_motion.h264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69aa79365c)

Conflicts:

	libavcodec/h264_ps.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-06 04:41:12 +01:00
Michael Niedermayer
07a55344e6 avformat/thp: Check av_get_packet() for failure not only for partial output
Fixes null pointer dereference
Fixes: signal_sigsegv_db2c1f_3108_cov_163322880_pikmin2_opening1_partial.thp

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f2579dbb4b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-05 04:30:36 +01:00
Michael Niedermayer
813054c4bd swscale/utils: Limit filter shifting so as not to read from prior the array
Fixes out of array read
Fixes: asan_heap-oob_1fb2f9b_3780_cov_3984375136_usf.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 692b22626e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-05 00:42:45 +01:00
Michael Niedermayer
b844584c10 avcodec/mjpegdec: Check escape sequence validity
Fixes assertion failure
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-04 21:27:28 +01:00
Michael Niedermayer
a6ba4c1116 avcodec/mjpegdec: Check number of components for JPEG-LS
Fixes out of array accesses
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fabbfaa095)

Conflicts:

	libavcodec/mjpegdec.c
2015-02-04 21:27:25 +01:00
Michael Niedermayer
af0544ca3d avformat/mpc8: Use uint64_t in *_get_v() to avoid undefined behavior
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 05e1619529)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-04 15:08:32 +01:00
wm4
55b75ece70 avformat/mpc8: fix broken pointer math
This could overflow and crash at least on 32 bit systems.

Reviewed-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b737a2c528)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-04 01:05:26 +01:00
wm4
28f303542e avformat/mpc8: fix hang with fuzzed file
This can lead to an endless loop by seeking back a few bytes after each
attempted chunk read. Assuming negative sizes are always invalid, this
is easy to fix. Other code in this demuxer treats negative sizes as
invalid as well.

Fixes ticket #4262.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 56cc024220)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-04 00:48:43 +01:00
wm4
6e7f183ae6 avformat/tta: fix crash with corrupted files
av_add_index_entry() can fail, for example because the parameters are
invalid, or because memory allocation fails. Check this; it can actually
happen with corrupted files.

The second hunk is just for robustness. Just in case functions like
ff_reduce_index() remove entries. (Not sure if this can actually
happen.)

Fixes ticket #4294.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6a0cd529a3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-03 16:18:19 +01:00
Michael Niedermayer
37723e0d0c avcodec/mpegvideo_enc: Fix number suffixes in rc_buffer_size calculation
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4531e2c489)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-01 20:33:38 +01:00
Michael Niedermayer
5feffd6268 avcodec/h264_cabac: use int instead of long for mbb_xy
The mb address fits in int

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 592ba6ec10)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-01 20:32:55 +01:00
Michael Niedermayer
6862970dbf avformat/omadec: fix number suffix
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1f7f5903a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-01 20:32:13 +01:00
Michael Niedermayer
633ce6baaa avformat/smacker: Fix number suffix
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 465f3705b1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-01 20:23:30 +01:00
Michael Niedermayer
201d626bb9 avformat/matroskadec: Fix number suffixes
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc3cdb00d0)

Conflicts:

	libavformat/matroskadec.c
2015-02-01 20:22:17 +01:00
Michael Niedermayer
364bdddaad swresample/dither: Cleanup number suffixes
The <<31 case needs LL

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c77cc2c176)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-01 20:17:26 +01:00
Michael Niedermayer
e651a2f88c avformat/utils: Fix number suffixes in tb_unreliable()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b15bba2ae)

Conflicts:

	libavformat/utils.c
2015-02-01 20:15:22 +01:00
Michael Niedermayer
7c54f14168 avformat/rmdec: Check for overflow in ff_rm_read_mdpr_codecdata()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03abf55f25)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-18 00:27:57 +01:00
Vittorio Giovara
d00e2fa284 swscale: check memory allocations
Bug-Id: CID 1257779
(cherry picked from commit 1dd797e3c9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-17 22:49:55 +01:00
Michael Niedermayer
9d156e24d7 avcodec/flac_parser: fix handling EOF if no headers are found
Fixes assertion failure
Fixes Ticket4269

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c4d85fc23c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-17 02:13:04 +01:00
Michael Niedermayer
ccca7ae8c6 ffmpeg: Clear error message array at init.
This avoids printing uninitialized bytes if no error message is set

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6d1a2efb8a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-08 23:10:54 +01:00
wm4
5a98a00001 avcodec/dvdsubdec: fix accessing dangling pointers
dvdsub_decode() can call append_to_cached_buf() 2 times, the second time
with ctx->buf as argument. If the second append_to_cached_buf() reallocs
ctx->buf, the argument will be a pointer to the previous, freed block.
This can cause invalid reads at least with some fuzzed files - and
possibly with valid files.

Since packets can apparently not be larger than 64K (even if packets are
combined), just use a fixed size buffer. It will be allocated as part of
the DVDSubContext, and although some memory is "wasted", it's relatively
minimal by modern standards and should be acceptable.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 816577716b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-08 18:04:36 +01:00