Commit Graph

57999 Commits

Author SHA1 Message Date
Michael Niedermayer
a0aa5c34a9 avcodec/hevc: Check entry point arrays for malloc failure
Fixes null pointer dereference
Fixes: signal_sigsegv_e1d3b6_2192_DBLK_F_VIXS_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 22bfb4be28)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:17:08 +01:00
Luca Barbato
d63476347a hevc: Bound check slice_qp
The T-REC-H.265-2013044 page 79 states they have to be into the range
[-s->sps->qp_bd_offset, 51].

Fixes: asan_stack-oob_eae8e3_9522_WP_MAIN10_B_Toshiba_3.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit aead772b58)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:17:04 +01:00
Luca Barbato
39545c5482 hevc: Reject impossible dependent tile
The tile 0 cannot depend on a previous one.
Prevent an out of array bound load in ff_hevc_cabac_init().

Fixes: asan_heap-oob_e3a924_1630_DBLK_A_MAIN10_VIXS_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

Reviewed-by: Guillaume Martres <smarter@ubuntu.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 48a5b15543)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:16:58 +01:00
Guillaume Martres
e43805d401 hevc: remove useless clip in FUNC(sao_band_filter)()
The src buffer should only contain values in the interval
[0, (1 << BIT_DEPTH) - 1]. Since shift = (BIT_DEPTH - 5), src[x] >> shift
must be in the interval [0, 31], so no clip is needed.

This removes the code that was changed in 5856bca360
as the clip that was repositioned in that commit is removed

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b00a8b4d19)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:15:07 +01:00
Guillaume Martres
ea21b7b68c hevc: clip pixels when transquant bypass is used
Fixes: asan_stack-oob_eae8e3_7333_WPP_B_ericsson_MAIN10_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

This is a more proper fix than 5856bca360

The reconstructed picture should always be clipped (see section 8.6.5),
previously we did not clip coding units where
cu_transquant_bypass_flag == 1

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c9fe0caf7a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:12:17 +01:00
Luca Barbato
738a2a04b6 hevc: Clip the pixel before shifting
Prevent an out of array bound read.

Fixes: asan_stack-oob_eae8e3_7333_WPP_B_ericsson_MAIN10_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5856bca360)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:12:01 +01:00
Michael Niedermayer
706dca18d0 avcodec/hevc: use av_mallocz() for allocating tab_ipm
Fixes use of uninitialized memory and out of stack array read
Fixes: signal_sigsegv_ecc526_7846_WPP_C_ericsson_MAIN_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0999f1613b)

Conflicts:

	libavcodec/hevc.c
2014-01-13 16:11:50 +01:00
Michael Niedermayer
b3c3dc54a5 avcodec/alac: only set *got_frame_ptr when all channels have been decoded
Fixes use of uninitialized memory
Fixes: msan_uninit-mem_7f8b64436530_7895_quicktime_newcodec_applelosslessaudiocodec.m4a
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e11983bda0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:10:34 +01:00
Clément Bœsch
b6af89be6a avformat/pjsdec: dont increase pointer when its already at the end in read_ts()
Fixes use of uninitialized memory
Fixes: msan_uninit-mem_7f91f2de7764_2649_PJS_capability_tester.pjs
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b84a7330af)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:10:30 +01:00
Michael Niedermayer
57824d1ba6 avcodec/wmalosslessdec: shrink output on error so no uninitialized data is returned
Fixes use of uninitialized memory
partly fixes: msan_uninit-mem_7f7834b6a530_6473_luckynight-partial.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6b18a6839b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:10:25 +01:00
Michael Niedermayer
cd753ee36b avcodec/wmalosslessdec: deallocate uninitialized frame on decode_tilehdr() failure
Fixes use of uninitialized memory
partly fixes: msan_uninit-mem_7f7834b6a530_6473_luckynight-partial.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ae3856dcaf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:10:12 +01:00
Michael Niedermayer
aaaafc300d avcodec/wmalosslessdec: Pass on error code from decode_tilehdr()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03fff09b32)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:10:09 +01:00
Michael Niedermayer
6da213ce20 avformat/matroskadec: check generic audio deinterleaver sub_packet_size against frame_size
Fixes use of uninitialized memory
Fixes: msan_uninit-mem_7f67d052a530_7517_nosound.mkv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a1ed1c2193)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:09:26 +01:00
Michael Niedermayer
763a808c96 avformat/flvdec: initialize context before reading from it
Fixes use of uninitialized memory
Fixes: msan_uninit-mem_7f9b8387069e_5377_flv_with_pcm_s16be_audio_track.flv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 396ddcf22d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:09:22 +01:00
Michael Niedermayer
4f3b0afaa2 avformat/mxfdec: check avio_read(UID) result
Fixes use of uninitialized memory
Fixes: msan_uninit-mem_7fc9ba2fd98e_82_02785736.mxf
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4162ceea93)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:09:15 +01:00
Michael Niedermayer
050d8d727a avformat/rmdec: when reading audio blocks, dont leave holes when reading fails
The fate test is changed because the reference file depends on the use of
non cleared data at the very
end. Alternatively we could upload a new reference file, though that would
then have to be changed every time the handling of a truncated frame changes
or theres a change to error concealment, each time adding a new file ...

Fixes use of uninitialized memory
Fixed: msan_uninit-mem_7f3c02b81363_2787_RLG2_19.rm
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 77d2a1ca59)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 16:08:26 +01:00
Michael Niedermayer
6fb0f37def avcodec/h264: fix code that blindly dereferences NULL DPB
Fixes mixed flushing and decoding NULL packets
Found-by: wm4

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d9339ab553)

Conflicts:

	libavcodec/h264.c
2014-01-13 16:06:13 +01:00
Hendrik Leppkes
70028e917c vdpau: restore compatibility with deprecated fields in AVVDPAUContext
Fixes ticket #3133.

Signed-off-by: Hendrik Leppkes <h.leppkes@gmail.com>
Tested-by: EricV
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 23bc1351ad)
2014-01-11 00:21:06 +01:00
Carl Eugen Hoyos
6ce835d77f Fix libopenjpeg colour range adjust for 8<bpp<16.
Fixes ticket #3284.

Reviewed-by: Michael Bradshaw
(cherry picked from commit 8298b54179)
2014-01-10 14:07:36 +01:00
Michael Niedermayer
d45a724192 avcodec/msvideo1enc: fix SKIPS_MAX
Fixes Ticket3270

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fb8f5d0510)
2014-01-09 11:43:21 +01:00
Carl Eugen Hoyos
a48440d857 Use the h264 parser when decoding VSSH in avi.
Fixes ticket #3261 visually.

Analyzed-by: Michael Doilnitsyn
(cherry picked from commit 94cf4f8bac)
2014-01-08 10:19:52 +01:00
Michael Niedermayer
de14fe7e29 avformat/mxfdec: detect loops during header parsing
The header parser uses forward and backward parsing, making the
bulletproof prevention of loops difficult, thus this simple
detection code.
If someone improves the forward/backward parsing so it cannot loop
then this commit should be reverted

Fixes Ticket3278

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1c010fd035)
2014-01-08 10:19:36 +01:00
Michael Niedermayer
580d3ef6fe avformat/mov: Check that we have a stream before accessing it in mov_read_ares()
Fixes out of array read
Fixes: signal_sigsegv_6f1855_3910_avid_test_alpha.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a7f27453f6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-08 00:25:25 +01:00
Michael Niedermayer
2864c987d8 avformat/ipmovie: check OPCODE_INIT_VIDEO_BUFFERS size more completely
Fixes use of uninitialized data

Fixes: signal_sigsegv_1571228_5930_ipmovie_interplayvideo_interplay_dpcm__bislogo.mve

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2e97e24409)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-08 00:25:20 +01:00
Michael Niedermayer
d1a9195863 avcodec/mjpegdec: check len in mjpeg_decode_app() more completely
Avoids len from becoming negative and causing assertion failure

Fixes: signal_sigabrt_7ffff7126425_5140_fd44dc63fa7bdd12ee34fc602231ef02.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6060234d43)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-08 00:25:07 +01:00
Michael Niedermayer
898ab02557 avformat/avidec: Dont assert the existence of an index for video streams.
Its possible in various rare cases that an index cannot be created or allocated.
Fixes assertion failure
Fixes: signal_sigabrt_7ffff7126425_7712_pokem.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7865759409)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-08 00:25:02 +01:00
Michael Niedermayer
69aa3d5b88 avcodec/hevc: clear HEVClc when its deallocated in hevc_decode_free()
Fixes reading freed memory
Fixes: asan_heap-uaf_1abf8ef_3987_NUT_A_ericsson_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 21a2fb7e05)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-08 00:23:49 +01:00
Michael Niedermayer
dcecca0758 avutil/log: check that len is within the buffer before reading it
Fixes out of array read
Fixes: asan_heap-oob_19d6979_6857_mmw_deadzy.ogg
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 808c10e728)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-08 00:23:43 +01:00
Michael Niedermayer
413065aff4 avcodec/g2meet: check available space before copying palette
Fixes out of array read
Fixes: asan_heap-uaf_ae6067_5415_g2m4.wmv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6d9dad6a7c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-08 00:23:35 +01:00
Michael Niedermayer
c094aec76e avcodec/ac3dec: check bap before use.
Fixes out of array read
Fixes assertion failure
Fixes asan_static-oob_16431c0_8036_rio_bravo_mono_64_spx.ac3

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4782c4284f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-08 00:23:23 +01:00
Michael Niedermayer
5e21989de4 Revert "Merge remote-tracking branch 'qatar/master'" (43dec5ef9a)
Fixes out of array accesses
Fixes asan_static-oob_eb9812_5961_iv41.avi
This reverts the merge of c9ef6b0932

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit c3d5cd1ebf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-08 00:23:18 +01:00
Michael Niedermayer
908b951b4e avcodec/hevc: Fix modulo operations
Fixes qp fields becoming out of range
Fixes: asan_static-oob_e393a3_6998_WPP_A_ericsson_MAIN10_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4ced5d7780)

Conflicts:

	libavcodec/hevc_filter.c
2014-01-07 23:48:10 +01:00
Michael Niedermayer
4b0cecb457 avcodec/hevc_ps: check that VPS referenced from SPS exists
This matches how its done for SPS/PPS.
An alternative to this is to check it when its used.

Fixes null pointer dereference
Fixes: signal_sigsegv_e30a43_1437_CIP_A_Panasonic_3.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d66bab0a69)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 23:47:39 +01:00
Nicolas George
3dae9d13e5 lavc/mjpegenc: use proper error codes.
(cherry picked from commit 2ebaadf35c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Nicolas George
9189a0a71b lavc/mjpegenc: check av_frame_alloc() failure.
(cherry picked from commit 19a2d101ac)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Nicolas George
4b25b5a8a3 lavc/libopenjpegenc: check av_frame_alloc() failure.
(cherry picked from commit 97af2faaba)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Nicolas George
4c3cd88144 lavc/diracdec: check av_frame_alloc() failure.
(cherry picked from commit a91394f4de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Nicolas George
4d70639d53 lavc/utils: check av_frame_alloc() failure.
(cherry picked from commit 38004051b5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Nicolas George
a6ba0f7be9 ffprobe: check av_frame_alloc() failure.
(cherry picked from commit a55692a960)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Nicolas George
89205b637e lavc/ffwavesynth: fix dependency sizeof(AVFrame).
(cherry picked from commit bcfcb8b852)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Michael Niedermayer
88058b4650 ffprobe: Dont clear AVFrame between uses.
The old API required this clearing in the past, the new API does not
require it.

Fixes memleak
Regression introduced by 37a749012a

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 565f786d1d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Michael Niedermayer
ca22a2dec5 avcodec/utils: drop 2 dependancies on sizeof(AVFrame)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c90f31146e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Michael Niedermayer
d058583510 avcodec/libvorbisenc: drop dependancy on sizeof(AVFrame)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3c8b085764)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Michael Niedermayer
e0c3c612eb ffprobe: drop dependancy on sizeof(AVFrame)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bf1c87ee7a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Michael Niedermayer
4d4a10cfa1 avcodec/flashsv2enc: drop dependancy on sizeof(AVFrame)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b8f4410ff6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Michael Niedermayer
ce675bd54a avcodec/j2kenc: drop dependancy on sizeof(AVFrame)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8443b27072)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Michael Niedermayer
965eb42be0 avcodec/libopenjpegenc: drop dependancy on sizeof(AVFrame)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1458f0647c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Michael Niedermayer
d2578f8152 avcodec/mjpegenc: drop dependancy on sizeof(AVFrame)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5b3f4b3ef5)

Conflicts:

	libavcodec/mjpegenc.c
2014-01-07 21:28:41 +01:00
Michael Niedermayer
31c52cd442 avcodec/msvideo1enc: drop dependancy on sizeof(AVFrame)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c81234651f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00
Michael Niedermayer
fa220e7307 avcodec/diracdec: avoid depending on sizeof(AVFrame)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fca7943850)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-07 21:28:41 +01:00