30009 Commits

Author SHA1 Message Date
Michael Niedermayer
4b8f3c5bf3 avcodec/hevc_ps: Check num_long_term_ref_pics_sps
Fixes out of array access
Fixes: signal_sigsegv_35bd0f0_1182_cov_791726764_STRUCT_B_Samsung_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ea38e5a6b75706477898eb1e6582d667dbb9946c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:37 +01:00
Michael Niedermayer
18dba3d80d avcodec/mjpegdec: Fix integer overflow in shift
Fixes: signal_sigabrt_7ffff6ac7bb9_2683_cov_4120310995_m_ijpg.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 970a8f1c256f08d2f6414d573a54f2fa035c8e7a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
2ba17ac96c avcodec/hevc_ps: Check return code from pps_range_extensions()
Fixes out of array read
Fixes: asan_heap-oob_177e222_885_cov_1532528832_MERGE_D_TI_3.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9f9440bd8122cc8798139c9370db0873a24ae14b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
12e064d6c2 avcodec/rawdec: Check the return code of avpicture_get_size()
Fixes out of array access
Fixes: asan_heap-oob_22388d0_3435_cov_3297128910_small_roll5_FlashCine1.cine
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1d3a3b9f8907625b361420d48fe05716859620ff)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
f2595a6c38 avcodec/pngdec: Check IHDR/IDAT order
Fixes out of array access
Fixes: asan_heap-oob_20a6c26_2690_cov_3434532168_mail.png
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 79ceaf827be0b070675d4cd0a55c3386542defd8)

Conflicts:

	libavcodec/pngdec.c
2014-11-30 21:40:36 +01:00
Michael Niedermayer
a654f483cd avcodec/flacdec: Call ff_flacdsp_init() unconditionally
Fixes out of array access
Fixes: signal_sigsegv_324b135_3398_cov_246853371_short.flac
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5c01ccdf5a9a330d4c51a9b9ea721fd8f1fb70b)

Conflicts:

	libavcodec/flacdec.c
2014-11-30 21:40:36 +01:00
Michael Niedermayer
991a203479 avcodec/utils: Check that the data is complete in avpriv_bprint_to_extradata()
Fixes out of array read
Fixes: asan_heap-oob_4d2250_814_cov_2745172097_JACOsub_capability_tester.jss
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3d5d95db3f5d8e2093e9e19d0c46e86f54ed2a5d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
8524009161 avcodec/mjpegdec: Fix context fields becoming inconsistent
Fixes out of array access
Fixes: asan_heap-oob_1ca4f85_2760_cov_144449187_miss_congeniality_pegasus_ljpg.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0eecf40935b22644e6cd74c586057237ecfd6844)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
47f345de1d avcodec/mjpegdec: Check for pixfmtid 0x42111100 || 0x24111100 with more than 8 bits
These cases are not supported yet

Fixes assertion failure
Fixes: signal_sigabrt_7ffff6ac7bb9_1_cov_1553101927_00.jpg
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0bf416f2628137e5389050fa323c329692dd4ba6)

Conflicts:

	libavcodec/mjpegdec.c
2014-11-30 21:40:36 +01:00
Michael Niedermayer
a3fdc0b35b avcodec/mpeg4video_parser: fix spurious extradata parse warnings
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7d37e45f6bac198bc986aeb987afe219edbbd45a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
cfda2a677c avcodec/wmaprodec: Fix integer overflow in sfb_offsets initialization
Fixes out of array read
Fixes: asan_heap-oob_2aec5b0_1828_classical_22_16_2_16000_v3c_0_exclusive_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5dcb99033df16eccc4dbbc4a099ad64457f9f090)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
11dfd1fa20 avcodec/utvideodec: fix assumtation that slice_height >= 1
Fixes out of array read
Fixes: asan_heap-oob_2573085_3783_utvideo_rgba_median.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7656c4c6e66f8a787d384f027ad824cc1677fda1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
f7e9fcda2d avcodec/options_table fix min of audio channels and sample rate
Found-by: Lukasz Marek <lukasz.m.luki2@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 206c98f303e833c9e94427c9e3f9867f85265f78)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Carl Eugen Hoyos
2719ba9ee3 lavc/utils: Make pix_fmt desc pointer const.
Fixes an "initialization discards qualifiers from pointer target type" warning.
(cherry picked from commit f05855414ed4cce97c06ba2a31f4987af47e6d4e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
56b84b023d Merge commit 'ca78ee73db9e059f501706ba6108e23902e84933' into release/2.4
* commit 'ca78ee73db9e059f501706ba6108e23902e84933':
  opusdec: make sure all substreams have the same number of coded samples

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 01:19:28 +01:00
Anton Khirnov
ca78ee73db opusdec: make sure all substreams have the same number of coded samples
Fixes invalid writes with invalid multichannel streams.

CC:libav-stable@libav.org
(cherry picked from commit 1973079417e8701b52ba810a72cb6c7c6f7f9a56)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-11-27 14:11:51 +01:00
Michael Niedermayer
c7b64a904a avcodec/mpegaudio_parser: fix off by 1 error in bitrate calculation
Fixes Ticket3918

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 817663897e59f45f60016fa9d3d16e325b803967)
2014-11-06 01:02:53 +01:00
Michael Niedermayer
a5cc8775cf avcodec/h264_sei: ff_h264_decode_sei: dont try to parse trailing zeroes
reduces noise for tserrors.ts

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 688a40b4ed7d2e07d3f96c2feecf785a4866e60c)
2014-11-05 10:13:26 +01:00
Michael Niedermayer
cd57d608a4 avcodec/mpeg12dec: do not trust AVCodecContext input dimensions
Fixes initial wtv dimensions
Fixes Ticket4070
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 77f1199e8fd9a289ad64eb2bb5bd4deeda8bccb8)
2014-11-05 09:52:29 +01:00
Michael Niedermayer
043f326060 avcodec/h264_slice: Clear table pointers to avoid stale pointers
Might fix Ticket3889

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 547fce95858ef83f8c25ae347e3ae3b8ba437fd9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:10:51 +01:00
Michael Niedermayer
9a641b909c avcodec/h264_parser: rewrite the parse_nal_units() loop logic based on h264.c
Fixes Ticket4011

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69a9a90d2ef795162074be24e3ad2182a8676af2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 16:27:12 +01:00
Michael Niedermayer
5405ba7b63 avcodec/h264: simplify find_start_code()
this also uses avpriv_find_start_code(), though no speed change is expected as
the area searched is generally small

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3b678da5e386c138316954e867d595f946666051)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 16:27:12 +01:00
Michael Niedermayer
70f6d553d9 Move get_avc_nalsize() and find_start_code() to h264.h
This allows sharing them with the h264 parser

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4898440f6bd19152373969159fff057b532c6374)

Conflicts:

	libavcodec/h264.c
2014-11-01 16:27:12 +01:00
Reimar Döffinger
25fc3deed8 mpeg4vdpau: Fix priv data size.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit 46353759cb3c90ad8eb4af6526c236513d477296)
2014-11-01 16:18:50 +01:00
Michael Niedermayer
63523485f4 avcodec/svq1dec: zero terminate embedded message before printing
Fixes out of array access
Fixes: asan_stack-oob_49b1e5_10_009.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e91ba2efa949470e9157b652535d207a101f91e0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
35bc67503e avcodec/cook: check that the subpacket sizes fit in block_align
Fixes out of array read
Fixes: asan_heap-oob_fb5c50_19_018.rmvb
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 10e32618acce9c3fc64c061eb7907e8a8d2749ae)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
1f636a697f avcodec/g2meet: check tile dimensions to avoid integer overflow
Fixes out of array access
Fixes: asan_heap-oob_12a55d3_30_029.wmv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 32e666c354e4a3160d8cf1d303cb51990b095c87)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
24d725f455 avcodec/utils: Align dimensions by at least their chroma sub-sampling factors.
Fixes: out of array accesses
Fixes: asan_heap-oob_112c6b3_13_012.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df74811cd53e45fcbbd3b77a1c42416816687c5c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
66fcf1fa40 avcodec/dnxhddec: treat pix_fmt like width/height
Fixes out of array accesses
Fixes: asan_heap-oob_22c9a39_16_015.mxf
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f3c0e0bf6f53df0977f3878d4f5cec99dff8de9e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
c7b7e0790c avcodec/dxa: check dimensions
Fixes out of array access
Fixes: asan_heap-oob_11222fb_21_020.dxa
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e70312dfc22c4e54d5716f28f28db8f99c74cc90)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
3f3e5f8f60 avcodec/dirac_arith: fix integer overflow
Fixes: asan_heap-oob_1078676_9_008.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39680caceebfc6abf09b17032048752c014e57a8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
8e95ddbe82 avcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y
Fixes very long but finite loop
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5145d22b88b9835db81c4d286b931a78e08ab76a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
8cba067fe5 avcodec/diracdec: Use 64bit in calculation of codeblock coordinates
Fixes integer overflow
Fixes out of array read
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 526886e6069636a918c8c04db17e864e3d8151c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
73c6520c09 avcodec/sgidec: fix count check
Fixes: asan_heap-oob_22b30d4_39_038.sgi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a050cf0c451bdf1c1bd512c4fce6b6f8a5e85102)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
ca47574e16 avcodec/sgidec: fix linesize for 16bit
Fixes: asan_heap-oob_22b30d4_39_038.sgi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3b20ed85489a14cb5028c873d06960dbc5eef88a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
045670a6f7 avcodec/hevc_ps: Check default display window bitstream and skip if invalid
Fixes Ticket4035

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 852aaead1fc294bcb63a1f9e384e781f6e51ded6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Christophe Gisquet
30a0622a5d avcodec/tiffenc: properly compute packet size
The bytes per row is a better indication of it.

Helps resolving ticket #3874 by fixing ffmpeg's encoder and transforming
the issue in a issue with non-compliant decoders. ffmpeg's one is ok,
but unfortunately, many others aren't handling correctly unusual chroma
samplings.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0e8bfd3c934768f9812dd20d71fa4709de54186d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:13 +01:00
Christophe Gisquet
f3d34cff76 utvideoenc: properly set slice height/last line
Mimic decoder and obey sampling.

Does not affect fate tests for utvideo.
Fixes ticket #3949.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb530dda7d76790b08ee3b7f67e251f3ce48c359)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:13 +01:00
Michael Niedermayer
3b7db9c4f5 Merge commit 'e443165c323406d01da7e7930f042d265d01fb35' into release/2.4
* commit 'e443165c323406d01da7e7930f042d265d01fb35':
  imc: fix order of operations in coefficients read

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:21:40 +01:00
Michael Niedermayer
0ddcee172e Merge commit '0b41eeac45fb7f7ad6d3f4fc846b00d108824b0b' into release/2.4
* commit '0b41eeac45fb7f7ad6d3f4fc846b00d108824b0b':
  hevc_mvs: make sure to always initialize the temporal MV fully

Conflicts:
	libavcodec/hevc_mvs.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:06:25 +01:00
Michael Niedermayer
13ecdb06f8 Merge commit 'de31f857077a52714f3a2f2e92ac037d42d37769' into release/2.4
* commit 'de31f857077a52714f3a2f2e92ac037d42d37769':
  hevc_mvs: initialize the temporal MV in case of missing ref

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:05:46 +01:00
Vittorio Giovara
e443165c32 imc: fix order of operations in coefficients read
Reported-by: Ruoyu <liangry@ucweb.com>
2014-10-15 14:51:19 +01:00
Anton Khirnov
0b41eeac45 hevc_mvs: make sure to always initialize the temporal MV fully
The spec requires this.

Fixes uninitialized reads on some samples.

Remove now unnecessary initialization of the whole merge candidate list.
2014-10-08 12:31:08 +01:00
Anton Khirnov
de31f85707 hevc_mvs: initialize the temporal MV in case of missing ref
The caller expects the MV to always be initialized.
2014-10-08 12:30:54 +01:00
Michael Niedermayer
84d26ab6eb avcodec/on2avc: Check number of channels
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_7_asan_heap-oob_4da4f3_173_Xmen_avc_500.vp6

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 550f3e9df3410b3dd975e590042c0d83e20a8da3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
1d99adc953 avcodec/hevc: fix chroma transform_add size
Fixes accessing misaligned address
Fixes: signal_sigsegv_1feb99c_10_signal_sigsegv_2d1d35c_79_cov_691940146_EXT_A_ericsson_3.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df8394c693d3c1e13b7ebf6af01c5e55321cf952)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
67991f3a3e avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de3043f012ad0f1ef11027d9488067e6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
eefc3ca7be avcodec/vorbisdec: Fix off by 1 error in ptns_to_read
Fixes read of uninitialized memory
Fixes: asan_heap-uaf_18dac2b_9_asan_heap-uaf_22eb375_208_beta3_test_small.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c50704ebf1777bee76772c4835d9760b3721057)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
506368f563 avcodec/svq3: Dont memcpy AVFrame
This avoids out of array accesses

Fixes: asan_heap-uaf_21f42e4_9_asan_heap-uaf_21f42e4_278_gl2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 075a165d2715837d125a9cc714fb430ccf6c9d6b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
c0c24bc9b3 avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9d62335e89d118a5b4e202edf39d905)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00