30101 Commits

Author SHA1 Message Date
Michael Niedermayer
47f345de1d avcodec/mjpegdec: Check for pixfmtid 0x42111100 || 0x24111100 with more than 8 bits
These cases are not supported yet

Fixes assertion failure
Fixes: signal_sigabrt_7ffff6ac7bb9_1_cov_1553101927_00.jpg
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0bf416f2628137e5389050fa323c329692dd4ba6)

Conflicts:

	libavcodec/mjpegdec.c
2014-11-30 21:40:36 +01:00
Michael Niedermayer
a3fdc0b35b avcodec/mpeg4video_parser: fix spurious extradata parse warnings
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7d37e45f6bac198bc986aeb987afe219edbbd45a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
cfda2a677c avcodec/wmaprodec: Fix integer overflow in sfb_offsets initialization
Fixes out of array read
Fixes: asan_heap-oob_2aec5b0_1828_classical_22_16_2_16000_v3c_0_exclusive_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5dcb99033df16eccc4dbbc4a099ad64457f9f090)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
11dfd1fa20 avcodec/utvideodec: fix assumtation that slice_height >= 1
Fixes out of array read
Fixes: asan_heap-oob_2573085_3783_utvideo_rgba_median.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7656c4c6e66f8a787d384f027ad824cc1677fda1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
f7e9fcda2d avcodec/options_table fix min of audio channels and sample rate
Found-by: Lukasz Marek <lukasz.m.luki2@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 206c98f303e833c9e94427c9e3f9867f85265f78)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Carl Eugen Hoyos
2719ba9ee3 lavc/utils: Make pix_fmt desc pointer const.
Fixes an "initialization discards qualifiers from pointer target type" warning.
(cherry picked from commit f05855414ed4cce97c06ba2a31f4987af47e6d4e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer
56b84b023d Merge commit 'ca78ee73db9e059f501706ba6108e23902e84933' into release/2.4
* commit 'ca78ee73db9e059f501706ba6108e23902e84933':
  opusdec: make sure all substreams have the same number of coded samples

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-28 01:19:28 +01:00
Anton Khirnov
ca78ee73db opusdec: make sure all substreams have the same number of coded samples
Fixes invalid writes with invalid multichannel streams.

CC:libav-stable@libav.org
(cherry picked from commit 1973079417e8701b52ba810a72cb6c7c6f7f9a56)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-11-27 14:11:51 +01:00
Michael Niedermayer
c7b64a904a avcodec/mpegaudio_parser: fix off by 1 error in bitrate calculation
Fixes Ticket3918

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 817663897e59f45f60016fa9d3d16e325b803967)
2014-11-06 01:02:53 +01:00
Michael Niedermayer
a5cc8775cf avcodec/h264_sei: ff_h264_decode_sei: dont try to parse trailing zeroes
reduces noise for tserrors.ts

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 688a40b4ed7d2e07d3f96c2feecf785a4866e60c)
2014-11-05 10:13:26 +01:00
Michael Niedermayer
cd57d608a4 avcodec/mpeg12dec: do not trust AVCodecContext input dimensions
Fixes initial wtv dimensions
Fixes Ticket4070
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 77f1199e8fd9a289ad64eb2bb5bd4deeda8bccb8)
2014-11-05 09:52:29 +01:00
Michael Niedermayer
043f326060 avcodec/h264_slice: Clear table pointers to avoid stale pointers
Might fix Ticket3889

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 547fce95858ef83f8c25ae347e3ae3b8ba437fd9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 02:10:51 +01:00
Michael Niedermayer
9a641b909c avcodec/h264_parser: rewrite the parse_nal_units() loop logic based on h264.c
Fixes Ticket4011

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69a9a90d2ef795162074be24e3ad2182a8676af2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 16:27:12 +01:00
Michael Niedermayer
5405ba7b63 avcodec/h264: simplify find_start_code()
this also uses avpriv_find_start_code(), though no speed change is expected as
the area searched is generally small

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3b678da5e386c138316954e867d595f946666051)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 16:27:12 +01:00
Michael Niedermayer
70f6d553d9 Move get_avc_nalsize() and find_start_code() to h264.h
This allows sharing them with the h264 parser

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4898440f6bd19152373969159fff057b532c6374)

Conflicts:

	libavcodec/h264.c
2014-11-01 16:27:12 +01:00
Reimar Döffinger
25fc3deed8 mpeg4vdpau: Fix priv data size.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit 46353759cb3c90ad8eb4af6526c236513d477296)
2014-11-01 16:18:50 +01:00
Michael Niedermayer
63523485f4 avcodec/svq1dec: zero terminate embedded message before printing
Fixes out of array access
Fixes: asan_stack-oob_49b1e5_10_009.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e91ba2efa949470e9157b652535d207a101f91e0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
35bc67503e avcodec/cook: check that the subpacket sizes fit in block_align
Fixes out of array read
Fixes: asan_heap-oob_fb5c50_19_018.rmvb
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 10e32618acce9c3fc64c061eb7907e8a8d2749ae)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
1f636a697f avcodec/g2meet: check tile dimensions to avoid integer overflow
Fixes out of array access
Fixes: asan_heap-oob_12a55d3_30_029.wmv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 32e666c354e4a3160d8cf1d303cb51990b095c87)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
24d725f455 avcodec/utils: Align dimensions by at least their chroma sub-sampling factors.
Fixes: out of array accesses
Fixes: asan_heap-oob_112c6b3_13_012.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df74811cd53e45fcbbd3b77a1c42416816687c5c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
66fcf1fa40 avcodec/dnxhddec: treat pix_fmt like width/height
Fixes out of array accesses
Fixes: asan_heap-oob_22c9a39_16_015.mxf
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f3c0e0bf6f53df0977f3878d4f5cec99dff8de9e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
c7b7e0790c avcodec/dxa: check dimensions
Fixes out of array access
Fixes: asan_heap-oob_11222fb_21_020.dxa
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e70312dfc22c4e54d5716f28f28db8f99c74cc90)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
3f3e5f8f60 avcodec/dirac_arith: fix integer overflow
Fixes: asan_heap-oob_1078676_9_008.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39680caceebfc6abf09b17032048752c014e57a8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
8e95ddbe82 avcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y
Fixes very long but finite loop
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5145d22b88b9835db81c4d286b931a78e08ab76a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
8cba067fe5 avcodec/diracdec: Use 64bit in calculation of codeblock coordinates
Fixes integer overflow
Fixes out of array read
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 526886e6069636a918c8c04db17e864e3d8151c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
73c6520c09 avcodec/sgidec: fix count check
Fixes: asan_heap-oob_22b30d4_39_038.sgi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a050cf0c451bdf1c1bd512c4fce6b6f8a5e85102)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
ca47574e16 avcodec/sgidec: fix linesize for 16bit
Fixes: asan_heap-oob_22b30d4_39_038.sgi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3b20ed85489a14cb5028c873d06960dbc5eef88a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer
045670a6f7 avcodec/hevc_ps: Check default display window bitstream and skip if invalid
Fixes Ticket4035

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 852aaead1fc294bcb63a1f9e384e781f6e51ded6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Christophe Gisquet
30a0622a5d avcodec/tiffenc: properly compute packet size
The bytes per row is a better indication of it.

Helps resolving ticket #3874 by fixing ffmpeg's encoder and transforming
the issue in a issue with non-compliant decoders. ffmpeg's one is ok,
but unfortunately, many others aren't handling correctly unusual chroma
samplings.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0e8bfd3c934768f9812dd20d71fa4709de54186d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:13 +01:00
Christophe Gisquet
f3d34cff76 utvideoenc: properly set slice height/last line
Mimic decoder and obey sampling.

Does not affect fate tests for utvideo.
Fixes ticket #3949.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb530dda7d76790b08ee3b7f67e251f3ce48c359)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:13 +01:00
Michael Niedermayer
3b7db9c4f5 Merge commit 'e443165c323406d01da7e7930f042d265d01fb35' into release/2.4
* commit 'e443165c323406d01da7e7930f042d265d01fb35':
  imc: fix order of operations in coefficients read

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:21:40 +01:00
Michael Niedermayer
0ddcee172e Merge commit '0b41eeac45fb7f7ad6d3f4fc846b00d108824b0b' into release/2.4
* commit '0b41eeac45fb7f7ad6d3f4fc846b00d108824b0b':
  hevc_mvs: make sure to always initialize the temporal MV fully

Conflicts:
	libavcodec/hevc_mvs.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:06:25 +01:00
Michael Niedermayer
13ecdb06f8 Merge commit 'de31f857077a52714f3a2f2e92ac037d42d37769' into release/2.4
* commit 'de31f857077a52714f3a2f2e92ac037d42d37769':
  hevc_mvs: initialize the temporal MV in case of missing ref

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:05:46 +01:00
Vittorio Giovara
e443165c32 imc: fix order of operations in coefficients read
Reported-by: Ruoyu <liangry@ucweb.com>
2014-10-15 14:51:19 +01:00
Anton Khirnov
0b41eeac45 hevc_mvs: make sure to always initialize the temporal MV fully
The spec requires this.

Fixes uninitialized reads on some samples.

Remove now unnecessary initialization of the whole merge candidate list.
2014-10-08 12:31:08 +01:00
Anton Khirnov
de31f85707 hevc_mvs: initialize the temporal MV in case of missing ref
The caller expects the MV to always be initialized.
2014-10-08 12:30:54 +01:00
Michael Niedermayer
84d26ab6eb avcodec/on2avc: Check number of channels
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_7_asan_heap-oob_4da4f3_173_Xmen_avc_500.vp6

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 550f3e9df3410b3dd975e590042c0d83e20a8da3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
1d99adc953 avcodec/hevc: fix chroma transform_add size
Fixes accessing misaligned address
Fixes: signal_sigsegv_1feb99c_10_signal_sigsegv_2d1d35c_79_cov_691940146_EXT_A_ericsson_3.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df8394c693d3c1e13b7ebf6af01c5e55321cf952)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
67991f3a3e avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de3043f012ad0f1ef11027d9488067e6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
eefc3ca7be avcodec/vorbisdec: Fix off by 1 error in ptns_to_read
Fixes read of uninitialized memory
Fixes: asan_heap-uaf_18dac2b_9_asan_heap-uaf_22eb375_208_beta3_test_small.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c50704ebf1777bee76772c4835d9760b3721057)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
506368f563 avcodec/svq3: Dont memcpy AVFrame
This avoids out of array accesses

Fixes: asan_heap-uaf_21f42e4_9_asan_heap-uaf_21f42e4_278_gl2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 075a165d2715837d125a9cc714fb430ccf6c9d6b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
c0c24bc9b3 avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9d62335e89d118a5b4e202edf39d905)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
b89f279cd6 avcodec/qpeg: fix off by 1 error in MV bounds check
Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1ca26d0fff3a3baf61a40207032143f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
7f90eef87a avcodec/gifdec: factorize interleave end handling out
also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864be8fb9653643519dea1c6492f1dde57)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
71f0a3c4ad avcodec/cinepak: fix integer underflow
Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c506957f40aafd794e06de1a7e341e9d5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
d9bef14e41 avcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020c2dbf0201fe7df3dad9e7e0b2e1bb6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
4b4ed88e89 avcodec/pngdec: Calculate MPNG bytewidth more defensively
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934a29df05c7af65aef2a480b15f572c4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
f6476944e1 avcodec/tiff: more completely check bpp/bppcount
Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa419aa5d320540d5a1b3f8fd9b82ab7e5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
03d30d4c2c avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f21b761ca15dbb470cd619a1ebf86c3e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00
Michael Niedermayer
853a27e345 avcodec/h263dec: Fix decoding messenger.h263
Fixes http://samples.mplayerhq.hu/V-codecs/h263/h263-raw/messenger.h263
Fixes regression since b239f3f6

Found-by: Josh Allmann
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d225b0f7aaa65eafccc87165130e1c4bab71708b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 23:52:29 +02:00