30349 Commits

Author SHA1 Message Date
Michael Niedermayer
f3296b9454 avcodec/h264: Undefined behavior (left shift of 12852653 by 8 places cannot be represented in type 'int')
Fixes: asan_heap-oob_84f75d_8_asan_heap-oob_a2a00a_341_mbc.ts

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-08 05:35:10 +02:00
Michael Niedermayer
9db9b209e3 avcodec/mmvideo: Fix undefined behavior (left shift of negative value)
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-08 05:35:01 +02:00
Michael Niedermayer
aba873bbd0 Merge commit '9a03c2323593173a201cb75edd1b49887cf811ed'
* commit '9a03c2323593173a201cb75edd1b49887cf811ed':
  h263dec: Force padding bug workaround for H.263.

This is not merged as it breaks a good part of the error concealment/resilience for H.263
Also, messenger.h263 plays fine in ffmpeg.
If anyone has any other h263 files that do not work, please open an issue on trak or
mail me!

See: d225b0f7aaa65eafccc87165130e1c4bab71708b
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-07 13:00:08 +02:00
Michael Niedermayer
33c8500db4 Merge commit 'd208d1eba3799c58fd6d3602d31de3e686f14aec'
* commit 'd208d1eba3799c58fd6d3602d31de3e686f14aec':
  hevc_mvs: make sure to always initialize the temporal MV fully

Conflicts:
	libavcodec/hevc_mvs.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-07 12:29:07 +02:00
Michael Niedermayer
018ca8b7f6 Merge commit '79a60c8e779242b5ba4c531b2c706c871e8e6420'
* commit '79a60c8e779242b5ba4c531b2c706c871e8e6420':
  hevc_mvs: initialize the temporal MV in case of missing ref

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-07 12:21:43 +02:00
Josh Allmann
9a03c23235 h263dec: Force padding bug workaround for H.263.
Fixes decoding of http://samples.mplayerhq.hu/V-codecs/h263/h263-raw/messenger.h263

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-10-07 07:30:32 +00:00
Anton Khirnov
d208d1eba3 hevc_mvs: make sure to always initialize the temporal MV fully
The spec requires this.

Fixes uninitialized reads on some samples.

Remove now unnecessary initialization of the whole merge candidate list.
2014-10-07 07:13:01 +00:00
Anton Khirnov
79a60c8e77 hevc_mvs: initialize the temporal MV in case of missing ref
The caller expects the MV to always be initialized.
2014-10-07 07:12:55 +00:00
Michael Niedermayer
f0d1b3acdf avcodec/escape124: use av_freep(), do not leave stale pointers in memory
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-07 03:42:20 +02:00
Michael Niedermayer
1c5647f419 avcodec/proresenc_kostya: use av_freep(), do not leave stale pointers in memory
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-07 03:41:40 +02:00
Michael Niedermayer
c9eac8062e avcodec/avpacket: use av_freep(), do not leave stale pointers in memory
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-07 03:40:57 +02:00
Michael Niedermayer
b60938e4fd avcodec/avpacket: simplify freeing pkt->data
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-07 03:40:15 +02:00
Michael Niedermayer
a52384dcda Merge commit '153fadc390d05aa47e5e2c56290401898fe41a23'
* commit '153fadc390d05aa47e5e2c56290401898fe41a23':
  ff_get_format: fix infinite loop

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-07 02:15:26 +02:00
Michael Niedermayer
d47dd84391 Merge commit '577899a6458ccad9026eb268f10dc0b39c224c8d'
* commit '577899a6458ccad9026eb268f10dc0b39c224c8d':
  lavc: specify the behavior of av_lockmgr_register on failure.

Conflicts:
	libavcodec/avcodec.h
	libavcodec/utils.c
	libavcodec/version.h

See: a950edb472e8823e34832c7313ba447b2db76f27
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 21:08:55 +02:00
Rémi Denis-Courmont
153fadc390 ff_get_format: fix infinite loop
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-10-06 18:39:45 +00:00
Manfred Georg
577899a645 lavc: specify the behavior of av_lockmgr_register on failure.
The register function now specifies that the user callback should
leave things in the same state that it found them on failure but
that failure to destroy is ignored by the library.  The register
function is now explicit about its behavior on failure
(it unregisters the previous callback and destroys all mutex).

Signed-off-by: Manfred Georg <mgeorg@google.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-10-06 18:20:17 +00:00
Michael Niedermayer
67ddf21611 avcodec/vdpau: do not dereference hwctx before checking it for NULL
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 17:39:28 +02:00
Michael Niedermayer
0e57c05118 avcodec/vdpau: fix render2() check
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 16:23:36 +02:00
Michael Niedermayer
573d333010 Merge commit 'bef067f88c74190cdf7e76d12f02e12e069974aa'
* commit 'bef067f88c74190cdf7e76d12f02e12e069974aa':
  vdpau: check video surface and decoder capabilities

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 15:12:19 +02:00
Michael Niedermayer
a61899a0f1 Merge commit 'e3e158e81f0666b8fe66be9ce1cad63a535920e0'
* commit 'e3e158e81f0666b8fe66be9ce1cad63a535920e0':
  vdpau: add av_vdpau_bind_context()

Conflicts:
	doc/APIchanges
	libavcodec/vdpau.h
	libavcodec/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 15:04:03 +02:00
Michael Niedermayer
067d11bf71 Merge commit '502cde409ca5ee97ef70c2cdede88b9101746ff6'
* commit '502cde409ca5ee97ef70c2cdede88b9101746ff6':
  vdpau: force reinitialization when output resolution changes

Conflicts:
	libavcodec/vdpau.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 14:17:40 +02:00
Michael Niedermayer
ec6a855b3a avcodec/vdpau: warn if the user application has not setup avctx->hwaccel_context instead of potentially crashing
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 14:10:37 +02:00
Michael Niedermayer
0ba887bbf4 Merge commit '89ac99ba5f2dc9f69ad3bc294753930eb0b3e4a4'
* commit '89ac99ba5f2dc9f69ad3bc294753930eb0b3e4a4':
  vdpau: pass codec-specific parameters from hwaccel

Conflicts:
	libavcodec/vdpau.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 13:21:52 +02:00
Michael Niedermayer
8df41976b7 avcodec/vdpau_internal: add comment to #endif
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 12:47:08 +02:00
Michael Niedermayer
10b6d7462c Merge commit 'ce083282f0a8b7d63c4047c30b7bac498f9806dd'
* commit 'ce083282f0a8b7d63c4047c30b7bac498f9806dd':
  vdpau: common support for managing the VdpDecoder in avcodec

Conflicts:
	libavcodec/vdpau.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 12:38:26 +02:00
Michael Niedermayer
b64b719ad5 Merge commit 'fcc1022611f79c2f3aa2f392a5ce14c74be9c1d7'
* commit 'fcc1022611f79c2f3aa2f392a5ce14c74be9c1d7':
  vdpau: factor out common end-of-frame handling

Conflicts:
	libavcodec/vdpau.c
	libavcodec/vdpau_h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 12:09:17 +02:00
Michael Niedermayer
567c0dcee6 Merge commit '319424d25c53f82b87187ce03ed984d438f1bee6'
* commit '319424d25c53f82b87187ce03ed984d438f1bee6':
  vdpau: add helper for VDPAU to libav error codes conversion

Conflicts:
	libavcodec/vdpau.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-06 11:38:06 +02:00
Rémi Denis-Courmont
bef067f88c vdpau: check video surface and decoder capabilities
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-10-06 06:20:40 +00:00
Rémi Denis-Courmont
e3e158e81f vdpau: add av_vdpau_bind_context()
This function provides an explicit VDPAU device and VDPAU driver to
libavcodec, so that the application is relieved from codec specifics
and VdpDevice life cycle management.

A stub flags parameter is added for future extension. For instance, it
could be used to ignore codec level capabilities (if someone feels
dangerous).

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-10-06 06:20:40 +00:00
Rémi Denis-Courmont
502cde409c vdpau: force reinitialization when output resolution changes
This is necessary to recreate the decoder with the correct parameters,
as not all codecs invoke get_format() in this case.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-10-06 06:18:56 +00:00
James Almer
42111bb960 avcodec: remove obsolete FF_API_FAST_MALLOC cruft
Signed-off-by: James Almer <jamrial@gmail.com>
2014-10-05 17:09:44 -03:00
Benoit Fouet
904cfd257d avcodec/h264_mp4toannexb_bsf: reset the new IDR flag when SPS/PPS is seen.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-05 14:47:04 +02:00
Michael Niedermayer
436206c175 avcodec/webp: add assert to ensure palette is not larger than 256
it should not be possible to be larger as its stored as 8bit value
but we would be overwriting a stack buffer if it is ...

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-05 13:00:40 +02:00
Pascal Massimino
19fb476841 avcodec/webp: add optimization: use local palette with extra padding
for big enough pictures.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-05 12:47:44 +02:00
Rémi Denis-Courmont
89ac99ba5f vdpau: pass codec-specific parameters from hwaccel
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-10-05 10:41:31 +00:00
Rémi Denis-Courmont
ce083282f0 vdpau: common support for managing the VdpDecoder in avcodec
Using the not so new init and uninit callbacks, avcodec can now take
care of creating and destroying the VDPAU decoder instance.

The application is still responsible for creating the VDPAU device
and allocating video surfaces - this is necessary to keep video
surfaces on the GPU all the way to the output. But the application
will no longer needs to care about any codec-specific aspects.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-10-05 10:17:02 +00:00
Rémi Denis-Courmont
fcc1022611 vdpau: factor out common end-of-frame handling
Also add error handling.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-10-05 10:13:44 +00:00
Rémi Denis-Courmont
319424d25c vdpau: add helper for VDPAU to libav error codes conversion
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-10-05 10:08:53 +00:00
Luca Barbato
68211c70af vc1: Do not assume seek happens after decoding
If a seek is requested before the decoding start there is no
current picture.

CC: libav-stable@libav.org
2014-10-04 23:46:46 +02:00
Michael Niedermayer
550f3e9df3 avcodec/on2avc: Check number of channels
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_7_asan_heap-oob_4da4f3_173_Xmen_avc_500.vp6

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 22:15:54 +02:00
Mickaël Raulet
4ba6371a83 x86/hevc: get rid off packusdw for ssse3 compatibility
cherry picked from commit df8ebe304df453f26c28ff8f11d607f49b90a4c2

Fixes out of array access
Fixes: asan_stack-oob_1046454_9_asan_stack-oob_15a9e7c_170_WP_MAIN10_B_Toshiba_3.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 21:14:15 +02:00
Michael Niedermayer
df8394c693 avcodec/hevc: fix chroma transform_add size
Fixes accessing misaligned address
Fixes: signal_sigsegv_1feb99c_10_signal_sigsegv_2d1d35c_79_cov_691940146_EXT_A_ericsson_3.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 17:51:52 +02:00
Carl Eugen Hoyos
aaf65bc2a5 lavc/libx264: Fix compilation with libx264 version < 142. 2014-10-04 17:43:24 +02:00
Michael Niedermayer
9734a7a1de avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 14:53:13 +02:00
Michael Niedermayer
8c50704ebf avcodec/vorbisdec: Fix off by 1 error in ptns_to_read
Fixes read of uninitialized memory
Fixes: asan_heap-uaf_18dac2b_9_asan_heap-uaf_22eb375_208_beta3_test_small.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 03:13:39 +02:00
Michael Niedermayer
075a165d27 avcodec/svq3: Dont memcpy AVFrame
This avoids out of array accesses

Fixes: asan_heap-uaf_21f42e4_9_asan_heap-uaf_21f42e4_278_gl2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 00:21:17 +02:00
Michael Niedermayer
c727401aa9 avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 22:51:29 +02:00
Michael Niedermayer
dd3bfe3cc1 avcodec/qpeg: fix off by 1 error in MV bounds check
Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 21:09:40 +02:00
Michael Niedermayer
8f1457864b avcodec/gifdec: factorize interleave end handling out
also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 20:20:23 +02:00
Michael Niedermayer
e7e5114c50 avcodec/cinepak: fix integer underflow
Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 19:34:28 +02:00