Commit Graph

30654 Commits

Author SHA1 Message Date
Michael Niedermayer
61296d41e2 avcodec/h264: Check *log2_weight_denom
Fixes undefined behavior
Fixes: signal_sigsegv_14768d2_2248_cov_3629497219_h264_h264___pi_20070614T182942.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-18 03:45:59 +01:00
Michael Niedermayer
3281fa8925 avcodec/hevc_ps: Check diff_cu_qp_delta_depth
Fixes undefined behavior
Fixes: asan_static-oob_17aa046_582_cov_1577759978_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-18 02:19:53 +01:00
Michael Niedermayer
99f8c9e4d1 avcodec/hevc: move qp_block_mask to where its used
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-18 01:55:02 +01:00
Michael Niedermayer
e8714f6f93 avcodec/h264: Clear delayed_pic on deallocation
Fixes use of freed memory

Fixes: case5_av_frame_copy_props.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-17 21:55:45 +01:00
Michael Niedermayer
8aa8d12554 avcodec/hevc: clear filter_slice_edges() on allocation
This avoids use of uninitialized memory
Fixes: asan_static-oob_17aa046_582_cov_212287884_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-17 19:43:58 +01:00
Michael Niedermayer
7d593495e4 avcodec/dcadec: Check that the added xch channel isnt already there
Fixes null pointer dereference
Fixes: signal_sigsegv_369609d_623_cov_2008234281_ES_6.1_16bit.dts
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-17 15:33:59 +01:00
Michael Niedermayer
3ba1050292 avcodec/imc: dont read bits beyond the end
Fixes use of uninitialized memory

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-17 05:14:11 +01:00
Michael Niedermayer
ddd7dac7ae avcodec/utils: check the private context class
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-17 05:14:11 +01:00
Michael Niedermayer
3305acdc92 avcodec/indeo3: use signed variables to avoid underflow
Fixes out of array read
Fixes: signal_sigsegv_1b0a4da_1865_cov_2167818389_computer_anger.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-17 05:14:11 +01:00
Reimar Döffinger
254da44bf9 flacenc: calculate lower sum levels in-place.
Should improve cache usage and reduces stack usage.
Also reduces number of copies in case many levels
have the same number of bits.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
2014-12-17 04:27:09 +01:00
Michael Niedermayer
f3b5b139ad avcodec/h264: make the first field of H264Context an AVClass
Fixes use of freed memory
Fixes: asan_heap-uaf_3660f67_757_cov_1257014655_Hi422FR1_SONY_A.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-17 01:33:06 +01:00
Michael Niedermayer
3881606240 avcodec/utvideodec: Fix handling of slice_height=0
Fixes out of array accesses
Fixes: asan_heap-oob_25bcd7e_3783_cov_3553517262_utvideo_rgba_median.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-16 23:00:04 +01:00
Michael Niedermayer
211200e0c0 avcodec/xface: Add asserts to limit nb_words from becoming too large
Approved-by: Stefano Sabatini <stefasab@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-16 19:19:35 +01:00
Michael Niedermayer
93a5a16f13 avcodec/xface: correct the XFACE_MAX_* values
Fixes out of array access

Fixes: asan_stack-oob_32c12e5_2536_cov_2442316831_lena.xface
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-16 19:19:28 +01:00
Michael Niedermayer
3030fb7e0d avcodec/vmdvideo: Check len before using it in method 3
Fixes out of array access
Fixes: asan_heap-oob_4d23ba_91_cov_3853393937_128.vmd

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-16 16:27:45 +01:00
Derek Buitenhuis
402a61c919 snow_dwt: Don't try and free members of non-existent arrays
If allocation fails earlier on, and the next frame is processed,
the slice buffer could be left in a state where line and data_stack
have already been freed, or are otherwise null pointers.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2014-12-15 21:52:19 -05:00
Ronald S. Bultje
0a7964dca5 vp9/x86: save one register on 32bit idct32x32.
Fixes build on win32.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-16 02:51:26 +01:00
Michael Niedermayer
4f1923a91b avcodec/tiffenc: Consistently use ADD_ENTRY1()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 22:45:41 +01:00
Michael Niedermayer
4cffcbc809 Merge commit '10d4c5e55e5b23e165aa4c7723073ebe2c2e7da0'
* commit '10d4c5e55e5b23e165aa4c7723073ebe2c2e7da0':
  tiff: set the correct return value when check_size() fails

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 22:14:14 +01:00
Michael Niedermayer
07b4d4b1a2 Merge commit '3c27275c1309190f2d6ed69140b67d014215b6c9'
* commit '3c27275c1309190f2d6ed69140b67d014215b6c9':
  tiff: Check the check_size() return value and forward it

Conflicts:
	libavcodec/tiffenc.c

See: d50aa006fb
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 22:00:52 +01:00
Michael Niedermayer
41915d0b86 Merge commit '4690e01c3aaf495c87127e5dc74aa347197dbc0b'
* commit '4690e01c3aaf495c87127e5dc74aa347197dbc0b':
  prores: Evaluate all the quantizers

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 21:46:29 +01:00
Michael Niedermayer
38c3ffbd55 Merge commit 'd75190aa93206c02cb4f18a66d2c927d6a0198e1'
* commit 'd75190aa93206c02cb4f18a66d2c927d6a0198e1':
  mpegvideo: move REBASE_PICTURE where it is used

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 21:27:10 +01:00
Michael Niedermayer
0a3ffcfc82 Merge commit '63fcedcf01247f15b72c0d1f959bb3b612853ba8'
* commit '63fcedcf01247f15b72c0d1f959bb3b612853ba8':
  mpegvideo: remove unused function declaration

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 21:19:40 +01:00
Michael Niedermayer
b799ec7e0b avcodec/hevc: Print an error if luma_log2_weight_denom is out of range
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 21:12:46 +01:00
Michael Niedermayer
241943d3a2 Merge commit 'b2bece5e9a4dd96142962c6798e187c77b88bbfa'
* commit 'b2bece5e9a4dd96142962c6798e187c77b88bbfa':
  hevc: always clip luma_log2_weight_denom

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 21:06:20 +01:00
Michael Niedermayer
1307c11ce6 avcodec/avdct: support >8bit per sample dcts
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 18:25:00 +01:00
Vittorio Giovara
10d4c5e55e tiff: set the correct return value when check_size() fails
Only one instance affected and solved as other occurences.
2014-12-15 15:46:34 +01:00
Luca Barbato
3c27275c13 tiff: Check the check_size() return value and forward it
Also use the same type for add_entry and check_size.

Bug-Id: CID 700699
CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Vittorio Giovara <vittorio.giovarao@gmail.com>
2014-12-15 15:46:34 +01:00
Luca Barbato
4690e01c3a prores: Evaluate all the quantizers
Prevent an uninitialized data access.

CC: libav-stable@libav.org
Bug-Id: CID 703824 / CID 703825
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-12-15 15:46:34 +01:00
Vittorio Giovara
d75190aa93 mpegvideo: move REBASE_PICTURE where it is used
Drop an unused #undef from h264 decoder.
2014-12-15 15:46:34 +01:00
Vittorio Giovara
63fcedcf01 mpegvideo: remove unused function declaration 2014-12-15 15:46:34 +01:00
Vittorio Giovara
b2bece5e9a hevc: always clip luma_log2_weight_denom
Its value shall be between 0 and 7 according to the specifications.

CC: libav-stable@libav.org
Bug-Id: CID 1257502
2014-12-15 15:46:34 +01:00
Michael Niedermayer
228fded036 Merge commit 'bab8d73765238bfeea7a8dfd454f40ac213b3d22'
* commit 'bab8d73765238bfeea7a8dfd454f40ac213b3d22':
  vaapi: wrap codec specific functions in appropiate #ifs

Conflicts:
	libavcodec/vaapi.c

See: f2118d44c9
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 13:03:01 +01:00
Michael Niedermayer
d2f9b9da03 Merge commit '5a82ad644f281701eb22e0359fd732fac67aafa4'
* commit '5a82ad644f281701eb22e0359fd732fac67aafa4':
  svq1dec: Unbreak the scratch buffer allocation

Conflicts:
	libavcodec/svq1dec.c

See: 4213fc5b9e
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 12:23:27 +01:00
Michael Niedermayer
0c7733904e Merge commit 'bcbdeb318e0828cf00b458885ffa1e1596c85d06'
* commit 'bcbdeb318e0828cf00b458885ffa1e1596c85d06':
  asvenc: Forward the error instead of asserting on it

Conflicts:
	libavcodec/asvenc.c

Not merged, conditions which are impossible should be checked by assert*()
adding impossible to be true error checks has the potential to confuse

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 12:04:31 +01:00
Ronald S. Bultje
cae893f692 vp9/x86: sse2 MC assembly.
Also a slight change to the ssse3 code, which prevents a theoretical
overflow in the sharp filter.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 02:34:05 +01:00
Ronald S. Bultje
fd77fbb390 vp9/x86: 32bit and sse2 support for vp9 inverse transform assembly
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-15 00:38:05 +01:00
Michael Niedermayer
27f2e211aa Merge commit '559fa0d41b5cf8f7e1ba89d1ac70e74712eb2816'
* commit '559fa0d41b5cf8f7e1ba89d1ac70e74712eb2816':
  vdpau: add mapping for H.264 Constrained Baseline profile and fallback

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-14 22:23:53 +01:00
Michael Niedermayer
3f679f73a2 Merge commit '8502c1e9ff9c1dbb6e467630c048d098f4064021'
* commit '8502c1e9ff9c1dbb6e467630c048d098f4064021':
  vdpau: add mapping for H.264 Extended profile

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-14 22:14:58 +01:00
Michael Niedermayer
0b30d186f1 Merge commit '8b51bcfed75721e0d20832e5a90e1d585318c9d9'
* commit '8b51bcfed75721e0d20832e5a90e1d585318c9d9':
  vdpau: revector macro to reduce line span

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-14 22:07:41 +01:00
Thiago Santos
bab8d73765 vaapi: wrap codec specific functions in appropiate #ifs
Fix linking when only a subset of vaapi decoders is enabled.

Bug-Id: 760
CC: libav-stable@libav.org

Signed-off-by: Rémi Denis-Courmont <remi@remlab.net>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-12-14 22:04:38 +01:00
Michael Niedermayer
5a82ad644f svq1dec: Unbreak the scratch buffer allocation
The input packets are always assumed to be padded and
the av_fast_ family of function takes a pointer to a pointer.

Thanks to Nicolas Dufresne <nicolas.dufresne@collabora.com> for
a similar patch.

Introduced in 7b588bb691.

Bug-Id: 766
CC: libav-stable@libav.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-12-14 22:03:58 +01:00
Luca Barbato
bcbdeb318e asvenc: Forward the error instead of asserting on it
Bug-Id: CID 732997
CC: libav-stable@libav.org
2014-12-14 22:03:22 +01:00
Rémi Denis-Courmont
559fa0d41b vdpau: add mapping for H.264 Constrained Baseline profile and fallback
Old VDPAU drivers do not support this newly defined profile, so falling
back to Main profile is necessary for backward binary compatibility.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-12-14 18:30:03 +01:00
Rémi Denis-Courmont
8502c1e9ff vdpau: add mapping for H.264 Extended profile
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-12-14 18:29:27 +01:00
Rémi Denis-Courmont
8b51bcfed7 vdpau: revector macro to reduce line span
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-12-14 18:29:07 +01:00
Michael Niedermayer
ad2deb02e5 avcodec/xiph: mark returned header pointers const from avpriv_split_xiph_headers()
Reviewed-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-14 14:30:45 +01:00
Clément Bœsch
d823edee8d avcodec: fix wording in AVSubtitleRect doxy 2014-12-13 23:35:22 +01:00
Michael Niedermayer
5d0cfb58ef avcodec/dvdsubdec: Avoid strerror() due to thread saftey
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-13 15:15:07 +01:00
Michael Niedermayer
3ab82e9206 avcodec/libxvid_rc: Avoid strerror() for thread saftey
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-13 15:12:37 +01:00