1271 Commits

Author SHA1 Message Date
Michael Niedermayer
6086a4d74d Merge commit '747fbe0c212b81952bb27ec7b99fa709081e2d63' into release/1.1
* commit '747fbe0c212b81952bb27ec7b99fa709081e2d63':
  roqvideodec: fix a potential infinite loop in roqvideo_decode_frame().
  mp3dec: Fix VBR bit rate parsing
  wmaprodec: return an error, not 0, when the input is too small.
  vmdaudio: fix invalid reads when packet size is not a multiple of chunk size
  h264: check for luma and chroma bit dept being equal
  Prepare for 9.4 Release

Conflicts:
	RELEASE
	libavcodec/vmdav.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-14 02:49:31 +01:00
Luca Barbato
146eac0a0c h264: check for luma and chroma bit dept being equal
The decoder assumes a single bit depth for all the planes
while the specification allows different bit depths for luma
and chroma.

Avoid the possible problems described in CVE-2013-2277

CC: libav-stable@libav.org
(cherry picked from commit 4987faee78b9869f8f4646b8dd971d459df218a5)

Conflicts:

	libavcodec/h264.c
2013-03-09 18:01:09 +01:00
Michael Niedermayer
992957ac30 Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  update Changelog
  h264: set ref_count to 0 for intra slices.
  h264: on reference overflow, reset the reference count to 0, not 1.
  flvdec: Check the return value of a malloc

Conflicts:
	Changelog
	libavcodec/h264.c
	libavformat/flvdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-03 12:15:14 +01:00
Anton Khirnov
704952fee5 h264: set ref_count to 0 for intra slices.
CC:libav-stable@libav.org
(cherry picked from commit 437211ae73ef1ed8285b4fed7620502ea4999e11)

Fixes deadlocks waiting for non-existing references with some fuzzed files.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-02 11:20:59 +01:00
Anton Khirnov
b6f5a1ca58 h264: on reference overflow, reset the reference count to 0, not 1.
Since decode_slice_header() returns before the reference lists are
constructed, there are zero valid references.

CC:libav-stable@libav.org
(cherry picked from commit 668e16a0dd1ff56d4beeff5c658d8a2a08dbfac8)

Conflicts:

	libavcodec/h264.c
2013-03-02 11:20:59 +01:00
Michael Niedermayer
a642be972d h264: ensure that get_format() is called when changing format but not otherwise.
Fixes Ticket2288

Tested-by: Stefano Pigozzi <stefano.pigozzi@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 04220b473e9d7b22d737405348f9904f22bcfb96)

Conflicts:

	libavcodec/h264.c
2013-02-22 22:53:11 +01:00
Michael Niedermayer
c6f59b95c5 h264: avoid calling get_format() multiple times
Some applications do not like that.
Fixes VDA
Reduces noise for VDPAU

Tested-by: Guillaume POIRIER <poirierg@gmail.com>
Tested-by: Carl Eugen Hoyos <cehoyos@ag.or.at>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dece584a639c9fd61a72e21800815e8397b3b617)

Conflicts:

	libavcodec/h264.c
2013-02-18 18:14:11 +01:00
Michael Niedermayer
2ac6b573a4 h264: Reset last_pocs in case of reference or frame number inconsistencies
This prevents faulty increasing of has_b_frames
Should fix Ticket 2062

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c230af9bccc3cadb373f9007ba14fffb6c2acc75)
2013-02-14 13:33:44 +01:00
Michael Niedermayer
eaa9d2cd6b h264: skip error concealment when SPS and slices are mismatching
Fixes out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 695af8eed642ff0104834495652d1ee784a4c14d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:53:12 +01:00
Michael Niedermayer
d3bec24739 h264: Only apply error concealment if theres a frame
Without any correctly decoded slices, there can be no frame.

Fixes out of array reads

Found-by: Rafaël Carré
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 60af6c3138dc501a647bc69b374d5d33d5d86ab5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:53:03 +01:00
Michael Niedermayer
3ef1538121 h264: check the pixel format directly and force a reinit on mismatches.
The existing checks are insufficient to detect a pixel format
changes in case of some damaged streams.
Fixes inconsistency and later out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11c99c78bafa77f679a1a3ba06ad00984b9a4cae)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-05 16:50:52 +01:00
Michael Niedermayer
002ad7cd39 Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  fate: update ref after rv30_loop_filter fix
  rv30: fix masking in rv30_loop_filter()
  libcdio: support recent cdio-paranoia
  theora: Skip zero-sized headers
  h264: add 3 pixels below for subpixel filter wait position
  h264: fix ff_generate_sliding_window_mmcos() prototype.
  h264: don't clobber mmco opcode tables for non-first slice headers.

Conflicts:
	configure
	libavcodec/h264_refs.c
	tests/ref/fate/filter-delogo
	tests/ref/fate/rv30

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-19 15:54:36 +01:00
Michael Niedermayer
30f0cd2f1e h264: fix () placement
Fixes null pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c13e4e288c84ba0629ead15e1460c0e498ee2bce)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-18 05:14:31 +01:00
Ronald S. Bultje
9d60f608af h264: add 3 pixels below for subpixel filter wait position
If the motion vector is at a subpixel position, we need 3 pixels below
the motion vector's wholepel position available, not 2, since the MC
filter is a sixtap filter for the hpel position, and then a bilin filter
for the qpel position.

This patch fixes highly irreproducible (0.1%) fate failures in frame 2
and 4 of h264-conformance-cama2_vtc_b (e.g. first P-frame, first field,
last line of MB x=40,y=2 and second field and last lines of MBs x=39-40,
y=3). These used pre-loopfilter instead of post-loopfilter data because
the await_progress() waited for one line too little in that field, and
the motion vector of these particular MBs happened to align exactly to a
position where that demonstrates the bug.

CC: libav-stable@libav.org

(cherry picked from commit fb845ffdd335a1efd6dfd43e8adeb530397b348e)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-15 22:20:10 +01:00
Anton Khirnov
6a4803a6a9 h264: fix ff_generate_sliding_window_mmcos() prototype.
It's been returning an error value since
bad446e251405dc250c3cbee199072e083a1e4b9

Also check for the errors it returns.
(cherry picked from commit ea382767ad2191acbe97e90624059723e15f0e4b)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-15 22:19:42 +01:00
Ronald S. Bultje
c3b67720f9 h264: don't clobber mmco opcode tables for non-first slice headers.
Clobbering these tables will temporarily clobber the template used
as a basis for other threads to start decoding from. If the other
decoding thread updates from the template right at that moment,
subsequent threads will get invalid (or, usually, none at all) mmco
tables. This leads to invalid reference lists and subsequent decode
failures.

Therefore, instead, decode the mmco tables only for the first slice in
a field or frame. For other slices, decode the bits and ensure they
are identical to the mmco tables in the first slice, but don't ever
clobber the context state. This prevents other threads from using a
clobbered/invalid template as starting point for decoding, and thus
fixes decoding in these cases.

This fixes occasional (~1%) failures of h264-conformance-mr1_bt_a with
frame-multithreading enabled.

(cherry picked from commit bad446e251405dc250c3cbee199072e083a1e4b9)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-15 22:19:07 +01:00
Michael Niedermayer
1894302a44 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  cmdutils: update copyright year to 2013
  h264: check SPS entries directly to detect pixel format changes
  forgotten changelogs for 9_beta2

Conflicts:
	Changelog
	cmdutils.c
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-03 13:22:45 +01:00
Janne Grunau
9ac44ad9d0 h264: check SPS entries directly to detect pixel format changes
Comparing AVCodecContext.pix_fmt against the get_pixel_format() return
value has the side effect of calling the get_format() callback on each
slice. Users of the callback will probably handle hardware accelerator
initialization in the callback.
2013-01-03 11:09:00 +01:00
Janne Grunau
e9fd51b0d6 h264: check SPS entries directly to detect pixel format changes
Comparing AVCodecContext.pix_fmt against the get_pixel_format() return
value has the side effect of calling the get_format() callback on each
slice. Users of the callback will probably handle hardware accelerator
initialization in the callback.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-02 02:28:57 +01:00
Michael Niedermayer
d69238e991 Merge commit 'f3298f12997eb4b7ad203766f768f92e3dd72a2a'
* commit 'f3298f12997eb4b7ad203766f768f92e3dd72a2a':
  Return proper error code after av_log_ask_for_sample()
  configure: cosmetics: Separate hwaccel dependencies from decoders/encoders
  oggdec: check memory allocation

Conflicts:
	configure
	libavcodec/pictordec.c
	libavformat/anm.c
	libavformat/oggdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-24 15:09:58 +01:00
Diego Biurrun
f3298f1299 Return proper error code after av_log_ask_for_sample() 2012-12-23 18:56:56 +01:00
Michael Niedermayer
985aa0be82 h264: Detect POC inconsistencies and try to handle them reasonably
Improves the file from Ticket2050

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-22 18:35:43 +01:00
Michael Niedermayer
a41bf09d9c Merge commit '6906b19346ae8a330bfaa1c16ce535be10789723'
* commit '6906b19346ae8a330bfaa1c16ce535be10789723':
  lavc: add missing files for arm
  lavc: introduce VideoDSPContext

Conflicts:
	configure
	libavcodec/arm/dsputil_init_armv5te.c
	libavcodec/dsputil.c
	libavcodec/dsputil.h
	libavcodec/dsputil_template.c
	libavcodec/h264.c
	libavcodec/mpegvideo.h
	libavcodec/mpegvideo_enc.c
	libavcodec/x86/dsputil_mmx.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-21 17:18:43 +01:00
Ronald S. Bultje
8c53d39e7f lavc: introduce VideoDSPContext
Move some functions from dsputil. The idea is that videodsp contains
functions that are useful for a large and varied set of video decoders.
Currently, it contains emulated_edge_mc() and prefetch().

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2012-12-20 13:40:45 +01:00
Michael Niedermayer
a9275b4f69 h264: Fix code to also handle pix format changes.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-20 12:56:47 +01:00
Michael Niedermayer
8525fa7c2c h264: remove unused variable
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-20 04:22:21 +01:00
Michael Niedermayer
ca4dd3810e h264: use must_reinit to simplify code
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer
af0fafcb6f h264: decode_update_thread_context() copy parameter sets before using them
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer
e33811bd26 h264: remove reschange blocking
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer
c2dfb1e37c h264: support frame size changes with multi threading
Based on code by Janne Grunau

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer
8ea4a5533f h264: move h264_set_parameter_from_sps() call up
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer
afc03268b4 h264: move list_count and current_slice reset to flush_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer
971c469a0d h264: split flush_change() out of flush_dpb()
Based on a patch by Janne Grunau
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer
25a0af51da h264: factor get_pixel_format() out
Based on patch by Janne Grunau

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 21:46:11 +01:00
Michael Niedermayer
642a655f7d Merge commit 'f1d8763a02b5fce9a7d9789e049d74a45b15e1e8'
* commit 'f1d8763a02b5fce9a7d9789e049d74a45b15e1e8':
  mpegvideo: allocate scratch buffers after linesize is known

Conflicts:
	libavcodec/mpegvideo.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-19 15:44:59 +01:00
Janne Grunau
9e696d2e5f h264: support frame parameter changes during frame-mt
Fixes CVE-2012-2782.
2012-12-18 19:55:10 +01:00
Janne Grunau
f1d8763a02 mpegvideo: allocate scratch buffers after linesize is known
Since we can't know which stride a custom get_buffer() implementation is
going to use we have to allocate this scratch buffers after the linesize
is known. It was pretty safe for 8 bit per pixel pixel formats since we
always allocated memory for up to 16 bits per pixel. It broke hoever
with cmdutis.c's alloc_buffer() and high pixel bit depth since it
allocated larger edges than mpegvideo expected.

Fixes fuzzed sample nasa-8s2.ts_s244342.
2012-12-18 19:48:30 +01:00
Michael Niedermayer
14f79ba18f h264: remove redundant parts of old slice in extradata code.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-18 14:52:44 +01:00
Michael Niedermayer
99321d1b03 h264: merge old and new "slice in extradata" checks
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-18 14:50:12 +01:00
Michael Niedermayer
09b4ae8331 Merge commit '61c6eef5456f2bc8b1dc49a0a759c975551cea29'
* commit '61c6eef5456f2bc8b1dc49a0a759c975551cea29':
  h264: prevent decoding of slice NALs in extradata
  doxy: Clarify what avpriv_set_pts_info does

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-18 14:38:10 +01:00
Janne Grunau
61c6eef545 h264: prevent decoding of slice NALs in extradata
It is not posible to call get_buffer during frame-mt codec
initialization. Libavformat might pass huge amounts of data as
extradata after parsing broken files. The 'extradata' for the fuzzed
sample sample_varPAR_s5374_r001-02.avi is 2.8M large and contains
multiple slices.
2012-12-18 11:01:14 +01:00
Michael Niedermayer
7973a07590 h264: Improve first slice and slice type checks
This prevents a null pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-17 01:08:19 +01:00
Michael Niedermayer
dde4832b64 Merge commit '27c8337e595a058347150269d5c2c48281e4285b'
* commit '27c8337e595a058347150269d5c2c48281e4285b':
  h264-mt: handle NAL_DPAs before calling ff_thread_finish_setup
  lavr: move AudioMix struct definition to audio_mix.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-16 13:09:30 +01:00
Michael Niedermayer
d7599bd8e2 h264: dont mess with frame gaps on second fields.
Fixes assertion failure

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-16 00:28:36 +01:00
Janne Grunau
27c8337e59 h264-mt: handle NAL_DPAs before calling ff_thread_finish_setup
Since a NAL_DPA can start a new frame it has to be handled before
ff_thread_finish_setup is called.
2012-12-15 19:06:37 +01:00
Michael Niedermayer
a01fe55077 Merge commit 'c0dc57f1264dad1e121772d03abdb9e14ed8857f'
* commit 'c0dc57f1264dad1e121772d03abdb9e14ed8857f':
  asyncts: merge two conditions
  x86inc: fully concatenate tokens to fix macro expansion for nasm
  h264: initialize frame-mt context copies properly

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-14 15:43:46 +01:00
Michael Niedermayer
3b5c0f5e36 h264: remove low_delay/has_b_frame setting code from nal loop
This code is now executed in h264_set_parameter_from_sps()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-14 15:22:19 +01:00
Michael Niedermayer
b9d887c225 Merge commit '072be3e8969f24113d599444be4d6a0ed04a6602'
* commit '072be3e8969f24113d599444be4d6a0ed04a6602':
  h264: set parameters from SPS whenever it changes
  asyncts: cosmetics: reindent

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-14 15:17:51 +01:00
Janne Grunau
072be3e896 h264: set parameters from SPS whenever it changes
Fixes a crash in the fuzzed sample sample_varPAR.avi_s26638 with
alternating bit depths.
2012-12-13 21:02:42 +01:00
Janne Grunau
0eae920c3c h264: initialize frame-mt context copies properly 2012-12-13 21:02:42 +01:00