995 Commits

Author SHA1 Message Date
Michael Niedermayer
0ca3dbbfce Merge tag 'n0.8.15' into release/0.7
FFmpeg 0.8.15 release

* tag 'n0.8.15': (49 commits)
  update for 0.8.15
  avcodec/ffv1enc: update buffer check for 16bps
  avcodec/dsputil: fix signedness in sizeof() comparissions
  avcodec/pngdsp: fix (un)signed type in end comparission
  matroska_read_seek: Fix used streams for subtitle index compensation
  jpeg2000: check log2_cblk dimensions
  avcodec/rpza: Perform pointer advance and checks before using the pointers
  update all trac links to use the trac subdomain
  doc/APIchanges: List merge commit hashes and version numbers
  apichanges: fix 2 wrong hashes
  avcodec/parser: reset indexes on realloc failure
  mpeg12dec: avoid reinitialization on PS changes when possible.
  mpegts: only reopen pmt_cb filter if its different from the previous.
  Autodetect idcin only if audio properties allow decoding.
  alacenc: Fix missing sign_extend()
  h264_cavlc: fix reading skip run
  Update changelog for 0.7.8 release
  aac: check the maximum number of channels
  oggdec: fix faulty cleanup prototype
  qdm2: check that the FFT size is a power of 2
  ...

Conflicts:
	Doxyfile
	RELEASE
	VERSION
	libavformat/matroskadec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-06 19:04:27 +02:00
Michael Niedermayer
0a41da3e9d Merge commit 'd785f6940144eb6ce4c24309ed034056b81395bc' into release/0.8
* commit 'd785f6940144eb6ce4c24309ed034056b81395bc':
  shorten: validate that the channel count in the header is not <= 0
  matroskadec: request a read buffer for the wav header
  h264: check for luma and chroma bit depth being equal
  xxan: fix invalid memory access in xan_decode_frame_type0()
  wmadec: require block_align to be set.

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 17:22:53 +02:00
Reinhard Tartler
7d4c38d58d h264: check for luma and chroma bit depth being equal
The decoder assumes a single bit depth for all the planes while
the specification allows different bit depths for luma and chroma.

Avoid the possible problems described in CVE-2013-2277

Conflicts:
	libavcodec/h264.c
2013-05-09 11:20:11 +02:00
Michael Niedermayer
7378101d41 Merge branch 'release/0.8' into release/0.7
* release/0.8: (92 commits)
  Update for 0.8.13
  pngdec/filter: dont access out of array elements at the end
  aacdec: check channel count
  vqavideo: check chunk sizes before reading chunks
  eamad: fix out of array accesses
  roqvideodec: check dimensions validity
  qdm2: check array index before use, fix out of array accesses
  alsdec: check block length
  huffyuvdec: Skip len==0 cases
  huffyuvdec: Check init_vlc() return codes.
  Update changelog for 0.7.7 release
  mpeg12: do not decode extradata more than once.
  indeo4/5: check empty tile size in decode_mb_info().
  dfa: improve boundary checks in decode_dds1()
  indeo5dec: Make sure we have had a valid gop header.
  rv34: error out on size changes with frame threading
  rtmp: fix buffer overflows in ff_amf_tag_contents()
  rtmp: fix multiple broken overflow checks
  Revert "h264: allow cropping to AVCodecContext.width/height"
  h264: check ref_count validity for num_ref_idx_active_override_flag
  ...

Conflicts:
	Doxyfile
	RELEASE
	VERSION
	libavcodec/rv34.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-18 00:56:01 +01:00
Michael Niedermayer
56cc629a64 Revert "h264: allow cropping to AVCodecContext.width/height"
This reverts commit a2ae183a382f063c5403922b5151d865ce7252a2.

This removes a duplicate hunk

Found-by: Joakim Plate <elupus@ecce.se>
2013-01-19 13:34:41 +01:00
Michael Niedermayer
685321e4bd Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7:
  h264: check ref_count validity for num_ref_idx_active_override_flag
  h264: check context state before decoding slice data partitions
  oggdec: free the ogg streams on read_header failure
  oggdec: check memory allocation
  Fix uninitialized reads on malformed ogg files.
  rtsp: Recheck the reordering queue if getting a new packet
  alacdec: do not be too strict about the extradata size
  h264: fix sps parsing for SVC and CAVLC 4:4:4 Intra profiles
  h264: check sps.log2_max_frame_num for validity
  ppc: always use pic for shared libraries
  h264: enable low delay only if no delayed frames were seen
  lavf: avoid integer overflow in ff_compute_frame_duration()

Conflicts:
	libavformat/oggdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-17 03:16:46 +01:00
Michael Niedermayer
3f1a58db6f Merge commit 'b143844ea0f6246e0d5a938d743e2e8a98453bec' into release/0.8
* commit 'b143844ea0f6246e0d5a938d743e2e8a98453bec': (22 commits)
  aacdec: Fix an off-by-one overwrite when switching to LTP profile from MAIN.
  vp6: properly fail on unsupported feature
  h264: Fix parameters to ff_er_add_slice() call
  flacenc: ensure the order is within the min/max range in LPC order search
  yuv4mpeg: reject unsupported codecs
  vp8: reset loopfilter delta values at keyframes.
  vp56: release frames on error
  vp56: make parse_header return standard error codes
  ivi_common: check that scan pattern is set before using it.
  Update RELEASE file for 0.7.7
  tiffenc: Check av_malloc() results.
  mpegaudiodec: fix short_start calculation
  h264: avoid stuck buffer pointer in decode_nal_units
  yuv4mpeg: return proper error codes.
  smacker audio: sign-extend the initial 16-bit predicted value
  vf_pad: don't give up its own reference to the output buffer.
  avidec: return 0, not packet size from read_packet().
  wmapro: prevent division by zero when sample rate is unspecified
  alsdec: fix number of decoded samples in first sub-block in BGMC mode.
  alsdec: remove dead assignments
  ...

Conflicts:
	RELEASE
	libavformat/avidec.c
	libavformat/yuv4mpeg.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-17 03:03:39 +01:00
Michael Niedermayer
597d709eb4 Merge commit 'aa45b90804ab21175b8c116bd8e5eb4b4e85fbcb' into release/0.8
* commit 'aa45b90804ab21175b8c116bd8e5eb4b4e85fbcb': (22 commits)
  alsdec: Check k used for rice decoder.
  cavsdec: check for changing w/h.
  avidec: use actually read size instead of requested size
  wmaprodec: check num_vec_coeffs for validity
  lagarith: check count before writing zeros.
  indeo5: check tile size in decode_mb_info().
  indeo5: prevent null pointer dereference on broken files
  indeo: check for invalid motion vectors
  indeo: clear allocated band buffers
  indeo: check custom Huffman tables for errors
  dfa: add some checks to ensure that decoder won't write past frame end
  dfa: check that the caller set width/height properly.
  bytestream: add a new set of bytestream functions with overread checking
  avsdec: Set dimensions instead of relying on the demuxer.
  lavfi: avfilter_merge_formats: handle case where inputs are same
  rv34: use AVERROR return values in ff_rv34_decode_frame()
  h263: Add ff_ prefix to nonstatic symbols
  eval: fix swapping of lt() and lte()
  bmpdec: only initialize palette for pal8.
  vc1dec: add flush function for WMV9 and VC-1 decoders
  ...

Conflicts:
	libavcodec/avs.c
	libavcodec/mpegvideo_enc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-17 02:56:12 +01:00
Janne Grunau
dd0c5e0fa9 h264: check ref_count validity for num_ref_idx_active_override_flag
Fixes segfault in the fuzzed sample bipbop234.ts_s226407.
CC: libav-stable@libav.org
(cherry-picked from commit 6e5cdf26281945ddea3aaf5eca4d127791f23ca8)
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
2013-01-12 19:36:38 +01:00
Janne Grunau
ad02537746 h264: check context state before decoding slice data partitions
Fixes mov_h264_aac__Demo_FlagOfOurFathers.mov.SIGSEGV.4e9.656.

Found-by: Mateusz "j00ru" Jurczyk
CC: libav-stable@libav.org
(cherry-picked from commit c1fcf563b13051f280db169ba41c6a1b21b25e08)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:36:38 +01:00
Janne Grunau
5fa739e685 h264: enable low delay only if no delayed frames were seen
Dropping frames is undesirable but that is the only way by which the
decoder could return to low delay mode. Instead emit a warning and
continue with delayed frames.
Fixes a crash in fuzzed sample nasa-8s2.ts_s20033 caused by a larger
than expected has_b_frames value. Low delay keeps getting re-enabled
from a presumely broken SPS.

CC: libav-stable@libav.org
(cherry picked from commit 706acb558a38eba633056773280155d66c2f4b24)

Conflicts:

	libavcodec/h264.c
2013-01-12 19:32:24 +01:00
Reinhard Tartler
ce8910d861 h264: Fix parameters to ff_er_add_slice() call
s->mb_x is reset to zero a couple of lines above. It does not make
sense to call ff_er_add_slice() with 0 as endx when the end of the
macroblock row was reached. Fixes unnecessary and counterproductive
error resilience in https://bugzilla.libav.org/show_bug.cgi?id=394.

(cherry picked from commit e6160bda98641b7d4f86de15761ad2a962f21a36)

Conflicts:

	libavcodec/h264.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:

	libavcodec/h264.c
2013-01-12 19:22:22 +01:00
Jindřich Makovička
11ecd8574a h264: avoid stuck buffer pointer in decode_nal_units
When decode_nal_units() previously encountered a NAL_END_SEQUENCE,
and there are some junk bytes left in the input buffer, but no start codes,
buf_index gets stuck 3 bytes before the end of the buffer.

This can trigger an infinite loop in the caller code, eg. in
try_decode_trame(), as avcodec_decode_video() then keeps returning zeroes,
with 3 bytes of the input packet still available.

With this change, the remaining bytes are skipped so the whole packet gets
consumed.

CC:libav-stable@libav.org

Signed-off-by: Jindřich Makovička <makovick@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 1a8c6917f68f7378465e18f7615762bfd22704c2)

Conflicts:

	libavcodec/h264.c
2013-01-04 07:43:38 +01:00
Mans Rullgard
a2ae183a38 h264: allow cropping to AVCodecContext.width/height
Override the frame size from the SPS with AVCodecContext values
if the latter specify a size smaller by less than one macroblock.
This is required for correct cropping of MOV files from Canon cameras.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 30f515091c323da59c0f1b533703dedca2f4b95d)

Conflicts:

	libavcodec/h264.c
2013-01-04 07:43:20 +01:00
Michael Niedermayer
e28814e0e1 Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7:
  vorbis: Validate that the floor 1 X values contain no duplicates.
  vorbisenc: check all allocations for failure
  lavfi: avfilter_merge_formats: handle case where inputs are same
  alsdec: check opt_order.
  lavf: don't segfault when a NULL filename is passed to avformat_open_input()
  mpegvideo: Don't use ff_mspel_motion() for vc1
  imgconvert: avoid undefined left shift in avcodec_find_best_pix_fmt
  nuv: check RTjpeg header for validity
  vc1dec: add flush function for WMV9 and VC-1 decoders
  ffmpeg: fix -force_key_frames
  mov: set AVCodecContext.width/height for h264
  h264: allow cropping to AVCodecContext.width/height

Conflicts:
	libavcodec/mpegvideo_common.h
	libavcodec/nuv.c
	libavcodec/vorbisenc.c
	libavfilter/formats.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-10-16 17:57:12 +02:00
Mans Rullgard
b102d5d97d h264: allow cropping to AVCodecContext.width/height
Override the frame size from the SPS with AVCodecContext values
if the latter specify a size smaller by less than one macroblock.
This is required for correct cropping of MOV files from Canon cameras.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 30f515091c323da59c0f1b533703dedca2f4b95d)

Conflicts:

	libavcodec/h264.c
(cherry picked from commit e1608014c50eeb9f4744a53de0794eb6bb1269a2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-06-10 09:54:22 +02:00
Michael Niedermayer
a56b07b5dc Merge branch 'release/0.8' into release/0.7
* release/0.8:
  Update RELEASE file for 0.7.6
  Update changelog for 0.7.6 release
  ea: check chunk_size for validity.
  png: check bit depth for PAL8/Y400A pixel formats.
  x86: fix build with gcc 4.7
  qdm2: clip array indices returned by qdm2_get_vlc().
  kmvc: Check palsize.
  aacsbr: prevent out of bounds memcpy().
  rtpdec_asf: Fix integer underflow that could allow remote code execution
  dpcm: ignore extra unpaired bytes in stereo streams.
  tqi: Pass errors from the MB decoder
  h264: Add check for invalid chroma_format_idc
  adpcm: ADPCM Electronic Arts has always two channels
  h263dec: Disallow width/height changing with frame threads.
  vqavideo: return error if image size is not a multiple of block size
  celp filters: Do not read earlier than the start of the 'out' vector.
  motionpixels: Clip YUV values after applying a gradient.
  h263: more strictly forbid frame size changes with frame-mt.
  h264: additional protection against unsupported size/bitdepth changes.
  Update for 0.8.11

Conflicts:
	Doxyfile
	RELEASE
	VERSION

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-06-04 13:12:41 +02:00
Michael Niedermayer
64bc5f3bf7 Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7:
  Update RELEASE file for 0.7.6
  Update changelog for 0.7.6 release
  ea: check chunk_size for validity.
  png: check bit depth for PAL8/Y400A pixel formats.
  x86: fix build with gcc 4.7
  qdm2: clip array indices returned by qdm2_get_vlc().
  kmvc: Check palsize.
  aacsbr: prevent out of bounds memcpy().
  rtpdec_asf: Fix integer underflow that could allow remote code execution
  dpcm: ignore extra unpaired bytes in stereo streams.
  tqi: Pass errors from the MB decoder
  h264: Add check for invalid chroma_format_idc
  adpcm: ADPCM Electronic Arts has always two channels
  h263dec: Disallow width/height changing with frame threads.
  vqavideo: return error if image size is not a multiple of block size
  celp filters: Do not read earlier than the start of the 'out' vector.
  motionpixels: Clip YUV values after applying a gradient.
  h263: more strictly forbid frame size changes with frame-mt.
  h264: additional protection against unsupported size/bitdepth changes.

Conflicts:
	Changelog
	RELEASE
	libavcodec/aacsbr.c
	libavcodec/h264_ps.c
	libavcodec/pngdec.c
	libavformat/rtpdec_asf.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-06-04 13:05:25 +02:00
Ronald S. Bultje
26ac878cc2 h264: additional protection against unsupported size/bitdepth changes.
Fixes crashes in codepaths not covered by original checks.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 732f9fcfe54fc9a0a7bbce53fe86b38744c2d301)

Conflicts:

	libavcodec/h264.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 746f1594d71dece6fd6f786447e19be9c200a07d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-05-06 21:40:58 +02:00
Michael Niedermayer
336d5ae37b Merge branch 'release/0.8' into release/0.7
* release/0.8: (182 commits)
  id3v2: fix skipping extended header in id3v2.4
  Update RELEASE file for 0.7.5
  lcl: use AVERROR_INVALIDDATA instead of AVERROR_UNKNOWN
  kgv1dec: Increase offsets array size so it is large enough.
  kgv1: use avctx->get/release_buffer().
  kvmc: fix invalid reads
  nsvdec: Propagate error values instead of returning 0 in nsv_read_header().
  mjpegbdec: Fix overflow in SOS.
  shorten: Use separate pointers for the allocated memory for decoded samples.
  shorten: check for realloc failure (cherry picked from commit 9e5e2c2d010c05c10337e9c1ec9d0d61495e0c9c)
  atrac3: Fix crash in tonal component decoding.
  ws_snd1: Fix wrong samples count and crash.
  ws_snd: add some checks to prevent buffer overread or overwrite. (cherry picked from commit 417364ce1f979031ef6fee661fc15e1869bdb1b4)
  ws_snd: decode to AV_SAMPLE_FMT_U8 instead of S16.
  dca: include libavutil/mathematics.h for possibly missing M_SQRT1_2
  h264: stricter reference limit enforcement.
  jvdec: unbreak video decoding
  xxan: don't read before start of buffer in av_memcpy_backptr().
  dsicinvideo: validate buffer offset before copying pixels.
  huffyuv: add padding to classic (v1) huffman tables.
  ...

Conflicts:
	Doxyfile
	RELEASE
	VERSION

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-04-02 01:44:30 +02:00
Michael Niedermayer
b6cc1c77fd Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7: (84 commits)
  id3v2: fix skipping extended header in id3v2.4
  Update RELEASE file for 0.7.5
  lcl: use AVERROR_INVALIDDATA instead of AVERROR_UNKNOWN
  kgv1dec: Increase offsets array size so it is large enough.
  kgv1: use avctx->get/release_buffer().
  kvmc: fix invalid reads
  nsvdec: Propagate error values instead of returning 0 in nsv_read_header().
  mjpegbdec: Fix overflow in SOS.
  shorten: Use separate pointers for the allocated memory for decoded samples.
  shorten: check for realloc failure (cherry picked from commit 9e5e2c2d010c05c10337e9c1ec9d0d61495e0c9c)
  atrac3: Fix crash in tonal component decoding.
  ws_snd1: Fix wrong samples count and crash.
  ws_snd: add some checks to prevent buffer overread or overwrite. (cherry picked from commit 417364ce1f979031ef6fee661fc15e1869bdb1b4)
  ws_snd: decode to AV_SAMPLE_FMT_U8 instead of S16.
  dca: include libavutil/mathematics.h for possibly missing M_SQRT1_2
  h264: stricter reference limit enforcement.
  jvdec: unbreak video decoding
  xxan: don't read before start of buffer in av_memcpy_backptr().
  dsicinvideo: validate buffer offset before copying pixels.
  huffyuv: add padding to classic (v1) huffman tables.
  ...

Conflicts:
	RELEASE
	libavcodec/atrac3.c
	libavcodec/h264.c
	libavcodec/h264_parser.c
	libavcodec/kgv1dec.c
	libavcodec/shorten.c
	libavcodec/svq3.c
	libavcodec/ws-snd1.c
	libavcodec/xxan.c
	libswscale/utils.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-04-02 01:25:31 +02:00
Ronald S. Bultje
8b819fd9d3 h264: stricter reference limit enforcement.
Progressive images can have only 16 references, error out if there are
more, since the data is almost certainly corrupt, and the invalid value
will lead to random crashes or invalid writes later on.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e0febda22d0e0fab094a9c886b0e0f0f662df1ef)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:29 +02:00
Michael Niedermayer
a3d331f2d8 Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7: (96 commits)
  intfloat_readwrite: fix signed addition overflows
  smacker: validate channels and sample format.
  smacker: check buffer size before reading output size
  smacker: validate number of channels
  sipr: fix get_bits(0) calls
  motion_est: make MotionExtContext.map_generation unsigned
  4xm: prevent NULL dereference with invalid huffman table
  4xmdemux: prevent use of uninitialized memory
  4xm: clear FF_INPUT_BUFFER_PADDING_SIZE bytes in temporary buffers
  ptx: check for out of bound reads
  tiffdec: fix out of bound reads/writes
  eacmv: check for out of bound reads
  eacmv: fix potential pointer arithmetic overflows
  adpcm: fix out of bound reads due to integer overflow
  anm: prevent infinite loop
  avsdemux: check for out of bound writes
  avs: check for out of bound reads
  avsdemux: check for corrupted data
  mxfdec: Fix some buffer overreads caused by the misuse of AVPacket related functions.
  vaapi: Fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
  ...

Conflicts:
	libavcodec/adpcm.c
	libavcodec/bink.c
	libavcodec/h264.c
	libavcodec/h264.h
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/motion_est_template.c
	libavcodec/mpegvideo.c
	libavcodec/nellymoserdec.c
	libavcodec/ptx.c
	libavcodec/svq3.c
	libavcodec/vaapi_vc1.c
	libavcodec/xan.c
	libavfilter/vf_scale.c
	libavformat/4xm.c
	libavformat/flvdec.c
	libavformat/mpeg.c
	tests/ref/fate/motionpixels

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-19 05:14:44 +01:00
Ronald S. Bultje
047c6ad752 h264: disallow constrained intra prediction modes for luma.
Conversion of the luma intra prediction mode to one of the constrained
("alzheimer") ones can happen by crafting special bitstreams, causing
a crash because we'll call a NULL function pointer for 16x16 block intra
prediction, since constrained intra prediction functions are only
implemented for chroma (8x8 blocks).

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 45b7bd7c53b41bc5ff6fc2158831f2b1b1256113)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 248d4e461578ff327a2fd75fd0db4f38c270918a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-19 15:41:16 +01:00
Laurent Aimar
ba31a01681 h264: reset h->ref_count in case of errors in ff_h264_decode_ref_pic_list_reordering()
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 4c7a232fc81fdbdee279ab819a255f624a22b083)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-19 15:08:42 +01:00
Michael Niedermayer
c6bb93dcd9 H264: Only wait before triggering ff_thread_setup_complete() until the next slice that contains a start-of-field/frame macroblock
This allows concurrent decoding of the last field/frame, rather than
only the last slice, of data packets with multiple NAL units packed
together.

This will fix the slowdown reported in e.g. bug 52.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 14c21c1ff509eac97f6437aeb51202b15af3a700)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-19 15:08:42 +01:00
Ronald S. Bultje
485f85aa90 h264: correct implicit_weight for field-interlaced pictures.
(cherry picked from commit 4418aa9cb3b2f0b83748e37d2952560cf84b3611)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-19 15:08:42 +01:00
Laurent Aimar
ec2a1d91e2 h264: check for out of bounds reads in ff_h264_decode_extradata().
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit d1186ff72d75b6067770890758c4feb92abd84f7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-19 15:08:42 +01:00
Michael Niedermayer
ba5bb0562b h264: Use mismatching frame numbers in fields
to synchronize the first/second field state independant of them being reference or not.
Fixes Ticket354

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 545ec935a4b4e0f032ebd975907b41f6fe4465c9)
2011-12-06 23:39:42 +01:00
Michael Niedermayer
1550c0885d h264: Use mismatching frame numbers in fields
to synchronize the first/second field state independant of them being reference or not.
Fixes Ticket354

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 545ec935a4b4e0f032ebd975907b41f6fe4465c9)
2011-12-06 23:31:39 +01:00
Michael Niedermayer
3e17543491 Merge branch 'release/0.8' into release/0.7
* release/0.8: (96 commits)
  Version numbers for 0.8.6
  snow: emu edge support Fixes Ticket592
  imc: validate channel count
  imc: check for ff_fft_init() failure (cherry picked from commit 95fee70d6773fde1c34ff6422f48e5e66f37f263)
  libgsmdec: check output buffer size before decoding (cherry picked from commit b03761b1309293bbf30edef767503875277b01cf)
  configure: fix arch x86_32
  mp3enc: avoid truncating id3v1 tags by one byte
  asfdec: Check packet_replic_size earlier
  cin audio: validate the channel count
  binkaudio: add some buffer overread checks.
  atrac1: validate number of channels (cherry picked from commit bff5b2c1ca1290ea30587ff2f76171f9e3854872)
  atrac1: check output buffer size before decoding (cherry picked from commit 33684b9c12b74c0140fb91e8150263db4a48d55e)
  vp3: fix oob read for negative tokens and memleaks on error. (cherry picked from commit 8370e426e42f2e4b9d14a1fb8107ecfe5163ce7f)
  apedec: set s->currentframeblocks after validating nblocks
  apedec: use unsigned int for 'nblocks' and make sure that it's within int range
  apedec: check for data buffer realloc failure (cherry picked from commit 11ca8b2d7486e879926488404b3b79af774f0f2d)
  apedec: check for filter buffer allocation failure (cherry picked from commit 7500781313d11b37772c05a28da20fbc112db478)
  mpegaudiodec: check output data size based on avctx->frame_size
  resample: Fix array size
  resample2: fix potential overflow
  ...

Conflicts:
	Doxyfile
	RELEASE
	VERSION

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 20:20:37 +01:00
Laurent Aimar
b491c15c85 h254: explicitly initialize bit depth/chroma idc
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:53:56 +01:00
Laurent Aimar
f74d1c6de7 h264: do not let invalid values in h->ref_count after a decoder reset.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0333d234b0355b375762447e93674e3fe3c5bff1)
2011-10-11 21:34:15 +02:00
Michael Niedermayer
09a288476f H264: hotfix for speedloss on frame threading and h264 files with slices.
This fix is not ideal as it still limits the multithreading on field pictures
to the 2nd field only.
Ill try to fix it properly to allow both fields to decode concurrently but this
needs more work.

This bug exists since and was caused by:
commit ea6331f8bbaf3343faec6ffe77a6218a016a3af5
Author: Ronald S. Bultje <rsbultje@gmail.com>
Date:   Mon Jun 20 10:24:33 2011 -0400

    h264-mt: fix deadlock in packets with multiple slices (e.g. MP4).
(cherry picked from commit eaa21b6870ba8bba4b0370e91f1941307c1c9681)
2011-10-11 21:34:14 +02:00
Michael Niedermayer
7fc85451fd Merge branch 'release/0.8' into release/0.7
* release/0.8: (185 commits)
  h264: fix intra 16x16 mode check when using mbaff and constrained_intra_pred.
  h264: check for invalid bit depth value.
  h264: add entries for 11 and 12 bits in ff_h264_chroma_qp[][]
  h264: fix the check for invalid SPS:num_ref_frames.
  h264: do not let invalid values in h->ref_count on ff_h264_decode_ref_pic_list_reordering() errors.
  Reject video with non multiple of 16 width/height in the 4xm decoder.
  4xm decoder: fix data size for i2 frames.
  4xm decoder: print some error messages in case of errors.
  Check for out of bound accesses in the 4xm decoder.
  Prevent block size from inreasing in the shorten decoder.
  Check for out of bound reads in PTX decoder.
  Clear FF_INPUT_BUFFER_PADDING_SIZE bytes at the end of the temporary buffers used in 4xm decoder.
  Fix the check for missing references in ff_er_frame_end() for H264.
  Prevent NULL dereference when the huffman table is invalid in the 4xm decoder.
  Fix use of uninitialized memory in 4X Technologies demuxer.
  h264: increase ref_poc size to 32 as it can be per field.
  h264: set unused ref_counts to 0 as a precautionary meassure.
  Remove Chnagelog it has nothing to do with reality
  fate: fix motion pixels checksum change caused by backported bugfix
  avienc: Add a limit on the number of skiped frames muxed in a row.
  ...

Conflicts:
	Doxyfile
	RELEASE
	VERSION
	libavformat/utils.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2011-10-02 21:37:59 +02:00
Laurent Aimar
b89a0c9d7f h264: fix intra 16x16 mode check when using mbaff and constrained_intra_pred.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a4fd95b5d511384ed3ce388d8d20a16b1c4c0530)
2011-10-02 21:30:21 +02:00
Laurent Aimar
cf0052931d h264: do not let invalid values in h->ref_count on ff_h264_decode_ref_pic_list_reordering() errors.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2428b53f6d306d8d71dec34fa7b0af733d76cfac)
2011-10-02 21:29:51 +02:00
Michael Niedermayer
3e0dbb8a7e h264: set unused ref_counts to 0 as a precautionary meassure.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3af2de76ac8b7aa762e4ee129ec29140e6247d81)
2011-10-02 05:44:35 +02:00
Alexander Strasser
7aa24b157d h264: ff_h264_decode_extradata: check buffer args
The buffer size and pointer were not checked prior to testing the first
byte of the buffer. These were sometimes checked before calling, but it is
better to add it inside the function as it takes buf and size arguments.

Signed-off-by: Alexander Strasser <eclipse7@gmx.net>
(cherry picked from commit 715f259bf949b06df1b5ed0307606dc258754c99)
2011-10-01 20:29:07 +02:00
Michael Niedermayer
f71c761a9e h264: pass buffer & size to ff_h264_decode_extradata()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-10-01 20:11:00 +02:00
Laurent Aimar
101e38e08a h264: Check for out of bounds reads in ff_h264_decode_extradata().
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57764c699671b32f0c70795feafc0c6d7491f198)
2011-10-01 19:54:49 +02:00
Michael Niedermayer
be9183de2e h264: More correct ref_count check in decode_slice_header()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dc9ce40069bde3d28f8d0b3e5bd733ae255fecb5)
2011-09-28 23:36:39 +02:00
Jeff Downs
ce8f40a7b9 h264: fix PCM intra-coded blocks in monochrome case
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 6581e161c5f46733a5619208483de29416eb9a51)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Jeff Downs
45b3f7c71e h264: correct implicit weight table computation for long ref pics
Correct computation of implicit weight tables when referencing pictures
that are marked for long reference.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 87cf70eb237e7586cc7399627dafa1b980ec0b7d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Dustin Brody
59a22afa0b h264: notice memory allocation failure
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit bac3ab13ea6a9dd8853e79ef3eacf51d234c8774)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Michael Niedermayer
a99a35c8ea Merge branch 'release/0.8' into release/0.7
* release/0.8: (154 commits)
  vp6: partially propagate huffman tree building errors during coeff model parsing and fix misspelling
  Check for huffman tree building error in vp6 decoder.
  Release old pictures after a resolution change in vp5/6 decoder
  Check for missing reference in vp5/6 decoder.
  Check for invalid slices offsets in RV30/40 decoder.
  Check output buffer size in nellymoser decoder.
  Hack around gcc 4.6 breaking asm using call.
  Fix dxva2 decoding for some H264 samples.
  mp3demux: pass on error code on packet read.
  Check for invalid slice offsets in real decoder.
  rmdec: Reject invalid deinterleaving parameters
  Use deinterleavers for demangling audio packets in RealMedia.
  rv10: Reject slices that does not have the same type as the first one
  rmdec: use the deinterleaving mode and not the codec when creating audio packets.
  MAINTAINERS: add my GPG fingerprint. (cherry picked from commit 7882dc10f871bf25a848fe62a152f63814f9c7d1)
  Support 3IVD in isom, produced by 3ivx DivX Doctor.
  mpegpsdec: fix reading first mpegps packet (cherry picked from commit b2f230e23dd61112ac090b0c059d87b5f6bcb307)
  Avoid NULL dereference on corrupted bitstream with real decoder.
  Reject slices that does not have the same type than the first one in RV10/RV20 decoder.
  check all svq3_get_ue_golomb() returns.
  ...

Conflicts:
	Doxyfile
	RELEASE
	VERSION
	libavcodec/rv34.c
	libavformat/utils.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2011-09-22 01:48:45 +02:00
Michael Niedermayer
ec7f0b527c Merge remote-tracking branch 'khirnov/release/0.7' into release/0.8
* khirnov/release/0.7: (64 commits)
  rv34: Check for invalid slice offsets
  rv34: Fix potential overreads
  rv34: Avoid NULL dereference on corrupted bitstream
  rv10: Reject slices that does not have the same type as the first one
  lavf: Fix context pointer in av_open_input_stream when avformat_open_input fails
  oggdec: fix out of bound write in the ogg demuxer
  Fixed size given to init_get_bits().
  smacker: fix a few off by 1 errors
  Check for invalid VLC value in smacker decoder.
  Check and propagate errors when VLC trees cannot be built in smacker decoder.
  Fixed off by one packet size allocation in the smacker demuxer.
  Check for invalid packet size in the smacker demuxer.
  ape demuxer: fix segfault on memory allocation failure.
  xan: Add some buffer checks (cherry picked from commit 0872bb23b4bd2d94a8ba91070f706d1bc1c3ced8)
  Fixed size given to init_get_bits() in xan decoder. (cherry picked from commit 393d5031c6aaaf8c2dda4eb5d676974c349fae85)
  smacker demuxer: handle possible av_realloc() failure.
  Fixed segfault with wavpack decoder on corrupted decorrelation terms sub-blocks.
  cljr: init_get_bits size in bits instead of bytes (cherry picked from commit 0c1f5b93d9b97c4cc3684ba91a040e90bfc760d2)
  indeo2: fail if input buffer too small (cherry picked from commit b7ce4f1d1c3add86ece7ca595ea6c4a10b471055)
  indeo2: init_get_bits size in bits instead of bytes (cherry picked from commit 68ca330cbd479111db9cb7649d7530ad59f04cc8)
  ...

Conflicts:
	ffmpeg.c
	libavdevice/alsa-audio.h
	libavformat/gxf.c
	libswscale/x86/swscale_template.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2011-09-22 01:10:24 +02:00
Michael Niedermayer
fa3f7391be h264: allow disabling bitstream overread protection by using the fast flag.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 205c13685f36a17f406b7cd07e14b6d2a61f396a)
2011-09-21 19:49:30 +02:00
Michael Niedermayer
54544100a3 h264: prevent an out of array read in decode_nal_units()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ea0ac11e52b9cf5264f3d4eb7543e760c2a5fbb4)
2011-09-21 18:02:18 +02:00
Michael Niedermayer
97437dada6 h264dec: Prevent CABAC and CAVLC bitsteram overreading
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 23f5cff92cdcfa55a735c458fcb5f95c0e0f3b1f)
2011-09-21 18:02:13 +02:00