979 Commits

Author SHA1 Message Date
Michael Niedermayer
0a41da3e9d Merge commit 'd785f6940144eb6ce4c24309ed034056b81395bc' into release/0.8
* commit 'd785f6940144eb6ce4c24309ed034056b81395bc':
  shorten: validate that the channel count in the header is not <= 0
  matroskadec: request a read buffer for the wav header
  h264: check for luma and chroma bit depth being equal
  xxan: fix invalid memory access in xan_decode_frame_type0()
  wmadec: require block_align to be set.

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 17:22:53 +02:00
Reinhard Tartler
7d4c38d58d h264: check for luma and chroma bit depth being equal
The decoder assumes a single bit depth for all the planes while
the specification allows different bit depths for luma and chroma.

Avoid the possible problems described in CVE-2013-2277

Conflicts:
	libavcodec/h264.c
2013-05-09 11:20:11 +02:00
Michael Niedermayer
56cc629a64 Revert "h264: allow cropping to AVCodecContext.width/height"
This reverts commit a2ae183a382f063c5403922b5151d865ce7252a2.

This removes a duplicate hunk

Found-by: Joakim Plate <elupus@ecce.se>
2013-01-19 13:34:41 +01:00
Michael Niedermayer
685321e4bd Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7:
  h264: check ref_count validity for num_ref_idx_active_override_flag
  h264: check context state before decoding slice data partitions
  oggdec: free the ogg streams on read_header failure
  oggdec: check memory allocation
  Fix uninitialized reads on malformed ogg files.
  rtsp: Recheck the reordering queue if getting a new packet
  alacdec: do not be too strict about the extradata size
  h264: fix sps parsing for SVC and CAVLC 4:4:4 Intra profiles
  h264: check sps.log2_max_frame_num for validity
  ppc: always use pic for shared libraries
  h264: enable low delay only if no delayed frames were seen
  lavf: avoid integer overflow in ff_compute_frame_duration()

Conflicts:
	libavformat/oggdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-17 03:16:46 +01:00
Michael Niedermayer
3f1a58db6f Merge commit 'b143844ea0f6246e0d5a938d743e2e8a98453bec' into release/0.8
* commit 'b143844ea0f6246e0d5a938d743e2e8a98453bec': (22 commits)
  aacdec: Fix an off-by-one overwrite when switching to LTP profile from MAIN.
  vp6: properly fail on unsupported feature
  h264: Fix parameters to ff_er_add_slice() call
  flacenc: ensure the order is within the min/max range in LPC order search
  yuv4mpeg: reject unsupported codecs
  vp8: reset loopfilter delta values at keyframes.
  vp56: release frames on error
  vp56: make parse_header return standard error codes
  ivi_common: check that scan pattern is set before using it.
  Update RELEASE file for 0.7.7
  tiffenc: Check av_malloc() results.
  mpegaudiodec: fix short_start calculation
  h264: avoid stuck buffer pointer in decode_nal_units
  yuv4mpeg: return proper error codes.
  smacker audio: sign-extend the initial 16-bit predicted value
  vf_pad: don't give up its own reference to the output buffer.
  avidec: return 0, not packet size from read_packet().
  wmapro: prevent division by zero when sample rate is unspecified
  alsdec: fix number of decoded samples in first sub-block in BGMC mode.
  alsdec: remove dead assignments
  ...

Conflicts:
	RELEASE
	libavformat/avidec.c
	libavformat/yuv4mpeg.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-17 03:03:39 +01:00
Michael Niedermayer
597d709eb4 Merge commit 'aa45b90804ab21175b8c116bd8e5eb4b4e85fbcb' into release/0.8
* commit 'aa45b90804ab21175b8c116bd8e5eb4b4e85fbcb': (22 commits)
  alsdec: Check k used for rice decoder.
  cavsdec: check for changing w/h.
  avidec: use actually read size instead of requested size
  wmaprodec: check num_vec_coeffs for validity
  lagarith: check count before writing zeros.
  indeo5: check tile size in decode_mb_info().
  indeo5: prevent null pointer dereference on broken files
  indeo: check for invalid motion vectors
  indeo: clear allocated band buffers
  indeo: check custom Huffman tables for errors
  dfa: add some checks to ensure that decoder won't write past frame end
  dfa: check that the caller set width/height properly.
  bytestream: add a new set of bytestream functions with overread checking
  avsdec: Set dimensions instead of relying on the demuxer.
  lavfi: avfilter_merge_formats: handle case where inputs are same
  rv34: use AVERROR return values in ff_rv34_decode_frame()
  h263: Add ff_ prefix to nonstatic symbols
  eval: fix swapping of lt() and lte()
  bmpdec: only initialize palette for pal8.
  vc1dec: add flush function for WMV9 and VC-1 decoders
  ...

Conflicts:
	libavcodec/avs.c
	libavcodec/mpegvideo_enc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-17 02:56:12 +01:00
Janne Grunau
dd0c5e0fa9 h264: check ref_count validity for num_ref_idx_active_override_flag
Fixes segfault in the fuzzed sample bipbop234.ts_s226407.
CC: libav-stable@libav.org
(cherry-picked from commit 6e5cdf26281945ddea3aaf5eca4d127791f23ca8)
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
2013-01-12 19:36:38 +01:00
Janne Grunau
ad02537746 h264: check context state before decoding slice data partitions
Fixes mov_h264_aac__Demo_FlagOfOurFathers.mov.SIGSEGV.4e9.656.

Found-by: Mateusz "j00ru" Jurczyk
CC: libav-stable@libav.org
(cherry-picked from commit c1fcf563b13051f280db169ba41c6a1b21b25e08)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-01-12 19:36:38 +01:00
Janne Grunau
5fa739e685 h264: enable low delay only if no delayed frames were seen
Dropping frames is undesirable but that is the only way by which the
decoder could return to low delay mode. Instead emit a warning and
continue with delayed frames.
Fixes a crash in fuzzed sample nasa-8s2.ts_s20033 caused by a larger
than expected has_b_frames value. Low delay keeps getting re-enabled
from a presumely broken SPS.

CC: libav-stable@libav.org
(cherry picked from commit 706acb558a38eba633056773280155d66c2f4b24)

Conflicts:

	libavcodec/h264.c
2013-01-12 19:32:24 +01:00
Reinhard Tartler
ce8910d861 h264: Fix parameters to ff_er_add_slice() call
s->mb_x is reset to zero a couple of lines above. It does not make
sense to call ff_er_add_slice() with 0 as endx when the end of the
macroblock row was reached. Fixes unnecessary and counterproductive
error resilience in https://bugzilla.libav.org/show_bug.cgi?id=394.

(cherry picked from commit e6160bda98641b7d4f86de15761ad2a962f21a36)

Conflicts:

	libavcodec/h264.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:

	libavcodec/h264.c
2013-01-12 19:22:22 +01:00
Jindřich Makovička
11ecd8574a h264: avoid stuck buffer pointer in decode_nal_units
When decode_nal_units() previously encountered a NAL_END_SEQUENCE,
and there are some junk bytes left in the input buffer, but no start codes,
buf_index gets stuck 3 bytes before the end of the buffer.

This can trigger an infinite loop in the caller code, eg. in
try_decode_trame(), as avcodec_decode_video() then keeps returning zeroes,
with 3 bytes of the input packet still available.

With this change, the remaining bytes are skipped so the whole packet gets
consumed.

CC:libav-stable@libav.org

Signed-off-by: Jindřich Makovička <makovick@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 1a8c6917f68f7378465e18f7615762bfd22704c2)

Conflicts:

	libavcodec/h264.c
2013-01-04 07:43:38 +01:00
Mans Rullgard
a2ae183a38 h264: allow cropping to AVCodecContext.width/height
Override the frame size from the SPS with AVCodecContext values
if the latter specify a size smaller by less than one macroblock.
This is required for correct cropping of MOV files from Canon cameras.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 30f515091c323da59c0f1b533703dedca2f4b95d)

Conflicts:

	libavcodec/h264.c
2013-01-04 07:43:20 +01:00
Michael Niedermayer
e28814e0e1 Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7:
  vorbis: Validate that the floor 1 X values contain no duplicates.
  vorbisenc: check all allocations for failure
  lavfi: avfilter_merge_formats: handle case where inputs are same
  alsdec: check opt_order.
  lavf: don't segfault when a NULL filename is passed to avformat_open_input()
  mpegvideo: Don't use ff_mspel_motion() for vc1
  imgconvert: avoid undefined left shift in avcodec_find_best_pix_fmt
  nuv: check RTjpeg header for validity
  vc1dec: add flush function for WMV9 and VC-1 decoders
  ffmpeg: fix -force_key_frames
  mov: set AVCodecContext.width/height for h264
  h264: allow cropping to AVCodecContext.width/height

Conflicts:
	libavcodec/mpegvideo_common.h
	libavcodec/nuv.c
	libavcodec/vorbisenc.c
	libavfilter/formats.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-10-16 17:57:12 +02:00
Mans Rullgard
b102d5d97d h264: allow cropping to AVCodecContext.width/height
Override the frame size from the SPS with AVCodecContext values
if the latter specify a size smaller by less than one macroblock.
This is required for correct cropping of MOV files from Canon cameras.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 30f515091c323da59c0f1b533703dedca2f4b95d)

Conflicts:

	libavcodec/h264.c
(cherry picked from commit e1608014c50eeb9f4744a53de0794eb6bb1269a2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-06-10 09:54:22 +02:00
Michael Niedermayer
64bc5f3bf7 Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7:
  Update RELEASE file for 0.7.6
  Update changelog for 0.7.6 release
  ea: check chunk_size for validity.
  png: check bit depth for PAL8/Y400A pixel formats.
  x86: fix build with gcc 4.7
  qdm2: clip array indices returned by qdm2_get_vlc().
  kmvc: Check palsize.
  aacsbr: prevent out of bounds memcpy().
  rtpdec_asf: Fix integer underflow that could allow remote code execution
  dpcm: ignore extra unpaired bytes in stereo streams.
  tqi: Pass errors from the MB decoder
  h264: Add check for invalid chroma_format_idc
  adpcm: ADPCM Electronic Arts has always two channels
  h263dec: Disallow width/height changing with frame threads.
  vqavideo: return error if image size is not a multiple of block size
  celp filters: Do not read earlier than the start of the 'out' vector.
  motionpixels: Clip YUV values after applying a gradient.
  h263: more strictly forbid frame size changes with frame-mt.
  h264: additional protection against unsupported size/bitdepth changes.

Conflicts:
	Changelog
	RELEASE
	libavcodec/aacsbr.c
	libavcodec/h264_ps.c
	libavcodec/pngdec.c
	libavformat/rtpdec_asf.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-06-04 13:05:25 +02:00
Ronald S. Bultje
26ac878cc2 h264: additional protection against unsupported size/bitdepth changes.
Fixes crashes in codepaths not covered by original checks.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 732f9fcfe54fc9a0a7bbce53fe86b38744c2d301)

Conflicts:

	libavcodec/h264.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 746f1594d71dece6fd6f786447e19be9c200a07d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-05-06 21:40:58 +02:00
Michael Niedermayer
b6cc1c77fd Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7: (84 commits)
  id3v2: fix skipping extended header in id3v2.4
  Update RELEASE file for 0.7.5
  lcl: use AVERROR_INVALIDDATA instead of AVERROR_UNKNOWN
  kgv1dec: Increase offsets array size so it is large enough.
  kgv1: use avctx->get/release_buffer().
  kvmc: fix invalid reads
  nsvdec: Propagate error values instead of returning 0 in nsv_read_header().
  mjpegbdec: Fix overflow in SOS.
  shorten: Use separate pointers for the allocated memory for decoded samples.
  shorten: check for realloc failure (cherry picked from commit 9e5e2c2d010c05c10337e9c1ec9d0d61495e0c9c)
  atrac3: Fix crash in tonal component decoding.
  ws_snd1: Fix wrong samples count and crash.
  ws_snd: add some checks to prevent buffer overread or overwrite. (cherry picked from commit 417364ce1f979031ef6fee661fc15e1869bdb1b4)
  ws_snd: decode to AV_SAMPLE_FMT_U8 instead of S16.
  dca: include libavutil/mathematics.h for possibly missing M_SQRT1_2
  h264: stricter reference limit enforcement.
  jvdec: unbreak video decoding
  xxan: don't read before start of buffer in av_memcpy_backptr().
  dsicinvideo: validate buffer offset before copying pixels.
  huffyuv: add padding to classic (v1) huffman tables.
  ...

Conflicts:
	RELEASE
	libavcodec/atrac3.c
	libavcodec/h264.c
	libavcodec/h264_parser.c
	libavcodec/kgv1dec.c
	libavcodec/shorten.c
	libavcodec/svq3.c
	libavcodec/ws-snd1.c
	libavcodec/xxan.c
	libswscale/utils.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-04-02 01:25:31 +02:00
Ronald S. Bultje
8b819fd9d3 h264: stricter reference limit enforcement.
Progressive images can have only 16 references, error out if there are
more, since the data is almost certainly corrupt, and the invalid value
will lead to random crashes or invalid writes later on.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e0febda22d0e0fab094a9c886b0e0f0f662df1ef)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-01 18:33:29 +02:00
Michael Niedermayer
a3d331f2d8 Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7: (96 commits)
  intfloat_readwrite: fix signed addition overflows
  smacker: validate channels and sample format.
  smacker: check buffer size before reading output size
  smacker: validate number of channels
  sipr: fix get_bits(0) calls
  motion_est: make MotionExtContext.map_generation unsigned
  4xm: prevent NULL dereference with invalid huffman table
  4xmdemux: prevent use of uninitialized memory
  4xm: clear FF_INPUT_BUFFER_PADDING_SIZE bytes in temporary buffers
  ptx: check for out of bound reads
  tiffdec: fix out of bound reads/writes
  eacmv: check for out of bound reads
  eacmv: fix potential pointer arithmetic overflows
  adpcm: fix out of bound reads due to integer overflow
  anm: prevent infinite loop
  avsdemux: check for out of bound writes
  avs: check for out of bound reads
  avsdemux: check for corrupted data
  mxfdec: Fix some buffer overreads caused by the misuse of AVPacket related functions.
  vaapi: Fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
  ...

Conflicts:
	libavcodec/adpcm.c
	libavcodec/bink.c
	libavcodec/h264.c
	libavcodec/h264.h
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/motion_est_template.c
	libavcodec/mpegvideo.c
	libavcodec/nellymoserdec.c
	libavcodec/ptx.c
	libavcodec/svq3.c
	libavcodec/vaapi_vc1.c
	libavcodec/xan.c
	libavfilter/vf_scale.c
	libavformat/4xm.c
	libavformat/flvdec.c
	libavformat/mpeg.c
	tests/ref/fate/motionpixels

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-19 05:14:44 +01:00
Ronald S. Bultje
047c6ad752 h264: disallow constrained intra prediction modes for luma.
Conversion of the luma intra prediction mode to one of the constrained
("alzheimer") ones can happen by crafting special bitstreams, causing
a crash because we'll call a NULL function pointer for 16x16 block intra
prediction, since constrained intra prediction functions are only
implemented for chroma (8x8 blocks).

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 45b7bd7c53b41bc5ff6fc2158831f2b1b1256113)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 248d4e461578ff327a2fd75fd0db4f38c270918a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-02-19 15:41:16 +01:00
Laurent Aimar
ba31a01681 h264: reset h->ref_count in case of errors in ff_h264_decode_ref_pic_list_reordering()
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 4c7a232fc81fdbdee279ab819a255f624a22b083)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-19 15:08:42 +01:00
Michael Niedermayer
c6bb93dcd9 H264: Only wait before triggering ff_thread_setup_complete() until the next slice that contains a start-of-field/frame macroblock
This allows concurrent decoding of the last field/frame, rather than
only the last slice, of data packets with multiple NAL units packed
together.

This will fix the slowdown reported in e.g. bug 52.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 14c21c1ff509eac97f6437aeb51202b15af3a700)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-19 15:08:42 +01:00
Ronald S. Bultje
485f85aa90 h264: correct implicit_weight for field-interlaced pictures.
(cherry picked from commit 4418aa9cb3b2f0b83748e37d2952560cf84b3611)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-19 15:08:42 +01:00
Laurent Aimar
ec2a1d91e2 h264: check for out of bounds reads in ff_h264_decode_extradata().
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit d1186ff72d75b6067770890758c4feb92abd84f7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-02-19 15:08:42 +01:00
Michael Niedermayer
1550c0885d h264: Use mismatching frame numbers in fields
to synchronize the first/second field state independant of them being reference or not.
Fixes Ticket354

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 545ec935a4b4e0f032ebd975907b41f6fe4465c9)
2011-12-06 23:31:39 +01:00
Laurent Aimar
b491c15c85 h254: explicitly initialize bit depth/chroma idc
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:53:56 +01:00
Laurent Aimar
f74d1c6de7 h264: do not let invalid values in h->ref_count after a decoder reset.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0333d234b0355b375762447e93674e3fe3c5bff1)
2011-10-11 21:34:15 +02:00
Michael Niedermayer
09a288476f H264: hotfix for speedloss on frame threading and h264 files with slices.
This fix is not ideal as it still limits the multithreading on field pictures
to the 2nd field only.
Ill try to fix it properly to allow both fields to decode concurrently but this
needs more work.

This bug exists since and was caused by:
commit ea6331f8bbaf3343faec6ffe77a6218a016a3af5
Author: Ronald S. Bultje <rsbultje@gmail.com>
Date:   Mon Jun 20 10:24:33 2011 -0400

    h264-mt: fix deadlock in packets with multiple slices (e.g. MP4).
(cherry picked from commit eaa21b6870ba8bba4b0370e91f1941307c1c9681)
2011-10-11 21:34:14 +02:00
Laurent Aimar
b89a0c9d7f h264: fix intra 16x16 mode check when using mbaff and constrained_intra_pred.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a4fd95b5d511384ed3ce388d8d20a16b1c4c0530)
2011-10-02 21:30:21 +02:00
Laurent Aimar
cf0052931d h264: do not let invalid values in h->ref_count on ff_h264_decode_ref_pic_list_reordering() errors.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2428b53f6d306d8d71dec34fa7b0af733d76cfac)
2011-10-02 21:29:51 +02:00
Michael Niedermayer
3e0dbb8a7e h264: set unused ref_counts to 0 as a precautionary meassure.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3af2de76ac8b7aa762e4ee129ec29140e6247d81)
2011-10-02 05:44:35 +02:00
Alexander Strasser
7aa24b157d h264: ff_h264_decode_extradata: check buffer args
The buffer size and pointer were not checked prior to testing the first
byte of the buffer. These were sometimes checked before calling, but it is
better to add it inside the function as it takes buf and size arguments.

Signed-off-by: Alexander Strasser <eclipse7@gmx.net>
(cherry picked from commit 715f259bf949b06df1b5ed0307606dc258754c99)
2011-10-01 20:29:07 +02:00
Michael Niedermayer
f71c761a9e h264: pass buffer & size to ff_h264_decode_extradata()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-10-01 20:11:00 +02:00
Laurent Aimar
101e38e08a h264: Check for out of bounds reads in ff_h264_decode_extradata().
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57764c699671b32f0c70795feafc0c6d7491f198)
2011-10-01 19:54:49 +02:00
Michael Niedermayer
be9183de2e h264: More correct ref_count check in decode_slice_header()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dc9ce40069bde3d28f8d0b3e5bd733ae255fecb5)
2011-09-28 23:36:39 +02:00
Jeff Downs
ce8f40a7b9 h264: fix PCM intra-coded blocks in monochrome case
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 6581e161c5f46733a5619208483de29416eb9a51)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Jeff Downs
45b3f7c71e h264: correct implicit weight table computation for long ref pics
Correct computation of implicit weight tables when referencing pictures
that are marked for long reference.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 87cf70eb237e7586cc7399627dafa1b980ec0b7d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Dustin Brody
59a22afa0b h264: notice memory allocation failure
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit bac3ab13ea6a9dd8853e79ef3eacf51d234c8774)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-26 19:26:30 +02:00
Michael Niedermayer
ec7f0b527c Merge remote-tracking branch 'khirnov/release/0.7' into release/0.8
* khirnov/release/0.7: (64 commits)
  rv34: Check for invalid slice offsets
  rv34: Fix potential overreads
  rv34: Avoid NULL dereference on corrupted bitstream
  rv10: Reject slices that does not have the same type as the first one
  lavf: Fix context pointer in av_open_input_stream when avformat_open_input fails
  oggdec: fix out of bound write in the ogg demuxer
  Fixed size given to init_get_bits().
  smacker: fix a few off by 1 errors
  Check for invalid VLC value in smacker decoder.
  Check and propagate errors when VLC trees cannot be built in smacker decoder.
  Fixed off by one packet size allocation in the smacker demuxer.
  Check for invalid packet size in the smacker demuxer.
  ape demuxer: fix segfault on memory allocation failure.
  xan: Add some buffer checks (cherry picked from commit 0872bb23b4bd2d94a8ba91070f706d1bc1c3ced8)
  Fixed size given to init_get_bits() in xan decoder. (cherry picked from commit 393d5031c6aaaf8c2dda4eb5d676974c349fae85)
  smacker demuxer: handle possible av_realloc() failure.
  Fixed segfault with wavpack decoder on corrupted decorrelation terms sub-blocks.
  cljr: init_get_bits size in bits instead of bytes (cherry picked from commit 0c1f5b93d9b97c4cc3684ba91a040e90bfc760d2)
  indeo2: fail if input buffer too small (cherry picked from commit b7ce4f1d1c3add86ece7ca595ea6c4a10b471055)
  indeo2: init_get_bits size in bits instead of bytes (cherry picked from commit 68ca330cbd479111db9cb7649d7530ad59f04cc8)
  ...

Conflicts:
	ffmpeg.c
	libavdevice/alsa-audio.h
	libavformat/gxf.c
	libswscale/x86/swscale_template.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2011-09-22 01:10:24 +02:00
Michael Niedermayer
fa3f7391be h264: allow disabling bitstream overread protection by using the fast flag.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 205c13685f36a17f406b7cd07e14b6d2a61f396a)
2011-09-21 19:49:30 +02:00
Michael Niedermayer
54544100a3 h264: prevent an out of array read in decode_nal_units()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ea0ac11e52b9cf5264f3d4eb7543e760c2a5fbb4)
2011-09-21 18:02:18 +02:00
Michael Niedermayer
97437dada6 h264dec: Prevent CABAC and CAVLC bitsteram overreading
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 23f5cff92cdcfa55a735c458fcb5f95c0e0f3b1f)
2011-09-21 18:02:13 +02:00
Michael Niedermayer
ca5dfd1550 h264: clean all non null elements of delayed_pic[]
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 66ce282df54edb33515d9ef3015b737428b6e0c3)
2011-09-21 17:56:14 +02:00
Michael Niedermayer
1979a9b4f2 h264: change MAX_DELAYED_PIC_COUNT check to av_assert0
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b955ab2f492e3b8c4cbf614f68f8d5bf79e1d1e1)
2011-09-21 17:56:14 +02:00
Jeff Downs
566d26923e h264: fix PCM intra-coded blocks in monochrome case
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 6581e161c5f46733a5619208483de29416eb9a51)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-11 13:03:01 +02:00
Jeff Downs
767efcb46e h264: correct implicit weight table computation for long ref pics
Correct computation of implicit weight tables when referencing pictures
that are marked for long reference.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 87cf70eb237e7586cc7399627dafa1b980ec0b7d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-11 13:02:55 +02:00
Dustin Brody
cc4718196a h264: notice memory allocation failure
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit bac3ab13ea6a9dd8853e79ef3eacf51d234c8774)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-09-11 12:59:09 +02:00
Michael Niedermayer
095946afa7 Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7: (65 commits)
  riff: Add mpgv MPEG-2 fourcc
  Update Changelog
  matroskadec: fix integer underflow if header length < probe length.
  ffmpeg: fix operation with --disable-avfilter
  vf_libopencv: replace opencv/cxtypes.h #include by opencv/cxcore.h
  build: Create mlib optimization directories during out-of-tree builds.
  changelog: misc typo and wording fixes (cherry picked from commit b047941d7da470ba0dcedb1fd0aa828075265ffc)
  doc: Remove outdated comments about gcc 2.95 and gcc 3.3 support. (cherry picked from commit 5ccbf80963c1cc54aed97b1c81b1657ab91baf6a)
  matroskadec: matroska_read_seek after after EBML_STOP leads to failure.
  Update RELEASE file
  update Changelog
  mt: proper locking around release_buffer calls.
  vp8/mt: flush worker thread, not application thread context, on seek.
  docs: Mention the upstream bugzilla url about the dlltool vs MSVC issue
  docs: Use proper markup for a literal command line option
  docs: Don't recommend adding --enable-memalign-hack
  docs: Remove needless configure options
  oggdec: prevent heap corruption.
  oggdec: Abort Ogg header parsing when encountering a data packet.
  Add LGPL license boilerplate to files lacking it.
  ...

Conflicts:
	Changelog
	configure
	doc/developer.texi
	libavcodec/libvpxenc.c
	libavcodec/rawdec.c
	libavfilter/x86/gradfun.c
	libavformat/Makefile
	libavformat/isom.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2011-07-26 00:14:04 +02:00
Jason Garrett-Glaser
e54fd33848 H.264: disable 2tap qpel with CODEC_FLAG2_FAST and >8-bit
2tap qpel isn't implemented yet for high bit depth, so it just breaks decoding.
(cherry picked from commit 9a0dda8b3ab07fa7be60335715a6c350c907a7b8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-06-27 08:39:30 +02:00
Jason Garrett-Glaser
13c943ffb1 H.264: fix 4:4:4 + deblocking + MBAFF
(cherry picked from commit 7c9079ab4cf0bcf34103fc9c5e49ec1fd7dd390c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-06-23 09:14:53 +02:00