14959 Commits

Author SHA1 Message Date
Laurent Aimar
4474051370 eatgv: fix out of bound reads on corrupted motions vectors.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 09302a897d1990b1338f049fcd29638d736b8823)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:29 +01:00
Laurent Aimar
1646d2d2ae eamad: clear FF_INPUT_BUFFER_PADDING_SIZE bytes at the end of the temporary buffer
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 74b9c598396f76407c6b3841c10bc67ddddb2a98)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:23 +01:00
Mans Rullgard
f7be632cbd vp8: fix signed overflows
In addition to avoiding undefined behaviour, an unsigned type
makes more sense for packing multiple 8-bit values.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit bb59156606e00057a706ed30165bc7329db3823f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:12 +01:00
Mans Rullgard
4ba0e03759 motion_est: fix some signed overflows
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit e708afd3c026a9eb547dab07781320a7e2564312)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:09 +01:00
Mans Rullgard
37ce6ba425 dca: fix signed overflow in shift
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 559c244d42be7a02c23976216b47fd63b80d6c7f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:04 +01:00
Mans Rullgard
c2c83dcb32 aacdec: fix undefined shifts
Since nnz can be zero, this is needed to avoid a shift by 32.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit d12294304acd82cb219e3f66ca9cd6efb2194fa4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:58 +01:00
Mans Rullgard
4c5cdb493c put_bits: fix invalid shift by 32 in flush_put_bits()
If flush_put_bits() is called when the 32-bit buffer is empty,
e.g. after writing a multiple of 32 bits, and invalid shift by
32 is performed.  Since flush_put_bits() is called infrequently,
this additional check should have negligible performance impact.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ac6eab1496aad6f8b09deabbef4fe5fd829e142d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:53 +01:00
Laurent Aimar
06b15b3715 h264: fix the size of PPS::chroma_qp_table
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e588a5c2d418fdcf08ece076a2642de44f444d55)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:41 +01:00
Michael Niedermayer
614ef0dc0d h264: fix fill_colmap() to not store entries mbaff style when the reference is not mbaff at all
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a3ba542af39ba4425cbb44d2a0ae09c212b40001)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:34 +01:00
Ronald S. Bultje
5d2b6006f0 mpegvideo: fix position of bottom edge.
It was wrong in colorspaces where horizontal and vertical chroma
subsampling are not the same, e.g. 422.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:58:35 +01:00
Laurent Aimar
b491c15c85 h254: explicitly initialize bit depth/chroma idc
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:53:56 +01:00
Justin Ruggles
2809f4ab93 qcelp: check output buffer size before decoding
(cherry picked from commit e43dd3d2a8e106169e707484090a2d973ece2184)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:43:10 +01:00
Justin Ruggles
c2d017e88f sipr: fix the output data size check and only calculate it once.
(cherry picked from commit 1b5a189f06879338088809b3049ea7620f4e7e78)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:42:59 +01:00
Michael Niedermayer
4f45967cf5 ff_dv_frame_profile2: Check input buffer size.
Based on code by DivX, Inc. / drffmpeg

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 51b0694bc051cda2bfed048a35e694d1047c6ef0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:42:46 +01:00
Justin Ruggles
78eab18740 qdm2: check output buffer size before decoding
(cherry picked from commit 7d49f79f1cd47783a963a757a6563b9cac29db62)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:42:37 +01:00
Justin Ruggles
fc8c0ee09f mpc8: check output buffer size before decoding
(cherry picked from commit 5674d4b0a35a34b75e3533a8580e0b5a0a8895a7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:40:13 +01:00
Justin Ruggles
490617b6ff mpc7: return error if packet is too small.
(cherry picked from commit 8290d1f38b438f1b070de67645c8b4a42014c7ac)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:40:03 +01:00
Justin Ruggles
b833859daa mpc7: check output buffer size before decoding
(cherry picked from commit c8b5c4d27409dfdcec80868686b173ba446c998b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:39:47 +01:00
Gwenole Beauchesne
7d52ed686b vaapi: fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
(cherry picked from commit 825dd135d829d85778e6c37f685a6088fef01b2e)
2011-10-12 11:27:11 +02:00
Gwenole Beauchesne
7275dc28f6 vaapi: fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
(cherry picked from commit 825dd135d829d85778e6c37f685a6088fef01b2e)
2011-10-12 11:26:51 +02:00
Laurent Aimar
f74d1c6de7 h264: do not let invalid values in h->ref_count after a decoder reset.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0333d234b0355b375762447e93674e3fe3c5bff1)
2011-10-11 21:34:15 +02:00
Michael Niedermayer
e49abd1d92 libx264: Fix loop failure due to bufsize becoming 0
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 751a4efd4d754b15ebc6027b09051f27860bb1e8)
2011-10-11 21:34:15 +02:00
Michael Niedermayer
09a288476f H264: hotfix for speedloss on frame threading and h264 files with slices.
This fix is not ideal as it still limits the multithreading on field pictures
to the 2nd field only.
Ill try to fix it properly to allow both fields to decode concurrently but this
needs more work.

This bug exists since and was caused by:
commit ea6331f8bbaf3343faec6ffe77a6218a016a3af5
Author: Ronald S. Bultje <rsbultje@gmail.com>
Date:   Mon Jun 20 10:24:33 2011 -0400

    h264-mt: fix deadlock in packets with multiple slices (e.g. MP4).
(cherry picked from commit eaa21b6870ba8bba4b0370e91f1941307c1c9681)
2011-10-11 21:34:14 +02:00
Michael Niedermayer
7fc85451fd Merge branch 'release/0.8' into release/0.7
* release/0.8: (185 commits)
  h264: fix intra 16x16 mode check when using mbaff and constrained_intra_pred.
  h264: check for invalid bit depth value.
  h264: add entries for 11 and 12 bits in ff_h264_chroma_qp[][]
  h264: fix the check for invalid SPS:num_ref_frames.
  h264: do not let invalid values in h->ref_count on ff_h264_decode_ref_pic_list_reordering() errors.
  Reject video with non multiple of 16 width/height in the 4xm decoder.
  4xm decoder: fix data size for i2 frames.
  4xm decoder: print some error messages in case of errors.
  Check for out of bound accesses in the 4xm decoder.
  Prevent block size from inreasing in the shorten decoder.
  Check for out of bound reads in PTX decoder.
  Clear FF_INPUT_BUFFER_PADDING_SIZE bytes at the end of the temporary buffers used in 4xm decoder.
  Fix the check for missing references in ff_er_frame_end() for H264.
  Prevent NULL dereference when the huffman table is invalid in the 4xm decoder.
  Fix use of uninitialized memory in 4X Technologies demuxer.
  h264: increase ref_poc size to 32 as it can be per field.
  h264: set unused ref_counts to 0 as a precautionary meassure.
  Remove Chnagelog it has nothing to do with reality
  fate: fix motion pixels checksum change caused by backported bugfix
  avienc: Add a limit on the number of skiped frames muxed in a row.
  ...

Conflicts:
	Doxyfile
	RELEASE
	VERSION
	libavformat/utils.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2011-10-02 21:37:59 +02:00
Laurent Aimar
b89a0c9d7f h264: fix intra 16x16 mode check when using mbaff and constrained_intra_pred.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a4fd95b5d511384ed3ce388d8d20a16b1c4c0530)
2011-10-02 21:30:21 +02:00
Laurent Aimar
efedf09378 h264: check for invalid bit depth value.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c2b7f7748be447e1446f63b13da472ba7e00f329)
2011-10-02 21:30:14 +02:00
Laurent Aimar
46edabac3c h264: add entries for 11 and 12 bits in ff_h264_chroma_qp[][]
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 27d3361e34cf19ad5c0bfb7cfacae0db99b26064)
2011-10-02 21:30:08 +02:00
Laurent Aimar
bfd7238adb h264: fix the check for invalid SPS:num_ref_frames.
This patch set the limit to 16.

For information, thoses previous commits:
41f7e2d11d2dca23842ee89d530ca9fa15cec9d8
5cbb0e70a0a2ee99eb3cb09e837b9a1f7355b9bc
assumed it was either 30 or 32.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcf881a6858760ecbd9ff4352a38813dc4232dd6)
2011-10-02 21:29:58 +02:00
Laurent Aimar
cf0052931d h264: do not let invalid values in h->ref_count on ff_h264_decode_ref_pic_list_reordering() errors.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2428b53f6d306d8d71dec34fa7b0af733d76cfac)
2011-10-02 21:29:51 +02:00
Laurent Aimar
6b998720b2 Reject video with non multiple of 16 width/height in the 4xm decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit db5b4875514eb7740844f514dbf236c9179a6a93)
2011-10-02 21:29:45 +02:00
Michael Niedermayer
55a070870f 4xm decoder: fix data size for i2 frames.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0a19b4b0bae772cac7f8341e94c319c96356ee69)
2011-10-02 05:48:40 +02:00
Michael Niedermayer
54a1e7b0f2 4xm decoder: print some error messages in case of errors.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1008f639e2ebaa2b3d48ac765ca49c8869042d75)
2011-10-02 05:48:40 +02:00
Laurent Aimar
2c282e9679 Check for out of bound accesses in the 4xm decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9c661e952fbcbf044709f9a7031c68cc4860336b)
2011-10-02 05:48:26 +02:00
Laurent Aimar
55a96a984e Prevent block size from inreasing in the shorten decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b399cbfba5d901608c18e1a2d48a24c30541a634)
2011-10-02 05:48:13 +02:00
Laurent Aimar
64a9004d07 Check for out of bound reads in PTX decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 581898ae882dc37967b689b6ea5f2b2a9acd257a)
2011-10-02 05:48:07 +02:00
Laurent Aimar
f421b53400 Clear FF_INPUT_BUFFER_PADDING_SIZE bytes at the end of the temporary buffers used in 4xm decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 90a69b2f61cbd2d58723d712bdf283e5e56b2ea2)
2011-10-02 05:47:51 +02:00
Laurent Aimar
d2a276a3fd Fix the check for missing references in ff_er_frame_end() for H264.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-10-02 05:47:46 +02:00
Laurent Aimar
535112b365 Prevent NULL dereference when the huffman table is invalid in the 4xm decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a8ff0636d67c6df059f2ae2df49ad1181de14ca)
2011-10-02 05:45:01 +02:00
Michael Niedermayer
86491c5dbc h264: increase ref_poc size to 32 as it can be per field.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c851ef5a81fb0429453c43fdcfc974ce1b6e447)
2011-10-02 05:44:42 +02:00
Michael Niedermayer
3e0dbb8a7e h264: set unused ref_counts to 0 as a precautionary meassure.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3af2de76ac8b7aa762e4ee129ec29140e6247d81)
2011-10-02 05:44:35 +02:00
Michael Niedermayer
b0804f3705 Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7: (73 commits)
  Update Changelog for 0.7.2 release
  Update RELEASE file for 0.7.2
  lavf: do not set codec_tag for rawvideo
  fate: allow testing with libavfilter disabled
  fate: separate lavf-mxf_d10 test from lavf-mxf
  Fix memory (re)allocation in matroskadec.c, related to MSVR-11-0080.
  movenc: fix NULL reference in mov_write_tkhd_tag
  movenc: create an alternate group for each media type
  flvdec: Check for overflow before allocating arrays
  ppc: fix some pointer to integer casts
  ppc: fix 32-bit PIC build
  rv34: Check for invalid slice offsets
  rv34: Fix potential overreads
  rv34: Avoid NULL dereference on corrupted bitstream
  rv10: Reject slices that does not have the same type as the first one
  lavf: Fix context pointer in av_open_input_stream when avformat_open_input fails
  oggdec: fix out of bound write in the ogg demuxer
  Fixed size given to init_get_bits().
  smacker: fix a few off by 1 errors
  Check for invalid VLC value in smacker decoder.
  ...

Conflicts:
	RELEASE
	libavcodec/avs.c
	libavcodec/ppc/asm.S
	libavcodec/rv34.c
	libavcodec/xan.c
	libavdevice/alsa-audio.h
	libavformat/flvdec.c
	libavformat/gxf.c
	libavformat/utils.c
	libswscale/x86/swscale_template.c
	tests/ref/lavf/mov
	tests/ref/lavf/mxf

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2011-10-01 22:42:41 +02:00
Laurent Aimar
f144a70d60 Fix out of bound reads/writes in the TIFF decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ca5d432e028ffdd4067b87aed6702168c3207b6)
2011-10-01 21:03:49 +02:00
Laurent Aimar
b08df314dc Check for out of bound writes in the QDM2 decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a7876c6e4e62e94d51e364ba99aae4da7671238)
2011-10-01 21:03:45 +02:00
Laurent Aimar
e0fb22cea9 Fix out of bound reads in the QDM2 decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 491eaf35ae1f9b619441314bec33766e31580184)
2011-10-01 21:03:40 +02:00
Laurent Aimar
802045777a Fix out of bound reads due to integer overflow in the ADPCM IMA Electronic Arts EACS decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 346876ec168affe7c21be88d8f1acf1a75cc8409)
2011-10-01 21:03:35 +02:00
Laurent Aimar
e8fd4a43ba Check for out of bound reads in the Electronic Arts CMV decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a5d46235f3f70f0b620f8e54649ece45ecc5b170)
2011-10-01 21:03:31 +02:00
Laurent Aimar
d950461f59 Prevent NULL dereferences when missing the reference frame in the Electronic Arts CMV decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 113d7be62497c4e59db8f224fdb7f0a90cf17d03)
2011-10-01 21:03:26 +02:00
Laurent Aimar
df39708269 Fix potential pointer arithmetic overflows in the Electronic Arts CMV decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e9064c9ce8ed18c3a3aab61e58e663b8f5b0c551)
2011-10-01 20:59:57 +02:00
Laurent Aimar
1f2a93cf4b Prevent infinite loop in the ANM decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39993860e1525ca5d1b07521865b7e9e1b294ed7)
2011-10-01 20:59:49 +02:00
Laurent Aimar
3b840fab90 Check for out of bound reads in AVS decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7afe9e5638242a3210a0fc378e34e3af41e29176)
2011-10-01 20:59:34 +02:00