18748 Commits

Author SHA1 Message Date
Vitor Sessak
280590e338 Plug some memory leaks in the VP6 decoder
Originally committed as revision 22172 to svn://svn.ffmpeg.org/ffmpeg/trunk
(cherry picked from commit 0a41faa9a77dc83d8d933e99f1ba902ecd146e79)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-12-24 16:28:19 +01:00
Laurent Aimar
9767ea7aa7 vp6: Reset the internal state when aborting key frames header parsing
It prevents leaving the state only half initialized.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit a72cad0a6c05aa74940101e937cb3dc602d7d67b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit c76505e0dee0890e39636ddebd2707ab3ea5b8de)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit e28bb18fdc894dfdc1befa9f5e748ccb649a8c76)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-12-24 16:17:42 +01:00
Thierry Foucu
771ceb19f2 vp6: Fix illegal read.
Found with Address Sanitizer

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit e0966eb140b3569b3d6b5b5008961944ef229c06)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit ba4b08b78918f399f9c9524750b26e904d146078)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 94aacaf5083313378c6105bd71db04ce8f62c058)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-12-24 16:14:55 +01:00
Alex Converse
7739947671 vp6: Fix illegal read.
(cherry picked from commit 2a6eb06254df79e96b3d791b6b89b2534ced3119)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 67a7ed623b678a84c992dd7bf3e3d0329f83621b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 8d68083298e2481669de4db0b7b86c915119df6d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-12-24 16:14:41 +01:00
Laurent Aimar
8abf1d882e Fix out of bound reads in the QDM2 decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 5a19acb17ceb71657b0eec51dac651953520e5c8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 0d93d5c4614fafea74bdac681673f5b32eb49063)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-12-24 16:07:49 +01:00
Laurent Aimar
1a53095406 Check for out of bound writes in the QDM2 decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 291d74a46d32183653db07818c7b3407fd50a288)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit a31ccacb1a9b2abc0e140a812fb0ffca6f7c2591)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-12-24 16:07:36 +01:00
Justin Ruggles
60eebf5c12 qdm2: check output buffer size before decoding
(cherry picked from commit 7d49f79f1cd47783a963a757a6563b9cac29db62)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 73472053516f82b7d273a3d42c583f894077a191)

Conflicts:

	libavcodec/qdm2.c
(cherry picked from commit cfb9b47a1ecdc9e88e6561aa213d98245ee70267)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-12-24 16:06:10 +01:00
Baptiste Coudurier
30ee6c1995 Fix qdm2 decoder packet handling to match the api
Originally committed as revision 25767 to svn://svn.ffmpeg.org/ffmpeg/trunk
(cherry picked from commit b26c1a8b7ed1a199b19f92bb5d62c61f1c149215)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-12-24 16:05:52 +01:00
Michael Niedermayer
b769df8ff2 update for 0.5.6
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
n0.5.6
2011-11-21 22:22:04 +01:00
Michael Niedermayer
cc2580e6e9 svq1dec: call avcodec_set_dimensions() after dimensions changed.
Fixes NGS00148

Found-by: Phillip Langlois
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4931c8f0f10bf8dedcf626104a6b85bfefadc6f2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 661ee45f8881bb551eb403472e60c38a7c2818aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-21 22:19:22 +01:00
Michael Niedermayer
4b0f8aed13 qdm2dec: fix buffer overflow.
Fixes NGS00144

This also adds a few lines of code from master that are needed for this fix.

Thanks to Phillip for suggestions to improve the patch.
Found-by: Phillip Langlois
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a6a61a6d1d4da219a6fe29250e2a6b28f9d05524)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-21 22:01:29 +01:00
Michael Niedermayer
2c8ac66456 qdm2dec: check remaining input bits in the mainloop of qdm2_fft_decode_tones()
This is neccessary but likely not sufficient to prevent out of array reads.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 14db3af4f26dad8e6ddf2147e96ccc710952ad4d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8120a1d9bd4bcc4434b4f588f50c9d81aa8ad0e0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-21 21:59:45 +01:00
Michael Niedermayer
4f209fe960 cinepak: check strip_size
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cea0c82d9b9771dfa2ac729c13c0d9e03ea352a7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 211a107208ee636da81d2a89592181e2d78a0c8c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-21 21:59:34 +01:00
Michael Niedermayer
037b1142cd wma: Check channel number before init.
Fixes Ticket240

Based on patch by ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 20431a9982b9bd2c475042d919890a941ad70c71)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-21 21:59:17 +01:00
Alex Converse
37cc48861d vp5: Fix illegal read.
Found with Address Sanitizer
(cherry picked from commit bb4b0ad83b13c3af57675e80163f3f333adef96f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f62fa1ce9f12e4a43b41401a7416c6fa8da579c9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-21 21:57:27 +01:00
Thierry Foucu
0cd61bfa6d vp6: Fix illegal read.
Found with Address Sanitizer

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit e0966eb140b3569b3d6b5b5008961944ef229c06)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8a63deab15ef41fd439be1b46d8dcb73669ccfc1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-21 21:57:09 +01:00
Michael Niedermayer
16ea6af381 0.5: some updates.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
n0.5.5
2011-11-06 20:57:55 +01:00
Michael Niedermayer
8bd374858f resample: Fix array size
Found-by: Jim Radford
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e7db0a9ee758bf0570a141be1fea64f8d9c03db)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit edf3c5a3ebeee8df55c6a05f88a682091f10a364)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Michael Niedermayer
d39cc3c092 resample2: fix potential overflow
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a39b5e8b323785695fb0e3c0f30bd9e24287db87)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Michael Niedermayer
e124c3c298 resample: Fix overflow
Found-by: Jim Radford
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6ae93d030476ddd7fa2ab4d9d2dd25df85725390)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Ronald S. Bultje
8acc0546bb matroskadec: fix out of bounds write
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 723229c11f1400e6a09c8a1c9c27193f376eb1d1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d51c7b4cbe022f6b3b026735dc7e29eb50bbf129)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Michael Niedermayer
36e4be0a0a mem: fix memalign hack av_realloc()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc11927890f38445a950b453d24928525da0e61a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ae87280e219e843c71201c580780e8e30083559)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
c603cf5170 qtrle: check for out of bound writes.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7fb92be7e50ea4ba5712804326c6814ae02dd190)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a65045915f5b4ec6da73df54d1914b320a861223)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
e1a46eff7a qtrle: check for invalid line offset
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a4ed7c3fe9f99b89f86b65710d8855dc572f1a25)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 67c46b9b3027fdd9fd737e21a80d3326748b1c15)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
23aaa82b1d vqa: fix double free on corrupted streams
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e3123856c79c36507772ada1bcda6cfe36a1e297)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Justin Ruggles
58087a4e64 mpc7: return error if packet is too small.
(cherry picked from commit 8290d1f38b438f1b070de67645c8b4a42014c7ac)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 490617b6ffa13f8e49a196a752f927d5ebad6e2b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Justin Ruggles
8d1fa1c97e mpc7: check output buffer size before decoding
(cherry picked from commit c8b5c4d27409dfdcec80868686b173ba446c998b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b833859daa4eb8fe0ec9117859b21a734905b895)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
2eb5f77bc8 h264: do not let invalid values in h->ref_count after a decoder reset.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0333d234b0355b375762447e93674e3fe3c5bff1)
(cherry picked from commit f74d1c6de7ef810544edae947db1eb1e2c7b6361)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
ddbbe500b0 h264: fix the check for invalid SPS:num_ref_frames.
This patch set the limit to 16.

For information, thoses previous commits:
41f7e2d11d2dca23842ee89d530ca9fa15cec9d8
5cbb0e70a0a2ee99eb3cb09e837b9a1f7355b9bc
assumed it was either 30 or 32.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcf881a6858760ecbd9ff4352a38813dc4232dd6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
d1a5b53ede h264: do not let invalid values in h->ref_count on ff_h264_decode_ref_pic_list_reordering() errors.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2428b53f6d306d8d71dec34fa7b0af733d76cfac)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
8e6173c76a Check for out of bound accesses in the 4xm decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9c661e952fbcbf044709f9a7031c68cc4860336b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
1a6f024520 Prevent block size from inreasing in the shorten decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b399cbfba5d901608c18e1a2d48a24c30541a634)
(cherry picked from commit 55a96a984ec65736475a8577a158abc5c48fd50a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
3699a46ed7 Check for out of bound writes in the QDM2 decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a7876c6e4e62e94d51e364ba99aae4da7671238)
(cherry picked from commit b08df314dca6946ed644caacb9d3a533a054c0f6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
62da9203fd Check for out of bound writes in the avs demuxer.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5d44c061cf511d97be5fac8d76be2f3915c6e798)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
2e1e3c1e41 Check for corrupted data in avs demuxer.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1cce7def0a8eff2e7db294b7d195a0fb1a5043b0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:12 +01:00
Laurent Aimar
635256a324 Fix out of bound writes in fix_bitshift() of the shorten decoder.
The data pointers s->decoded[*] already take into account s->nwrap.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f42b3195d3f2692a4dfc0a8668bb4ac35301f2ed)
(cherry picked from commit 107ea3057eb8de8a38c45c2f7181c42ea694b187)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:11 +01:00
Laurent Aimar
240546a185 Check for out of bounds writes in the Delphine Software International CIN decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3035c4034b6af3ad47f921e3385196e1b9d44ddf)
(cherry picked from commit 6e774cf67e6f30feb9b3dec11713d6b6dc0b521c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:11 +01:00
Laurent Aimar
07df40db6e Check for invalid update parameters in vmd video decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7aed1280ea14b60fceae04d71dfd03e1daf2d04)
(cherry picked from commit 1ed90c84f6ab75af91b08436cefb8ea464f8495b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 19:49:11 +01:00
Laurent Aimar
b24c2e59fe Release old pictures after a resolution change in vp5/6 decoder
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dba20b84784a7931b7eac50ced1d43e86801bde9)
(cherry picked from commit c9c6e5f4e8680b7b7801dd6943590ae9cd6bfd89)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 18:40:47 +01:00
Laurent Aimar
25bc1108c2 Check output buffer size in nellymoser decoder.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 741ec30bd2385f794efa9fafa84d39a917f2574e)
(cherry picked from commit 533dbaa55b7d45d5ca76f9ed46f5690282f86ea9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 18:40:19 +01:00
Michael Niedermayer
8ef917c033 check all svq3_get_ue_golomb() returns.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 979bea13003ef489d95d2538ac2fb1c26c6f103b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 18:34:49 +01:00
Michael Niedermayer
1883249be3 rv34: check for size mismatch
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 35f38b3ab9d755aede5bce8abbe1cb9c07027f8a)
(cherry picked from commit ed9e561490d70e317659f9e406c7920242e509eb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 18:32:40 +01:00
Laurent Aimar
648dc68098 Reject audio tracks with invalid interleaver parameters in RM demuxer.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4907f813581acd6cf68f1be9eb163464503e8208)
(cherry picked from commit 24e0a9e451e1aae427307a919d78f6790f4e413c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 18:32:11 +01:00
Michael Niedermayer
10da0edddc Merge remote-tracking branch 'qatar/release/0.5' into release/0.5
* qatar/release/0.5:
  update version
  Release notes and changelog for 0.5.5
  Fix ff_imdct_calc_sse() on gcc-4.6
  Make DECLARE_ALIGNED macros work with external array specifiers
  Fix MMX rgb24 to yuv conversion with gcc 4.6

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 01:35:41 +01:00
Reinhard Tartler
d6bf79993f update version 2011-11-05 12:57:22 +01:00
Reinhard Tartler
0f2735e839 Release notes and changelog for 0.5.5 2011-11-05 12:53:16 +01:00
Alex Converse
9463a28792 Fix ff_imdct_calc_sse() on gcc-4.6
Gcc 4.6 only preserves the first value when using an array with an "m"
constraint.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 770c410fbb8e1b87ce8ad7f3d7eddaa55e2b8295)

Conflicts:

	libavcodec/x86/fft_sse.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-11-05 11:29:12 +01:00
Måns Rullgård
1330a8a1cb Make DECLARE_ALIGNED macros work with external array specifiers
The macro implementation might need the name of the variable being
declared for compiler-specific syntax.  Moving array specifiers outside
the macro invocation allows this to work.

Originally committed as revision 21363 to svn://svn.ffmpeg.org/ffmpeg/trunk
(cherry picked from commit 8a24e98d506f0f44ec58e06291fa0fce703fb6a8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-11-05 11:29:12 +01:00
Mans Rullgard
8135c35528 Fix MMX rgb24 to yuv conversion with gcc 4.6
When built with gcc 4.6, the MMX rgb24 to yuv conversion gives
wrong output.  The compiler produces this warning:

libswscale/swscale_template.c:1885:5: warning: use of memory input without lvalue in asm operand 4 is deprecated

Changing the memory operand to a register makes it work.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit f344903ca5ce28a833fdd656bc1ed5b16d97e7e9)

Conflicts:

	libswscale/swscale_template.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-11-05 09:04:11 +01:00
Michael Niedermayer
38423fe0b7 smacker: add forgotten *
found by fenrir

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f98edc73c599badaa0c075fbffb519a150d03d80)
2011-11-03 03:52:55 +01:00