56 Commits

Author SHA1 Message Date
Michael Niedermayer
1883249be3 rv34: check for size mismatch
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 35f38b3ab9d755aede5bce8abbe1cb9c07027f8a)
(cherry picked from commit ed9e561490d70e317659f9e406c7920242e509eb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-06 18:32:40 +01:00
Kostya Shishkov
d8439f0452 rv34: check that subsequent slices have the same type as first one.
This prevents some crashes when corrupted bitstream reports e.g. P-type
slice in I-frame. Official RealVideo decoder demands all slices to be
of the same type too.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 23a1f0c59241465ba30103388029a7afc0ead909)
2011-11-03 03:52:55 +01:00
Laurent Aimar
a9ded3d272 rv34: Check for invalid slice offsets
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 4cc7732386eb36661ed22d1200339b38a5fa60bc)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-11-03 03:29:32 +01:00
Laurent Aimar
ef93642aac rv34: Avoid NULL dereference on corrupted bitstream
rv34_decode_slice() can return without allocating any pictures.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d0f6ab0298f2309c6104626787ed73416298b019)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2011-11-03 03:29:21 +01:00
Kostya Shishkov
808f9ce727 Call avcodec_set_dimensions() instead of simply setting avctx->width/height
when frame dimensions change in RV3/4.

Originally committed as revision 20595 to svn://svn.ffmpeg.org/ffmpeg/trunk
(cherry picked from commit d90aeeaf569e4a08c30b3d1d09c3cff3a86eb431)
2011-02-19 17:07:58 +01:00
Kostya Shishkov
44511b17cb Update dimensions in AVCodecContext when RV3/4 frame dimensions change
Originally committed as revision 20572 to svn://svn.ffmpeg.org/ffmpeg/trunk
(cherry picked from commit ec10d2d53999f6edf7d7b5ac88df263eccfb1fb0)

Fixes heap corruption crashes

Addresses: CVE-2011-0722
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2011-02-04 06:42:29 +01:00
Kostya Shishkov
7a47495079 cosmetics: reindent after last commit
Originally committed as revision 17585 to svn://svn.ffmpeg.org/ffmpeg/trunk
2009-02-25 08:36:00 +00:00
Kostya Shishkov
77f7156d85 Since motion vectors for intra blocks are always zero, do not try to retrieve them.
Originally committed as revision 17584 to svn://svn.ffmpeg.org/ffmpeg/trunk
2009-02-25 08:35:37 +00:00
Kostya Shishkov
7fa70598e8 Enable dropping frames for RV3/4
Originally committed as revision 17221 to svn://svn.ffmpeg.org/ffmpeg/trunk
2009-02-14 06:38:50 +00:00
Kostya Shishkov
86dd3974e5 In RV3/4 nonzero MB offset for the first slice also indicates error
Originally committed as revision 17201 to svn://svn.ffmpeg.org/ffmpeg/trunk
2009-02-13 07:06:36 +00:00
Kostya Shishkov
432f1f11ea Move RV3/4 frame type check up so 1-slice unanchored B-frames are checked, too.
Originally committed as revision 17031 to svn://svn.ffmpeg.org/ffmpeg/trunk
2009-02-07 10:46:51 +00:00
Kostya Shishkov
3ffabd4e49 Silence two pointer assignment compiler warnings in rv34.c
Originally committed as revision 16916 to svn://svn.ffmpeg.org/ffmpeg/trunk
2009-02-01 15:03:40 +00:00
Diego Biurrun
bad5537e2c Use full internal pathname in doxygen @file directives.
Otherwise doxygen complains about ambiguous filenames when files exist
under the same name in different subdirectories.

Originally committed as revision 16912 to svn://svn.ffmpeg.org/ffmpeg/trunk
2009-02-01 02:00:19 +00:00
Diego Biurrun
dc8a7c93d3 Add missing void keyword to parameterless function declarations.
Originally committed as revision 16860 to svn://svn.ffmpeg.org/ffmpeg/trunk
2009-01-30 18:38:25 +00:00
Kostya Shishkov
1aa7335149 Fix Doxygen comments for RV3/4 decoder.
Originally committed as revision 16842 to svn://svn.ffmpeg.org/ffmpeg/trunk
2009-01-28 07:16:20 +00:00
Aurelien Jacobs
199436b952 moves mid_pred() into mathops.h (with arch specific code split by directory)
Originally committed as revision 16681 to svn://svn.ffmpeg.org/ffmpeg/trunk
2009-01-18 22:57:40 +00:00
Kostya Shishkov
592beefe99 It turned out that RV30 uses motion vectors for forward motion B-frame
macroblocks to predict motion vectors for backward motion B-frame macroblocks
and vice versa.

Originally committed as revision 16381 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-12-29 10:20:03 +00:00
Kostya Shishkov
ffedb5c19a 352l: correct calculating number of bits for storing macroblock offset in RV3/4
Originally committed as revision 16293 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-12-23 18:27:06 +00:00
Kostya Shishkov
17b17c53f4 add const qualifier to some pointers for input data
Originally committed as revision 16195 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-12-17 20:17:07 +00:00
Kostya Shishkov
bb39171beb use precalculated mb_pos where available
Originally committed as revision 16194 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-12-17 20:10:15 +00:00
Kostya Shishkov
6c3fca6479 Merge deblocking pattern with CBP for RV30/40 loop filtering
Originally committed as revision 16192 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-12-17 20:00:48 +00:00
Kostya Shishkov
26bc319c1d Generate correct deblock pattern for RV30
Originally committed as revision 16191 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-12-17 19:39:56 +00:00
Kostya Shishkov
0dfcc36818 Do not attempt to decode RV30/40 B-frames without anchors.
This fixes issue 747.

Originally committed as revision 16031 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-12-07 16:30:08 +00:00
Kostya Shishkov
3e16cde430 Fix reading out of buffer during RV30/40 deblock mask calculation
Originally committed as revision 15983 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-12-02 18:14:53 +00:00
Kostya Shishkov
958a2b5d6b 34l: forgot to change one case of chroma MC for RV40
Originally committed as revision 15982 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-12-02 18:12:48 +00:00
Kostya Shishkov
1799d5223e Check RV30/40 slice offsets to be inside buffer.
This fixes issue 738

Originally committed as revision 15981 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-12-02 17:39:20 +00:00
Kostya Shishkov
a1c1c78019 Update ff_rv34_decode_frame() arguments definition
Originally committed as revision 15980 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-12-02 17:35:38 +00:00
Kostya Shishkov
de8cac167d Invoke future RV30/40 loop filter for already decoded rows instead of
the whole frame at once.

Originally committed as revision 15949 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-11-29 06:22:31 +00:00
Kostya Shishkov
f653095bdd Fix RV40 chroma MC
Originally committed as revision 15943 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-11-27 07:26:26 +00:00
Kostya Shishkov
cd98279126 33l Fix applying residue condition in RV30/40 decoder
Originally committed as revision 15891 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-11-20 14:07:37 +00:00
Kostya Shishkov
78a0fa8c8f Eliminate direct use of s->current_picture in RV30/40 decoder
Originally committed as revision 15890 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-11-20 14:05:29 +00:00
Kostya Shishkov
7690351cf3 Use wider margins for edge emulation in RV30/40
Originally committed as revision 15889 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-11-20 14:03:35 +00:00
Diego Biurrun
52ece41057 Remove unused variable.
Originally committed as revision 15803 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-11-12 17:29:02 +00:00
Kostya Shishkov
502ecc97af Split RV3/4 deblock pattern into horizontal and vertical parts
during calculating.

Originally committed as revision 15794 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-11-08 18:15:13 +00:00
Kostya Shishkov
ede0a5f973 Drop redundant check in RV3/4 deblock coefficients calculation
Originally committed as revision 15792 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-11-08 07:24:56 +00:00
Kostya Shishkov
e122311261 Calculating an additional MV-based deblocking pattern is the same
for both RV3 and RV4, so move it into common code.

Originally committed as revision 15786 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-11-07 07:18:22 +00:00
Kostya Shishkov
b215365817 Introduce RV3-specific motion vector prediction.
Now B-frames in RV3 look almost correct.

Originally committed as revision 15764 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-11-01 17:57:44 +00:00
Kostya Shishkov
5f621dd28a Correct motion vector scaling in B-frames for RV3/4
Originally committed as revision 15747 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-10-30 06:08:47 +00:00
Kostya Shishkov
2ee054c215 Fix chroma motion compensation for RV30
Originally committed as revision 15731 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-10-27 19:07:39 +00:00
Kostya Shishkov
60ce2f9cae Choose RV3/4 DC quantizer correctly
Originally committed as revision 15675 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-10-24 06:17:18 +00:00
Kostya Shishkov
ccfe8ae310 34l: free RV3/4 context variables at the end
Originally committed as revision 15652 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-10-20 05:59:50 +00:00
Kostya Shishkov
def4f63b85 RV3 and RV4 decoders set some deblocking coefs for each macroblock,
so store them in the context and register a function to calculate them.

Originally committed as revision 15651 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-10-20 05:58:05 +00:00
Kostya Shishkov
c0b821befb Mark some of RV3/4 block types as having separate DC subblock
Originally committed as revision 15650 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-10-20 05:54:31 +00:00
Diego Biurrun
ccd425e799 Remove unnecessary parentheses from return calls.
Originally committed as revision 13069 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-05-06 09:16:36 +00:00
Kostya Shishkov
39902a8c93 Calculate motion vector information based on PTS provided in slice header
Originally committed as revision 13011 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-04-28 13:37:36 +00:00
Kostya Shishkov
86c90cc7a6 Skip blocks in B-frames reuse motion vectors from next reference frame.
So if referenced blocks is 16x8, 8x16 or 8x8 partitions, skip block
will have them too.

Originally committed as revision 12987 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-04-26 13:09:36 +00:00
Zuxy Meng
98a6fff98c Apply 'cold' attribute to init/uninit functions in libavcodec
Originally committed as revision 12525 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-03-21 03:11:20 +00:00
Aurelien Jacobs
9701840bb5 add FF_ prefix to all (frame)_TYPE usage
Originally committed as revision 12399 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-03-09 23:31:02 +00:00
Kostya Shishkov
34093ab58e Save quantizer values as well
Originally committed as revision 11555 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-01-18 07:04:58 +00:00
Kostya Shishkov
b008e6b869 Save coded block patterns for future loop filtering.
Originally committed as revision 11554 to svn://svn.ffmpeg.org/ffmpeg/trunk
2008-01-18 07:04:11 +00:00