When the file to decode contains a sequence of binary values like
"1101110...", decode_frame() was reading the sequence of digits like a
unique integer value, which was resulting in integer overflows.
The change add support for parsing non-space-separated pixel digits
for mono formats, in particular fix decoding of file battrace.pbm, and
fix trac issue #154.
All decode_channel_map calls together can easily read
more data than the amount of padding available.
Thus below patch adds an input length check before reading them.
Fixes some invalid reads with sample from
http://bugzilla.mplayerhq.hu/show_bug.cgi?id=1138
This is required specifically for setting frame->format to -1,
otherwise it will be set to 0 = PIX_FMT_YUV420P and code reading
the format from the output decoded frame will get misled.
In particular fix regressions occurring with the pending vsrc_buffer
patch.
The format is a per-frame property, having it in AVFrame simplify the
operation of extraction of that information, since avoids the need to
access the codec/stream context.
width and height are per-frame properties, setting these values in
AVFrame simplify the operation of extraction of that information,
since avoids the need to check the codec/stream context.
* qatar/master: (23 commits)
doc: Check standalone compilation before submitting new components.
Fix standalone compilation of pipe protocol.
Fix standalone compilation of ac3_fixed encoder.
Fix standalone compilation of binkaudio_dct / binkaudio_rdft decoders.
Fix standalone compilation of IMC decoder.
Fix standalone compilation of WTV demuxer.
Fix standalone compilation of MXPEG decoder.
flashsv: K&R cosmetics
matroskaenc: fix memory leak
vc1: make overlap filter for I-frames bit-exact.
vc1dec: use s->start/end_mb_y instead of passing them as function args.
Revert "VC1: merge idct8x8, coeff adjustments and put_pixels."
Replace strncpy() with av_strlcpy().
indeo3: Eliminate use of long.
get_bits: make cache unsigned to eliminate undefined signed overflow.
asfdec: fix assert failure on invalid files
avfilter: check malloc return values.
Not pulled as reason for reindent is not pulled: mpegvideo: reindent.
nutenc: check malloc return values.
Not pulled due to much simpler solution in ffmpeg *: don't av_malloc(0).
...
Conflicts:
doc/developer.texi
libavcodec/Makefile
libavcodec/get_bits.h
libavcodec/mpegvideo.c
libavformat/Makefile
libavutil/log.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This reverts commit f8bed30d8b. The reason
for this is that the overlap filter, which runs after IDCT, should run
on unclamped values, and thus IDCT and put_pixels() cannot be merged if
we want to attempt to be bitexact.
