Commit Graph

38622 Commits

Author SHA1 Message Date
Ronald S. Bultje
0ce4fe482c h264: error out on invalid bitdepth.
Fixes invalid reads while initializing the dequant tables, which uses
the bit depth to determine the QP table size.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-01 12:15:51 -08:00
Christophe Gisquet
cc412b7104 aacsbr: use a swap index for the Y matrix rather than copy buffers.
Signed-off-by: Alex Converse <alex.converse@gmail.com>
2012-03-01 11:03:00 -08:00
Michael Niedermayer
dc945b1fa8 eatgq: Pass error code from tgq_decode_mb() and let the caller fail.
This fixes a over read.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 19:57:55 +01:00
Michael Niedermayer
32f0c65828 vc1: fix out of array reads in vc1_inv_trans_4x4_c()
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 19:35:12 +01:00
Michael Niedermayer
80c702efeb vc1: fix out of array reads in vc1_inv_trans_4x8_c()
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 19:35:12 +01:00
Michael Niedermayer
af796ba4b8 vc1: fix out of array reads in vc1_inv_trans_8x4_c()
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 19:35:12 +01:00
Ronald S. Bultje
63c9de6469 huffyuv: do not abort on unknown pix_fmt; instead, return an error.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-01 10:23:36 -08:00
Michael Niedermayer
b2a7c01733 mpc: Fix mpc_CC table and use.
This is based on the reference implementation and fixes
a global out of array read.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 16:16:13 +01:00
Michael Niedermayer
b84211ac71 mpc: Fix mpc_SCF use and content.
This fixes a out of global array read.
This change is based on the reference mpc imlementation.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 16:15:33 +01:00
Ronald S. Bultje
bd17a40a7e lcl: return negative error codes on decode_init() errors.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-03-01 07:12:29 -08:00
Martin Storsjö
984b914c55 rtpenc: Use MB info side data for splitting H263 packets for RFC 2190
This makes the packetization spec compliant for cases where one single
GOB doesn't fit into an RTP packet.

Signed-off-by: Martin Storsjö <martin@martin.st>
2012-03-01 16:08:32 +02:00
Martin Storsjö
bdc1220eeb h263enc: Add an option for outputting info about MBs as side data
Signed-off-by: Martin Storsjö <martin@martin.st>
2012-03-01 16:08:31 +02:00
Martin Storsjö
442c1320e7 avpacket: Add a function for shrinking already allocated side data
Signed-off-by: Martin Storsjö <martin@martin.st>
2012-03-01 16:08:30 +02:00
Michael Niedermayer
fd88a25701 rv34dsp: avoid use of crop table for idct.
Fixes out of array read.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 07:06:53 +01:00
Michael Niedermayer
8263212e86 mpegaudiodec: Enable checked bitstream reader.
It appears there are corner cases with damaged input that can lead
to small overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 06:39:48 +01:00
Michael Niedermayer
a057ef6923 mpegaudiodec: change granule skip code to avoid false checked bitstream reader errors.
Code ported from qatar/master, please see there for per line authorship.
Main authors AFAIK are Ronald and Justin. I have no authorship on this.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 06:35:55 +01:00
Michael Niedermayer
436f866f92 svq3dec: fix overread of the cliping table.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 05:25:11 +01:00
Michael Niedermayer
d2101bffa0 eval: fix infinite loop on opensolaris x86.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 03:24:45 +01:00
Michael Niedermayer
79ae084e9b Merge remote-tracking branch 'qatar/master'
* qatar/master: (58 commits)
  amrnbdec: check frame size before decoding.
  cscd: use negative error values to indicate decode_init() failures.
  h264: prevent overreads in intra PCM decoding.
  FATE: do not decode audio in the nuv test.
  dxa: set audio stream time base using the sample rate
  psx-str: do not allow seeking by bytes
  asfdec: Do not set AVCodecContext.frame_size
  vqf: set packet parameters after av_new_packet()
  mpegaudiodec: use DSPUtil.butterflies_float().
  FATE: add mp3 test for sample that exhibited false overreads
  fate: add cdxl test for bit line plane arrangement
  vmnc: return error on decode_init() failure.
  libvorbis: add/update error messages
  libvorbis: use AVFifoBuffer for output packet buffer
  libvorbis: remove unneeded e_o_s check
  libvorbis: check return values for functions that can return errors
  libvorbis: use float input instead of s16
  libvorbis: do not flush libvorbis analysis if dsp state was not initialized
  libvorbis: use VBR by default, with default quality of 3
  libvorbis: fix use of minrate/maxrate AVOptions
  ...

Conflicts:
	Changelog
	doc/APIchanges
	libavcodec/avcodec.h
	libavcodec/dpxenc.c
	libavcodec/libvorbis.c
	libavcodec/vmnc.c
	libavformat/asfdec.c
	libavformat/id3v2enc.c
	libavformat/internal.h
	libavformat/mp3enc.c
	libavformat/utils.c
	libavformat/version.h
	libswscale/utils.c
	tests/fate/video.mak
	tests/ref/fate/nuv
	tests/ref/fate/prores-alpha
	tests/ref/lavf/ffm
	tests/ref/vsynth1/prores
	tests/ref/vsynth2/prores

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-01 03:17:11 +01:00
Vitor Sessak
5cd1337f5d nellymoserdec: Saner and faster IMDCT windowing
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
2012-02-29 15:38:35 -08:00
Vitor Sessak
882abda5a2 amrnbdec: check frame size before decoding.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
2012-02-29 14:13:58 -08:00
Ronald S. Bultje
8a9faf33f2 cscd: use negative error values to indicate decode_init() failures.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-02-29 14:04:23 -08:00
Ronald S. Bultje
d1604b3de9 h264: prevent overreads in intra PCM decoding.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-02-29 13:17:34 -08:00
Justin Ruggles
f240df6a74 FATE: do not decode audio in the nuv test.
We already have sufficient coverage for 16-bit pcm.
2012-02-29 15:45:50 -05:00
Justin Ruggles
929dd8c108 dxa: set audio stream time base using the sample rate 2012-02-29 15:45:50 -05:00
Justin Ruggles
aa831c4093 psx-str: do not allow seeking by bytes 2012-02-29 15:45:50 -05:00
Justin Ruggles
bdbf1fa405 asfdec: Do not set AVCodecContext.frame_size 2012-02-29 15:45:50 -05:00
Justin Ruggles
4bf6775e9d vqf: set packet parameters after av_new_packet()
Otherwise the values are overwritten.
2012-02-29 15:45:50 -05:00
Michael Niedermayer
a77c8ade2e lavc: fix 10l oversight in realloc of avcodec_encode_video2.
Packets are not guranteed to be allocated by av_malloc().

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-02-29 21:33:00 +01:00
Michael Niedermayer
d7bce4a274 dca: dont overread dca_default_coeffs.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-02-29 21:18:54 +01:00
Michael Niedermayer
51db9a97e9 dca: Check scale_sum.
Fixes a out of array read.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-02-29 21:18:54 +01:00
Michael Niedermayer
d6bc273bac dca: Check LFEScaleIndex.
Its not clear from the spec what to do with values larger than 127
so iam opting for the safe side and ask for a sample.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-02-29 21:18:54 +01:00
Vitor Sessak
9b4cd58611 mpegaudiodec: use DSPUtil.butterflies_float().
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
2012-02-29 12:12:50 -08:00
Justin Ruggles
841c17177b FATE: add mp3 test for sample that exhibited false overreads
related to b716542691
Error messages and audible artifacts were fixed in that commit.
2012-02-29 15:12:18 -05:00
Paul B Mahol
31b132c094 fate: add cdxl test for bit line plane arrangement
Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
2012-02-29 15:11:05 -05:00
Ronald S. Bultje
07a180972f vmnc: return error on decode_init() failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-02-29 11:58:27 -08:00
Justin Ruggles
a45a1ea521 libvorbis: add/update error messages
also use AVERROR codes for some return values instead of -1
2012-02-29 14:54:25 -05:00
Justin Ruggles
592c4dbc7e libvorbis: use AVFifoBuffer for output packet buffer
simplifies the code and does less memmove()
2012-02-29 14:54:24 -05:00
Justin Ruggles
1fe7c1be54 libvorbis: remove unneeded e_o_s check
vorbis_bitrate_flushpacket() does not return any packets that should not be
output in the bitstream.
2012-02-29 14:54:24 -05:00
Justin Ruggles
94025d8a99 libvorbis: check return values for functions that can return errors 2012-02-29 14:54:24 -05:00
Justin Ruggles
c5063e0348 libvorbis: use float input instead of s16
libvorbis takes float input, so we can just deinterleave/reorder the input
as-is instead of also converting.
2012-02-29 14:54:24 -05:00
Justin Ruggles
f15c4281dc libvorbis: do not flush libvorbis analysis if dsp state was not initialized
Fixes a segfault if init() fails before initializing the dsp state
2012-02-29 14:54:23 -05:00
Justin Ruggles
147ff24a0e libvorbis: use VBR by default, with default quality of 3 2012-02-29 14:54:20 -05:00
Justin Ruggles
182d4f1f38 libvorbis: fix use of minrate/maxrate AVOptions
- enable the options for audio encoding
- properly check for user-set maxrate
- use correct calling order in vorbis_encode_setup_managed()
2012-02-29 14:44:15 -05:00
Justin Ruggles
eb35ef2932 libvorbis: cosmetics: renaming/pretty-printing/comments/unused code 2012-02-29 14:44:15 -05:00
Justin Ruggles
4e99501f62 (e)ac3enc: select a default bit rate based on the channel layout 2012-02-29 14:36:00 -05:00
Justin Ruggles
6aeea1dfb2 ac3enc: choose the closest bit rate to the one requested instead of failing 2012-02-29 14:36:00 -05:00
Ronald S. Bultje
78e9852a2e rpza: error out on buffer overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-02-29 11:21:01 -08:00
Ronald S. Bultje
e54ae60e46 qtrle: return error on decode_init() failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-02-29 11:20:29 -08:00
Ronald S. Bultje
791de61bbb swscale: fix another integer overflow.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-02-29 11:19:03 -08:00