100 Commits

Author SHA1 Message Date
Michael Niedermayer
17ecc8a771 avcodec/h264_ps: More completely check the bit depths
Fixes out of array read
Fixes: asan_static-oob_30328b6_719_cov_3325483287_H264_artifacts_motion.h264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69aa79365c1e8e1cb597d33e77bf1062c2ef47d4)

Conflicts:

	libavcodec/h264_ps.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-06 04:41:12 +01:00
Michael Niedermayer
70a73213b7 Merge commit 'c18838f5eb7d7001a9dc653f5162868c04c1b2a1'
* commit 'c18838f5eb7d7001a9dc653f5162868c04c1b2a1':
  h264_ps: Use more meaningful error values

Conflicts:
	libavcodec/h264_ps.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 13:03:40 +02:00
Michael Niedermayer
e853cf5325 Merge commit '330ad1f6a53a37dec228cb424ca57e1268fafc64'
* commit '330ad1f6a53a37dec228cb424ca57e1268fafc64':
  h264_ps: K&R formatting cosmetics

Conflicts:
	libavcodec/h264_ps.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 12:56:37 +02:00
Michael Niedermayer
8299ed261a Merge commit '8fed466b0a7d636ae5035f9c6074fba9a621539b'
* commit '8fed466b0a7d636ae5035f9c6074fba9a621539b':
  h264_ps: Drop commented-out cruft

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 12:32:33 +02:00
Michael Niedermayer
58e12732db Merge commit '2a61592573d725956a4377641344afe263382648'
* commit '2a61592573d725956a4377641344afe263382648':
  avcodec: Remove some commented-out debug cruft

Conflicts:
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_loopfilter.c
	libavcodec/h264_mvpred.h
	libavcodec/mjpegdec.c
	libavcodec/mjpegenc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-21 11:18:43 +02:00
Diego Biurrun
c18838f5eb h264_ps: Use more meaningful error values 2013-08-20 21:24:42 +02:00
Diego Biurrun
330ad1f6a5 h264_ps: K&R formatting cosmetics 2013-08-20 21:23:24 +02:00
Diego Biurrun
8fed466b0a h264_ps: Drop commented-out cruft 2013-08-20 21:09:05 +02:00
Diego Biurrun
2a61592573 avcodec: Remove some commented-out debug cruft 2013-08-20 19:59:50 +02:00
Michael Niedermayer
00b1401df6 avcodec/h264_ps: try to support Ambarella AVC
Fixes Ticket2763

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-09 05:21:26 +02:00
Michael Niedermayer
6952e2f82a h264_ps: fix memleak in ff_h264_decode_picture_parameter_set()
Fixes: CID1026763
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-06-03 14:09:12 +02:00
Michael Niedermayer
edabbfbaf2 ff_h264_decode_seq_parameter_set: dont set h->sps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-06 00:37:47 +02:00
Michael Niedermayer
ccdaa31d86 ff_h264_decode_picture_parameter_set: fix return code
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-06 00:01:47 +02:00
Michael Niedermayer
dc35b77b28 ff_h264_decode_picture_parameter_set: use the correct sps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-05-06 00:01:05 +02:00
Michael Niedermayer
c3bd306e78 h264_ps: check croping values
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-04-30 23:48:53 +02:00
Michael Niedermayer
f4b05cd841 Merge commit '5e83d9aced2fc2b2e1360452794c58aba55d497c'
* commit '5e83d9aced2fc2b2e1360452794c58aba55d497c':
  h264: fully support cropping.

Conflicts:
	doc/APIchanges
	libavcodec/h264.c
	libavcodec/h264_ps.c
	libavcodec/options_table.h
	libavcodec/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-04-19 22:46:51 +02:00
Anton Khirnov
5e83d9aced h264: fully support cropping.
Based on a patch by Vittorio Giovara <vittorio.giovara@gmail.com>

Fixes Bug 378.
2013-04-19 09:28:08 +02:00
Michael Niedermayer
a984efd104 Merge commit 'c242bbd8b6939507a1a6fb64101b0553d92d303f'
* commit 'c242bbd8b6939507a1a6fb64101b0553d92d303f':
  Remove unnecessary dsputil.h #includes

Conflicts:
	libavcodec/ffv1.c
	libavcodec/h261dec.c
	libavcodec/h261enc.c
	libavcodec/h264pred.c
	libavcodec/lpc.h
	libavcodec/mjpegdec.c
	libavcodec/rectangle.h
	libavcodec/x86/idct_sse2_xvid.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-26 13:05:10 +01:00
Diego Biurrun
c242bbd8b6 Remove unnecessary dsputil.h #includes 2013-02-26 00:51:34 +01:00
Michael Niedermayer
bdeb61ccc6 h264: check that luma and chroma depth match
Fixes out of array access

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-21 15:02:09 +01:00
Michael Niedermayer
b7fe35c9e5 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  h264: deMpegEncContextize

Conflicts:
	libavcodec/dxva2_h264.c
	libavcodec/h264.c
	libavcodec/h264.h
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/h264_loopfilter.c
	libavcodec/h264_mb_template.c
	libavcodec/h264_parser.c
	libavcodec/h264_ps.c
	libavcodec/h264_refs.c
	libavcodec/h264_sei.c
	libavcodec/svq3.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-16 23:30:09 +01:00
Anton Khirnov
2c54155407 h264: deMpegEncContextize
Most of the changes are just trivial are just trivial replacements of
fields from MpegEncContext with equivalent fields in H264Context.
Everything in h264* other than h264.c are those trivial changes.

The nontrivial parts are:
1) extracting a simplified version of the frame management code from
   mpegvideo.c. We don't need last/next_picture anymore, since h264 uses
   its own more complex system already and those were set only to appease
   the mpegvideo parts.
2) some tables that need to be allocated/freed in appropriate places.
3) hwaccels -- mostly trivial replacements.
   for dxva, the draw_horiz_band() call is moved from
   ff_dxva2_common_end_frame() to per-codec end_frame() callbacks,
   because it's now different for h264 and MpegEncContext-based
   decoders.
4) svq3 -- it does not use h264 complex reference system, so I just
   added some very simplistic frame management instead and dropped the
   use of ff_h264_frame_start(). Because of this I also had to move some
   initialization code to svq3.

Additional fixes for chroma format and bit depth changes by
Janne Grunau <janne-libav@jannau.net>

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-02-15 16:35:16 +01:00
Michael Niedermayer
25be63005f Merge commit 'b85a5e87af4254b80913fe33591d96361f30832b'
* commit 'b85a5e87af4254b80913fe33591d96361f30832b':
  lavu: Add av_strnstr()
  h264: Allow discarding the cropping information from SPS

Conflicts:
	Changelog
	doc/APIchanges
	libavcodec/avcodec.h
	libavcodec/version.h
	libavutil/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-01-25 14:36:45 +01:00
Vittorio Giovara
a84fb6e06f h264: Allow discarding the cropping information from SPS
Some 3D systems overload the meaning of the field for other purposes.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-01-25 07:50:35 +01:00
Michael Niedermayer
40ea343866 Merge commit 'c661cb6672af5ebcb900ec8766b24761bd2ab011'
* commit 'c661cb6672af5ebcb900ec8766b24761bd2ab011':
  cmdutils: pass number of groups to split_commandline().
  mov: handle h263 and flv1 for codec_tag 'H','2','6','3'
  h264: fix sps parsing for SVC and CAVLC 4:4:4 Intra profiles

Conflicts:
	libavcodec/h264_ps.c
	libavformat/isom.c
	libavformat/mov.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-20 12:06:15 +01:00
Victor Lopez
1c8bf3bfed h264: fix sps parsing for SVC and CAVLC 4:4:4 Intra profiles
Fixes bug 396.

CC: libav-stable@libav.org
2012-12-19 13:40:48 +01:00
Janne Grunau
a421bbfe83 h264: fix memleak on error during SPS parsing
Introduced in d7d6efe42b0d.
2012-12-15 19:16:39 +01:00
Michael Niedermayer
b9d887c225 Merge commit '072be3e8969f24113d599444be4d6a0ed04a6602'
* commit '072be3e8969f24113d599444be4d6a0ed04a6602':
  h264: set parameters from SPS whenever it changes
  asyncts: cosmetics: reindent

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-14 15:17:51 +01:00
Janne Grunau
072be3e896 h264: set parameters from SPS whenever it changes
Fixes a crash in the fuzzed sample sample_varPAR.avi_s26638 with
alternating bit depths.
2012-12-13 21:02:42 +01:00
Michael Niedermayer
7c425e4f2d Merge commit 'd7d6efe42b0d2057e67999b96b9a391f533d2333'
* commit 'd7d6efe42b0d2057e67999b96b9a391f533d2333':
  h264: check sps.log2_max_frame_num for validity
  mov: validate number of DataReferenceBox entries against box size
  mov: compute avg_frame_rate only if duration is known
  flac: change minimum and default of lpc_passes option to 1

Conflicts:
	libavcodec/h264_ps.c
	libavformat/mov.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-07 15:52:43 +01:00
Janne Grunau
d7d6efe42b h264: check sps.log2_max_frame_num for validity
Fixes infinite or long taking loop in frame num gap code in
the fuzzed sample bipbop234.ts_s223302.

CC: libav-stable@libav.org
2012-12-07 11:43:28 +01:00
Michael Niedermayer
039f4ff4c2 h264: 2nd try at getting sm2v working
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-11-11 12:50:03 +01:00
Michael Niedermayer
991f6bf93d h264: Support SMV2
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-11-11 02:51:23 +01:00
Hendrik Leppkes
572781b25f h264: fix parsing of old lossless profile (profile_idc == 144)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-10-14 16:04:10 +02:00
Michael Niedermayer
ae0312ab2e h264: print num_reorder_frames in debug output
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-08-04 15:42:44 +02:00
Michael Niedermayer
b5da7d4c1a Merge remote-tracking branch 'qatar/master'
* qatar/master:
  avformat: Drop pointless "format" from container long names
  swscale: bury one more piece of inline asm under HAVE_INLINE_ASM.
  wv: K&R formatting cosmetics
  configure: Add missing descriptions to help output
  h264_ps: declare array of colorspace strings on its own line.
  fate: amix: specify f32 sample format for comparison
  tiny_psnr: support 32-bit float samples
  eamad/eatgq/eatqi: call special EA IDCT directly
  eamad: remove use of MpegEncContext
  mpegvideo: remove unnecessary inclusions of faandct.h
  af_asyncts: avoid overflow in out_size with large delta values
  af_asyncts: add first_pts option

Conflicts:
	configure
	libavcodec/eamad.c
	libavcodec/h264_ps.c
	libavformat/crcenc.c
	libavformat/ffmdec.c
	libavformat/ffmenc.c
	libavformat/framecrcenc.c
	libavformat/md5enc.c
	libavformat/nutdec.c
	libavformat/rawenc.c
	libavformat/yuv4mpeg.c
	tests/tiny_psnr.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-30 23:28:31 +02:00
Ronald S. Bultje
be391fb6df h264_ps: declare array of colorspace strings on its own line. 2012-07-29 14:53:42 -07:00
Michael Niedermayer
1d890ec849 h264/decode_scaling_matrices: fix coding order of 8x8 matrixes.
This fixes decoding of PPCV444I3_Thomson_A.bits

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-10 02:17:53 +02:00
Michael Niedermayer
e162733dcd h264: 12 and 14 bit support
This works with I, P and B frames
CAVLC and CABAC

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-10 02:17:53 +02:00
Michael Niedermayer
acb52d3228 h264: print error on unsupported seperate color planes
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-05 19:18:10 +02:00
Michael Niedermayer
4330b8bba2 h264: sps: handle more profiles
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-03 00:40:26 +02:00
Michael Niedermayer
9091ba9dfa h264: add support for constraint_set4_flag / constraint_set5_flag
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-03 00:36:26 +02:00
Michael Niedermayer
0c9283a03f h264: check log2_max_poc_lsb for validity
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-07-01 20:49:31 +02:00
Michael Niedermayer
27744fe439 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  mpeg12: fixed parsing in some mpeg2 streams
  Add SMPTE240M transfer characteristics flag.
  mpegts: Some additional HDMV types and reg descriptors for mpegts
  motionpixels: Clip YUV values after applying a gradient.
  jpeg: handle progressive in second field of interlaced.
  ituh263dec: Implement enough of Annex O (scalability) to fix a FPE.
  h263: more strictly forbid frame size changes with frame-mt.
  h264: additional protection against unsupported size/bitdepth changes.
  tta: prevents overflows for 32bit integers in header.
  configure: remove malloc_aligned.
  vp8: update frame size changes on thread context switches.
  snowdsp: explicitily state instruction size.
  wmall: fix reconstructing audio with uncoded channels
  WMAL cosmetics: fix indentation
  gitignore: add Win32 library suffixes

Conflicts:
	configure
	libavcodec/h263dec.c
	libavcodec/h264.c
	libavcodec/ituh263dec.c
	libavcodec/mjpegdec.c
	libavcodec/wmalosslessdec.c
	libavcodec/x86/snowdsp_mmx.c
	libavformat/mpegts.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-05-03 21:02:32 +02:00
Ronald S. Bultje
732f9fcfe5 h264: additional protection against unsupported size/bitdepth changes.
Fixes crashes in codepaths not covered by original checks.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2012-05-02 10:19:37 -07:00
Michael Niedermayer
192efcf768 h264_ps: check log2_max_frame_num for validity.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-04-07 19:59:00 +02:00
Michael Niedermayer
9621646eb3 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  x86: dsputil: prettyprint gcc inline asm
  x86: K&R prettyprinting cosmetics for dsputil_mmx.c
  x86: conditionally compile H.264 QPEL optimizations
  dsputil_mmx: Surround QPEL macros by "do { } while (0);" blocks.
  Ignore generated files below doc/.
  dpcm: convert to bytestream2.
  interplayvideo: convert to bytestream2.
  movenc: Merge if statements
  h264: fix memleak in error path.
  pthread: Immediately release all frames in ff_thread_flush()
  h264: Add check for invalid chroma_format_idc
  utvideo: port header reading to bytestream2.

Conflicts:
	.gitignore
	configure
	libavcodec/h264_ps.c
	libavcodec/interplayvideo.c
	libavcodec/pthread.c
	libavcodec/x86/dsputil_mmx.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-25 23:10:30 +02:00
Alexander Strange
cb34867780 h264: fix memleak in error path.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
2012-03-24 15:34:33 -07:00
Alexander Strange
6ef4063957 h264: Add check for invalid chroma_format_idc
Fixes a crash when FF_DEBUG_PICT_INFO is used.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
2012-03-24 15:29:38 -07:00
Michael Niedermayer
6968a7d193 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  doc/general: update supported devices table.
  doc/general: add missing @tab to codecs table.
  h264: Fix invalid interlaced/progressive MB combinations for direct mode prediction.
  avconv: reindent
  avconv: link '-passlogfile' option to libx264 'stats' AVOption.
  libx264: add 'stats' private option for setting 2pass stats filename.
  libx264: fix help text for slice-max-size option.
  http: Clear the auth state on redirects
  http: Retry auth if it failed due to being stale
  rtsp: Resend new keepalive commands if they used stale auth
  rtsp: Retry authentication if failed due to being stale
  httpauth: Parse the stale field in digest auth
  dxva2_vc1: pass the overlap flag to the decoder
  dxva2_vc1: fix decoding of BI frames
  FATE: add shorthand to wavpack test
  dfa: convert to bytestream2 API
  anm decoder: move buffer allocation from decode_init() to decode_frame()
  h264: improve parsing of broken AVC SPS

Conflicts:
	ffmpeg.c
	libavcodec/anm.c
	libavcodec/dfa.c
	libavcodec/h264.c
	libavcodec/h264_direct.c
	libavcodec/h264_ps.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-14 02:10:11 +01:00