Commit Graph

229 Commits

Author SHA1 Message Date
Hendrik Leppkes
c6f1f334cb Merge commit '792b9c9dfcf44b657d7854368d975b5ca3bc22ca'
* commit '792b9c9dfcf44b657d7854368d975b5ca3bc22ca':
  h264: set frame_num in start_frame(), not decode_slice_header()

Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
2015-12-17 12:59:25 +01:00
Hendrik Leppkes
bc66451e5e Merge commit '741b494fa8cd28a7d096349bac183893c236e3f9'
* commit '741b494fa8cd28a7d096349bac183893c236e3f9':
  h264: eliminate default_ref_list

Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
2015-12-17 12:45:28 +01:00
Anton Khirnov
792b9c9dfc h264: set frame_num in start_frame(), not decode_slice_header()
That is a more appropriate place for it, since it is not allowed to
change between slices.
2015-12-06 09:43:45 +01:00
Anton Khirnov
741b494fa8 h264: eliminate default_ref_list
According to the spec, the reference list for a slice should be
constructed by first generating an initial (what we now call "default")
reference list and then optionally applying modifications to it.

Our code has an optimization where the initial reference list is
constructed for the first inter slice and then rebuilt for other slices
if needed. This, however, adds complexity to the code, requires an extra
2.5kB array in the codec context and there is no reason to think that it
has any positive effect on performance. Therefore, simplify the code by
generating the reference list from scratch for each slice.
2015-12-06 09:42:39 +01:00
Michael Niedermayer
8000d484b8 avcodec/cabac: Check initial cabac decoder state
Fixes integer overflows
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 13:52:47 +01:00
Michael Niedermayer
4ea4d2f438 avcodec/h264_slice: Limit max_contexts when slice_context_count is initialized
Fixes out of array access
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2049_f2192b6829ab6e0eefcb035329c03c60.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-24 22:17:36 +01:00
Michael Niedermayer
6f37226b68 avcodec/h264_slice: Clear top_borders on allocation
In case of bitstream errors the deblock filter and slices can access uninitialized
top_borders from previous slices which did not fill them as they stoped halfway due
to error or where entirely missing.

This also makes code using these tables deterministic in case of missing or damaged
slices

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-18 22:18:01 +01:00
Michael Niedermayer
507b01f899 avcodec/h264_slice: Disable slice threads if there are multiple access units in a packet
Fixes null pointer dereference
Fixes Ticket4977

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-10-31 22:56:48 +01:00
Andreas Cadhalpun
a4680e7170 avcodec: rename avpriv_color_frame to ff_color_frame
It is only used inside libavcodec.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-10-22 00:38:50 +02:00
wm4
17fe18d21a Revert "avcodec/h264: remove redundant and bogus get_format call"
This reverts commit be583c6fd3.

This was not approved, and was accidentally pushed. I'm very sorry.
2015-10-15 20:19:55 +02:00
wm4
be583c6fd3 avcodec/h264: remove redundant and bogus get_format call
The AVCodecContext.get_format callback is not only used for pixel format
negotiation with the API user, but also for hwaccel init. For the
latter, it's required that some codec parameters, in particular the
codec profile, are set when the callback is invoked.

This patch removes a get_format invocation where this is not guaranteed.
The codec parameters, including the profile, are really set further
below. (The same code path that sets the profile also calls get_format
properly too.)

This just happened to work by coincidence in most cases. For example, if
the API user just copied or reused the AVStream's AVCodecContext when
decoding, the profile would be set properly. But in some cases it
fails., such as with the sample WolfensteinTwitch.mp4 on the samples
server.

Remove the redundant get_format call. Apparently it serves no purpose
anymore, although it is possible that this was different at the time it
was added in commit ffd77f94a2.

This fixes hwaccel usage for API users which do not set the profile
when setting up the AVCodecContext (which is allowed).
2015-10-15 20:16:13 +02:00
Michael Niedermayer
1376084dcb avcodec/h264_slice: replace assert by normal error check
Fixes assertion failure
Fixes: c6075771557e4f3b7b74e63d2d24fb01/signal_sigabrt_7ffff6ac8cc9_133_cov_2853689970_CREDITS.FST

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-09-02 01:22:45 +02:00
Gwenole Beauchesne
9f8e57efe4 vaapi: define a unique pixel format for VA-API (AV_PIX_FMT_VAAPI).
Deprecate older VA pixel formats (MOCO, IDCT) as it is now very unlikely
to ever be useful in the future. Only keep plain AV_PIX_FMT_VAAPI format
that is aliased to the older VLD variant.

This is an API change.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2015-08-19 14:03:24 +02:00
Ronald S. Bultje
030b5a4f77 lavc: put remaining bits of vdpau-in-decoder under FF_API_CAP_VDPAU. 2015-08-18 15:57:19 -04:00
wm4
94c0df79c7 lavc: propagate hwaccel errors
At least the new videotoolbox decoder does not actually set a frame if
end_frame fails. This causes the API to return success and signals that
a picture was decoded, even though AVFrame->data[0] is NULL.

Fix this by propagating end_frame errors.
2015-08-06 11:05:02 +02:00
Sebastien Zwickert
11d923d414 avcodec: add new Videotoolbox hwaccel. 2015-08-03 10:12:10 +02:00
Michael Niedermayer
c3fdf536f6 avcodec/h264_slice: Also check sei_recovery_frame_cnt for skip_frame nokey
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-28 13:27:16 +02:00
Michael Niedermayer
accc91642b avcodec/h264_slice: do not skip 2nd field if first was not skipped
Found-by: John Högberg <john.hogberg@ericsson.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-28 11:40:57 +02:00
Michael Niedermayer
444e9874a7 Merge commit 'def97856de6021965db86c25a732d78689bd6bb0'
* commit 'def97856de6021965db86c25a732d78689bd6bb0':
  lavc: AV-prefix all codec capabilities

Conflicts:
	cmdutils.c
	ffmpeg.c
	ffplay.c
	libavcodec/8svx.c
	libavcodec/aacenc.c
	libavcodec/ac3dec.c
	libavcodec/adpcm.c
	libavcodec/alac.c
	libavcodec/atrac3plusdec.c
	libavcodec/bink.c
	libavcodec/dnxhddec.c
	libavcodec/dvdec.c
	libavcodec/dvenc.c
	libavcodec/ffv1dec.c
	libavcodec/ffv1enc.c
	libavcodec/fic.c
	libavcodec/flacdec.c
	libavcodec/flacenc.c
	libavcodec/flvdec.c
	libavcodec/fraps.c
	libavcodec/frwu.c
	libavcodec/gifdec.c
	libavcodec/h261dec.c
	libavcodec/hevc.c
	libavcodec/iff.c
	libavcodec/imc.c
	libavcodec/libopenjpegdec.c
	libavcodec/libvo-aacenc.c
	libavcodec/libvorbisenc.c
	libavcodec/libvpxdec.c
	libavcodec/libvpxenc.c
	libavcodec/libx264.c
	libavcodec/mjpegbdec.c
	libavcodec/mjpegdec.c
	libavcodec/mpegaudiodec_float.c
	libavcodec/msmpeg4dec.c
	libavcodec/mxpegdec.c
	libavcodec/nvenc_h264.c
	libavcodec/nvenc_hevc.c
	libavcodec/pngdec.c
	libavcodec/qpeg.c
	libavcodec/ra288.c
	libavcodec/rv10.c
	libavcodec/s302m.c
	libavcodec/sp5xdec.c
	libavcodec/takdec.c
	libavcodec/tiff.c
	libavcodec/tta.c
	libavcodec/utils.c
	libavcodec/v210dec.c
	libavcodec/vp6.c
	libavcodec/vp9.c
	libavcodec/wavpack.c
	libavcodec/yop.c

Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-27 22:50:18 +02:00
Michael Niedermayer
94d68a41fa Merge commit '7c6eb0a1b7bf1aac7f033a7ec6d8cacc3b5c2615'
* commit '7c6eb0a1b7bf1aac7f033a7ec6d8cacc3b5c2615':
  lavc: AV-prefix all codec flags

Conflicts:
	doc/examples/muxing.c
	ffmpeg.c
	ffmpeg_opt.c
	ffplay.c
	libavcodec/aacdec.c
	libavcodec/aacenc.c
	libavcodec/ac3dec.c
	libavcodec/ac3enc_float.c
	libavcodec/atrac1.c
	libavcodec/atrac3.c
	libavcodec/atrac3plusdec.c
	libavcodec/dcadec.c
	libavcodec/ffv1enc.c
	libavcodec/h264.c
	libavcodec/h264_loopfilter.c
	libavcodec/h264_mb.c
	libavcodec/imc.c
	libavcodec/libmp3lame.c
	libavcodec/libtheoraenc.c
	libavcodec/libtwolame.c
	libavcodec/libvpxenc.c
	libavcodec/libxavs.c
	libavcodec/libxvid.c
	libavcodec/mpeg12dec.c
	libavcodec/mpeg12enc.c
	libavcodec/mpegaudiodec_template.c
	libavcodec/mpegvideo.c
	libavcodec/mpegvideo_enc.c
	libavcodec/mpegvideo_motion.c
	libavcodec/nellymoserdec.c
	libavcodec/nellymoserenc.c
	libavcodec/nvenc.c
	libavcodec/on2avc.c
	libavcodec/options_table.h
	libavcodec/opus_celt.c
	libavcodec/pngenc.c
	libavcodec/ra288.c
	libavcodec/ratecontrol.c
	libavcodec/twinvq.c
	libavcodec/vc1_block.c
	libavcodec/vc1_loopfilter.c
	libavcodec/vc1_mc.c
	libavcodec/vc1dec.c
	libavcodec/vorbisdec.c
	libavcodec/vp3.c
	libavcodec/wma.c
	libavcodec/wmaprodec.c
	libavcodec/x86/hpeldsp_init.c
	libavcodec/x86/me_cmp_init.c

Merged-by: Michael Niedermayer <michael@niedermayer.cc>
2015-07-27 22:10:35 +02:00
Vittorio Giovara
7c6eb0a1b7 lavc: AV-prefix all codec flags
Convert doxygen to multiline and express bitfields more simply.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2015-07-27 15:24:58 +01:00
Michael Niedermayer
3c63d06d81 avcodec/h264_slice: Fix container cropping
Fixes out of array read
Fixes: asan_heap-oob_394322e_138_cov_4265020547_CVPCMNL1_SVA_C.264

Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-07-08 17:50:05 +02:00
Michael Niedermayer
2e9dcb82e5 Merge commit '3e3056f2a020dd77efdf379dbd4c06a65b4a499a'
* commit '3e3056f2a020dd77efdf379dbd4c06a65b4a499a':
  h264: Allow stream and container cropping at the same time

Conflicts:
	libavcodec/h264_slice.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-30 21:11:49 +02:00
Vittorio Giovara
3e3056f2a0 h264: Allow stream and container cropping at the same time
The container cropping is applied only when difference is within 16
pixels, and the smallest value between the two is chosen.

Bug-Id: 383
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2015-06-30 15:34:37 +02:00
Michael Niedermayer
330863c9f1 avcodec/h264_slice: Use w/h from the AVFrame instead of mb_w/h
Fixes out of array access
Fixes: asan_heap-oob_4d5bb0_682_cov_3124593265_Fraunhofer__a_driving_force_in_innovation__small.mp4

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-30 19:38:44 +02:00
Michael Niedermayer
6323647c35 avcodec/h264_slice: Reformat IN_RANGE() uses to be readable
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-27 23:56:41 +02:00
Michael Niedermayer
29b0923324 avcodec/h264_slice: Silence pointer type warnings
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-27 23:56:06 +02:00
Michael Niedermayer
9ea256814d avcodec/h264_slice: assert that the first slice implies unfinished setup
If this assert fails there is very likely a bug in the code

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-27 23:42:33 +02:00
Michael Niedermayer
bd5c6a519e Merge commit '5ec0bdf2c524224f30ba4786f47324970aed4aaa'
* commit '5ec0bdf2c524224f30ba4786f47324970aed4aaa':
  h264: do not update the context fields copied between threads after finish_setup()

Conflicts:
	libavcodec/h264.h
	libavcodec/h264_slice.c

See: f111831ed6 and others
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-27 22:15:01 +02:00
Michael Niedermayer
f620315fa5 Merge commit 'e49e0f58e273237f83b4486f3536931ed1943d18'
* commit 'e49e0f58e273237f83b4486f3536931ed1943d18':
  h264: make sure the slices do not overlap during slice threading

Conflicts:
	libavcodec/h264.h
	libavcodec/h264_slice.c

See: 43b434210e
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-27 18:46:11 +02:00
Anton Khirnov
5ec0bdf2c5 h264: do not update the context fields copied between threads after finish_setup()
Should fix a large number of possible races with frame threading.
2015-06-27 16:48:48 +02:00
Anton Khirnov
e49e0f58e2 h264: make sure the slices do not overlap during slice threading
Based on a patch by Michael Niedermayer <michaelni@gmx.at>.
CC: libav-stable@libav.org
Found-by: Kieran Kunhya <kierank@obe.tv>
2015-06-27 16:43:22 +02:00
Andreas Cadhalpun
fdc64a1044 h264: er: Copy from the previous reference only if compatible
Also use the frame pixel format instead of the one from the codec
context, which is more robust.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-06-15 18:32:18 +02:00
Andreas Cadhalpun
a4fbd55d6e h264: er: Copy from the previous reference only if compatible
Also use the frame pixel format instead of the one from the codec
context, which is more robust.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2015-06-15 13:39:07 +02:00
Andreas Cadhalpun
1189af4292 h264: update avctx width/height/pix_fmt when returning frame
Inconsistencies between the dimensions/pixel format of avctx and the
frame can confuse API users.
For example this can crash the demuxing_decoding example.

Back up the previous values and restore them, when decoding the next
frame. This is necessary, because these can be different between the
returned frame and the last decoded frame.

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-06-12 21:22:00 +02:00
Michael Niedermayer
aef0e0f009 avcodec/h264_slice: Use AVFrame diemensions for grayscale handling
The AVFrame values are closer to the AVFrame bitmap changed instead of
the AVCodecContext values, so this should be more robust

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-10 11:37:48 +02:00
Michael Niedermayer
08d006f5a8 h264: Fix HWACCEL_MAX for D3D11
Signed-off-by: Martin Storsjö <martin@martin.st>
2015-06-01 11:36:30 +03:00
Michael Niedermayer
e9d646f44c Merge commit '0181ae9af2de1526464d23209b82e6674d362f5d'
* commit '0181ae9af2de1526464d23209b82e6674d362f5d':
  h264: Make sure reinit failures mark the context as not initialized

Conflicts:
	libavcodec/h264_slice.c

See: e8714f6f93
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-30 21:33:00 +02:00
Luca Barbato
0181ae9af2 h264: Make sure reinit failures mark the context as not initialized
Bug-Id: CVE-2015-3417
CC: libav-stable@libav.org
2015-05-30 16:28:32 +02:00
Michael Niedermayer
1b236541a6 avcodec/h264: Fix HWACCEL_MAX for D3D11
Found-by: philipl
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-26 05:23:29 +02:00
Michael Niedermayer
947b74ee7d Merge commit 'd8039ef8d221ea273aa4f1e62e5df21bf618c772'
* commit 'd8039ef8d221ea273aa4f1e62e5df21bf618c772':
  D3D11va: add a Direct3D11 video decoder similar to DXVA2

Conflicts:
	Changelog
	configure
	libavcodec/Makefile
	libavcodec/allcodecs.c
	libavcodec/dxva2_vc1.c
	libavcodec/version.h
	libavutil/pixdesc.c
	libavutil/pixfmt.h
	libavutil/version.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-25 22:24:34 +02:00
Steve Lhomme
d8039ef8d2 D3D11va: add a Direct3D11 video decoder similar to DXVA2
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2015-05-25 11:47:08 +02:00
Michael Niedermayer
47cbcf20d6 avcodec/h264_slice: Fix ranges in assert
Fixes CID1297592, CID1297593

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-05-15 00:30:32 +02:00
Michael Niedermayer
36f862e04c Merge commit 'a0f2946068c62e18cb05ac25c0df3d86077251a6'
* commit 'a0f2946068c62e18cb05ac25c0df3d86077251a6':
  h264: use properly allocated AVFrames

Conflicts:
	libavcodec/h264.c
	libavcodec/h264.h
	libavcodec/h264_refs.c
	libavcodec/h264_slice.c
	libavcodec/svq3.c
	libavcodec/vda_h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-29 16:35:24 +02:00
Michael Niedermayer
07a79cf869 Merge commit '7a4f74eed51f914e9bbfebaffd4a92ac6791f819'
* commit '7a4f74eed51f914e9bbfebaffd4a92ac6791f819':
  h264: embed the DPB in the context

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-29 15:33:46 +02:00
Michael Niedermayer
4a95cd5d9b avcodec/h264_slice: Also copy x264_build in ff_h264_update_thread_context()
Fixes fate-h264-lossless

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-29 15:21:28 +02:00
Michael Niedermayer
bd7dce594e Merge commit '88c612e3a4d5b584e2d9f6e2541de78d67bdfb9f'
* commit '88c612e3a4d5b584e2d9f6e2541de78d67bdfb9f':
  h264: merge the init and reinit paths in update_thread_context()

Conflicts:
	libavcodec/h264_slice.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-29 15:21:20 +02:00
Michael Niedermayer
cc87493f06 Merge commit '8a66fd40260b7aae6226d68c4dbad43b05a8e524'
* commit '8a66fd40260b7aae6226d68c4dbad43b05a8e524':
  h264: drop the reinit parameter from init_slice_header()

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-29 13:55:58 +02:00
Michael Niedermayer
51f7ba9cee Merge commit 'c9ccbc7333eddd025ebbde5cc4f27d68a950c623'
* commit 'c9ccbc7333eddd025ebbde5cc4f27d68a950c623':
  h264: update the current bit depth/chroma type in init_slice_header()

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-29 13:48:45 +02:00
Michael Niedermayer
16b90d0bcb Merge commit '29dcde2b87559b72957454a7e9bcd5227d8057ba'
* commit '29dcde2b87559b72957454a7e9bcd5227d8057ba':
  h264: copy nal_length_size in update_thread_context

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2015-04-29 13:41:24 +02:00