Commit Graph

88 Commits

Author SHA1 Message Date
Michael Niedermayer
ca9f7e1832 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  hevc: Always consider VLC NALU type mismatch fatal

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-19 22:34:41 +01:00
Luca Barbato
521726ff57 hevc: Always consider VLC NALU type mismatch fatal
Sample-Id: 00001667-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-02-19 16:59:40 +01:00
Michael Niedermayer
bf2ce19e51 avcodec/hevc: Dont turn 32bit timebases into negative numbers
Fixes assertion failure
Fixes: 499f6ecaeae8e3f79a115a72c61ffd15-asan_static-oob_124a175_2792_cov_1782273879_RPLM_A_qualcomm_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-15 23:17:53 +01:00
Michael Niedermayer
3b6655ebff avcodec/hevc: remove FFUMOD() in slice qp_y init
This has become redundant

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-08 12:27:00 +01:00
Michael Niedermayer
1e263133cc avcodec/hevc: remove unused variables
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-08 12:27:00 +01:00
Michael Niedermayer
64278039e5 avcodec/hevc: Simplify get_qPy_pred()
Fixes use of uninitialized memory
Fixes: 93728afd9aa074ba14a09bfd93a632fd-asan_static-oob_124a17d_1445_cov_1021181966_DBLK_D_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-08 00:22:49 +01:00
Michael Niedermayer
1a3ed056c5 avcodec/hevc: make check for previous slice segment tighter
This ensures the previous one is matching the curent and not just any

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-08 00:22:49 +01:00
Michael Niedermayer
6ef57f4d9a avcodec/hevc: hls_decode_entry: check that the previous slice segment is available before decoding the next
Fixes use of uninitialized memory
Fixes out of array read
Fixes assertion failure
Fixes part of cb307d24befbd109c6f054008d6777b5/asan_static-oob_124a175_1445_cov_2355279992_DBLK_D_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-07 04:40:37 +01:00
Michael Niedermayer
a18f111582 avcodec/hevc: clear tab_slice_address of ctb on error.
This allows us to detect which areas have failed to decode

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-07 04:30:31 +01:00
Michael Niedermayer
56985d26d7 avcodec/hevc: clear tab_slice_address in hevc_frame_start()
Fixes inconsistencies
Fixes use of uninitilaized memory
Fixes part of  cb307d24befbd109c6f054008d6777b5/asan_static-oob_124a175_1445_cov_2355279992_DBLK_D_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-07 03:47:49 +01:00
Michael Niedermayer
a0d5204cd9 Merge commit 'b25e84b7399bd91605596b67d761d3464dbe8a6e'
* commit 'b25e84b7399bd91605596b67d761d3464dbe8a6e':
  hevc: check that the VCL NAL types are the same for all slice segments of a frame

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 13:44:39 +01:00
Anton Khirnov
b25e84b739 hevc: check that the VCL NAL types are the same for all slice segments of a frame
Fixes possible invalid memory access for mismatching skipped/non-skipped
slice segments.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Sample-Id: 00001533-google
2014-02-04 10:39:07 +01:00
Michael Niedermayer
d9bb7fc3cb Merge commit '816e5b997028c8215c804b1e58b2388592ed612b'
* commit '816e5b997028c8215c804b1e58b2388592ed612b':
  hevc: Reject impossible slice segment

Conflicts:
	libavcodec/hevc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-02 05:40:30 +01:00
Michael Niedermayer
96c4ba2392 avcodec/hevc: propagate error code from hls_coding_quadtree()
Fixes use of uninitialized memory
Fixes out of array read
Fixes: asan_static-oob_123cee5_2630_cov_1869071233_PICSIZE_A_Bossen_1.bin
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-02 02:35:11 +01:00
Luca Barbato
816e5b9970 hevc: Reject impossible slice segment
A dependent slice cannot have address 0.
Prevent an out of array bound load in ff_hevc_cabac_init().

Sample-Id: 00001406-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-02-01 17:04:49 +01:00
Michael Niedermayer
a69dd1163b Merge commit 'e22ebd04bcab7f86548794556c28ecca46d9c2ac'
* commit 'e22ebd04bcab7f86548794556c28ecca46d9c2ac':
  hevc: Bound check cu_qp_delta

Conflicts:
	libavcodec/hevc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 15:42:51 +01:00
Michael Niedermayer
58f437c3f6 Merge commit '0d999333f96a34903448579bf13a3209deaee9da'
* commit '0d999333f96a34903448579bf13a3209deaee9da':
  hevc: Bound check slice_qp

Conflicts:
	libavcodec/hevc.c

See: aead772b58
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 15:38:29 +01:00
Michael Niedermayer
5b93b2722d Merge commit '838740e6420538ad45982da6b1d3aa3ae91307f5'
* commit '838740e6420538ad45982da6b1d3aa3ae91307f5':
  hevc: Prevent some integer overflows

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 15:31:04 +01:00
Luca Barbato
e22ebd04bc hevc: Bound check cu_qp_delta
The T-REC-H.265-2013044 page 91 states it has to be in the range
[-(26 + s->sps->qp_bd_offset / 2), (25 + s->sps->qp_bd_offset / 2)].
2014-01-21 11:59:54 +01:00
Luca Barbato
0d999333f9 hevc: Bound check slice_qp
The T-REC-H.265-2013044 page 79 states it has to be in the range
[-s->sps->qp_bd_offset, 51].

Sample-Id: 00001386-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-01-21 11:59:30 +01:00
Luca Barbato
838740e642 hevc: Prevent some integer overflows
get_ue_golomb_long() returns an unsigned.

Sample-Id: 00001541-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-01-21 11:59:18 +01:00
Michael Niedermayer
d5c15ebeaf hevc: Fix modulo operations
Keep qp fields within the range.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-01-21 11:57:43 +01:00
Michael Niedermayer
22bfb4be28 avcodec/hevc: Check entry point arrays for malloc failure
Fixes null pointer dereference
Fixes: signal_sigsegv_e1d3b6_2192_DBLK_F_VIXS_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 05:04:13 +01:00
Luca Barbato
aead772b58 hevc: Bound check slice_qp
The T-REC-H.265-2013044 page 79 states they have to be into the range
[-s->sps->qp_bd_offset, 51].

Fixes: asan_stack-oob_eae8e3_9522_WP_MAIN10_B_Toshiba_3.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 05:04:13 +01:00
Luca Barbato
48a5b15543 hevc: Reject impossible dependent tile
The tile 0 cannot depend on a previous one.
Prevent an out of array bound load in ff_hevc_cabac_init().

Fixes: asan_heap-oob_e3a924_1630_DBLK_A_MAIN10_VIXS_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

Reviewed-by: Guillaume Martres <smarter@ubuntu.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 02:51:53 +01:00
Michael Niedermayer
0999f1613b avcodec/hevc: use av_mallocz() for allocating tab_ipm
Fixes use of uninitialized memory and out of stack array read
Fixes: signal_sigsegv_ecc526_7846_WPP_C_ericsson_MAIN_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-11 20:30:22 +01:00
Michael Niedermayer
c1362ca047 Merge commit 'e588615d938f8581f0d6f3771662d08cadfc00de'
* commit 'e588615d938f8581f0d6f3771662d08cadfc00de':
  hevc: fix decoding of one PU wide files

Conflicts:
	libavcodec/hevc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-09 15:30:50 +01:00
Guillaume Martres
e588615d93 hevc: fix decoding of one PU wide files
For those the block size may be larger than the source linesize (if the
edges are not allocated).

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-01-09 09:46:13 +01:00
Michael Niedermayer
4ced5d7780 avcodec/hevc: Fix modulo operations
Fixes qp fields becoming out of range
Fixes: asan_static-oob_e393a3_6998_WPP_A_ericsson_MAIN10_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 04:32:37 +01:00
Michael Niedermayer
21a2fb7e05 avcodec/hevc: clear HEVClc when its deallocated in hevc_decode_free()
Fixes reading freed memory
Fixes: asan_heap-uaf_1abf8ef_3987_NUT_A_ericsson_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-31 15:19:59 +01:00
Michael Niedermayer
d6b7ea06f1 Merge commit '2a41826bea3833895dc06939831b7f35ca1f597e'
* commit '2a41826bea3833895dc06939831b7f35ca1f597e':
  lavc: add HEVC profiles names

Conflicts:
	libavcodec/hevc.c
	libavcodec/version.h

See: fb7d70c1cd
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-21 00:24:43 +01:00
Mickaël Raulet
5c3fa74b94 hevc: support luma bit depth != chroma bit depth for PCM coding units
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-20 19:39:53 +01:00
Gildas Cocherel
321cb8b048 hevc: store profile and level in AVCodecContext
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-20 19:36:24 +01:00
Gildas Cocherel
2a41826bea lavc: add HEVC profiles names
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-20 19:35:27 +01:00
Guillaume Martres
ecb21d2437 hevc: rename ptl structs and variables
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-14 03:01:56 +01:00
Guillaume Martres
17a10d51b8 hevc: set time_base when possible
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-11 20:39:54 +01:00
Gildas Cocherel
33452aede6 hevc: store the VPS list as an AVBufferRef, just like the others *PS
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-11 20:39:54 +01:00
Michael Niedermayer
b769cf4b44 hevc: do not dereference pointer before NULL check in verify_md5()
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-11 20:39:54 +01:00
Guillaume Martres
f90281ca97 hevc: Correctly set time_base
- Try reading the time_base information from the VPS too, not just the VUI
- Only set time_base when an SPS is activated, not when it's decoded.
- Reduce the fraction before setting it.
- Don't set anything if the fraction is invalid (because the VUI is not
  present or because the encoded value is invalid).

Conflicts:
	libavcodec/hevc_ps.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-10 16:20:53 +01:00
Guillaume Martres
679a6377e4 hevc: avoid some unnecessary differences with libav
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-10 16:20:48 +01:00
Michael Niedermayer
31a52a60c9 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  hevc: parse frame packing arrangement SEI messages and save relevant stereo3d information

Conflicts:
	libavcodec/hevc.h
	libavcodec/hevc_sei.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-09 22:25:38 +01:00
Vittorio Giovara
acb77dff6a hevc: parse frame packing arrangement SEI messages and save relevant stereo3d information 2013-12-09 16:02:43 +01:00
Michael Niedermayer
f1783c05f1 Revert "avcodec/hevc: calculate checksum only if AV_EF_EXPLODE is set"
This reverts commit 758b6d39f6.

Requested-by: smarter
"now that http://git.videolan.org/?p=ffmpeg.git;a=commit;h=97de206b44a48da726807cc3e7b9448a8112760b
 has been merged, http://git.videolan.org/?p=ffmpeg.git;a=commit;h=758b6d39f685a510f48ff9e4c05fffa859d23c42
 could be reverted, this would reduce the delta with libav
 (and it makes sense)"
2013-12-04 11:46:31 +01:00
gcocherel
fb7d70c1cd hevc : add hevc profile
(cherry picked from commit 0c8dfd8ad6400b33da55a9f4d9aa8d3f5c9d209f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-22 01:29:02 +01:00
gcocherel
36658c978f hevc : update hevc_ps.c
(cherry picked from commit 088f2eb1ae42bffc63c2cee4e7eba8f47056043b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-22 01:29:02 +01:00
gcocherel
81d0252dac hevc : update hls_decode_neighbour(cherry picked from commit 14341c6d9a6885657ba365807223e96a67d6c0ed)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-21 22:51:51 +01:00
gcocherel
0afa254d4e hevc : fix pcm(cherry picked from commit 12bbfdba1899d11044d2bd8457159fc8b4b52a76)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-21 22:51:51 +01:00
Michael Niedermayer
91e00c4a78 Merge commit '458446acfa1441d283dacf9e6e545beb083b8bb0'
* commit '458446acfa1441d283dacf9e6e545beb083b8bb0':
  lavc: Edge emulation with dst/src linesize

Conflicts:
	libavcodec/cavs.c
	libavcodec/h264.c
	libavcodec/hevc.c
	libavcodec/mpegvideo_enc.c
	libavcodec/mpegvideo_motion.c
	libavcodec/rv34.c
	libavcodec/svq3.c
	libavcodec/vc1dec.c
	libavcodec/videodsp.h
	libavcodec/videodsp_template.c
	libavcodec/vp3.c
	libavcodec/vp8.c
	libavcodec/wmv2.c
	libavcodec/x86/videodsp.asm
	libavcodec/x86/videodsp_init.c

Changes to the asm are not merged, they are left for volunteers or
in their absence for later.
The changes this merge introduces are reordering of the function
arguments

See: face578d56
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-15 15:07:10 +01:00
Ronald S. Bultje
458446acfa lavc: Edge emulation with dst/src linesize
Allow supporting files for which the image stride is smaller than
the maximum block size + number of subpel mc taps, e.g. a 64x64 VP9
file or a 16x16 VP8 file with -fflags +emu_edge.
2013-11-15 10:16:27 +01:00
Michael Niedermayer
b906d04881 Merge commit '885ec9242554ad25922258a595ec5e317922a412'
* commit '885ec9242554ad25922258a595ec5e317922a412':
  hevc: Use parsed VUI colorimetry in avcodec

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-04 21:37:34 +01:00