Commit Graph

99 Commits

Author SHA1 Message Date
Michael Niedermayer
8447658550 Merge commit 'a1c2b48018b09d2613f075ec0748c95bd520ac00'
* commit 'a1c2b48018b09d2613f075ec0748c95bd520ac00':
  hevc: templatize intra_pred

Conflicts:
	libavcodec/hevc.h

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-05-19 14:10:13 +02:00
Anton Khirnov
a1c2b48018 hevc: templatize intra_pred 2014-05-19 07:10:03 +02:00
Mickaël Raulet
e2760de605 hevc: fix key_frame as an IRAP
it was always 1 before with ISOBMFF(cherry picked from commit fb1402b1ec78d80acd6ced76bf78d65560965c4c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-05-17 01:14:51 +02:00
Michael Niedermayer
2201d1a0f8 avcodec/hevc: Fix undefined shifts
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-05-09 13:34:12 +02:00
Michael Niedermayer
84655bdaed avcodec/hevc: remove unused variable
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-05-07 02:07:12 +02:00
Mickaël Raulet
83976e40e8 hevc: C code update for new motion compensation
pretty print C

Reviewed-by: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-05-06 18:22:34 +02:00
gcocherel
2eddf3a6ef avcodec/hevc: fix no output of prior pics and pic output flags(cherry picked from commit e99b96dff1d76d74cb5633aa9702828d863050e2)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-04-29 22:43:25 +02:00
Michael Niedermayer
6795dcfa65 avcodec/hevc: Export picture type
This only uses the first slice, improvement here is welcome
analyzing all slices the trivial way would interfere with threads

Fixes Ticket3185

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-26 04:20:52 +01:00
Michael Niedermayer
06219db935 Merge commit 'ed06e5d92b4c67b49068d538461fbbe0a53a8c5e'
* commit 'ed06e5d92b4c67b49068d538461fbbe0a53a8c5e':
  hevc: Do not turn 32bit timebases into negative numbers

Conflicts:
	libavcodec/hevc.c

See: bf2ce19e51
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-09 21:16:33 +01:00
Michael Niedermayer
ed06e5d92b hevc: Do not turn 32bit timebases into negative numbers
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
2014-03-09 17:22:53 +01:00
Michael Niedermayer
6cc94e9719 hevc: use av_mallocz() for allocating tab_ipm
Fixes use of uninitialized memory and out of stack array read.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
2014-03-09 17:08:11 +01:00
Michael Niedermayer
ca9f7e1832 Merge remote-tracking branch 'qatar/master'
* qatar/master:
  hevc: Always consider VLC NALU type mismatch fatal

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-19 22:34:41 +01:00
Luca Barbato
521726ff57 hevc: Always consider VLC NALU type mismatch fatal
Sample-Id: 00001667-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-02-19 16:59:40 +01:00
Michael Niedermayer
bf2ce19e51 avcodec/hevc: Dont turn 32bit timebases into negative numbers
Fixes assertion failure
Fixes: 499f6ecaeae8e3f79a115a72c61ffd15-asan_static-oob_124a175_2792_cov_1782273879_RPLM_A_qualcomm_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-15 23:17:53 +01:00
Michael Niedermayer
3b6655ebff avcodec/hevc: remove FFUMOD() in slice qp_y init
This has become redundant

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-08 12:27:00 +01:00
Michael Niedermayer
1e263133cc avcodec/hevc: remove unused variables
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-08 12:27:00 +01:00
Michael Niedermayer
64278039e5 avcodec/hevc: Simplify get_qPy_pred()
Fixes use of uninitialized memory
Fixes: 93728afd9aa074ba14a09bfd93a632fd-asan_static-oob_124a17d_1445_cov_1021181966_DBLK_D_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-08 00:22:49 +01:00
Michael Niedermayer
1a3ed056c5 avcodec/hevc: make check for previous slice segment tighter
This ensures the previous one is matching the curent and not just any

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-08 00:22:49 +01:00
Michael Niedermayer
6ef57f4d9a avcodec/hevc: hls_decode_entry: check that the previous slice segment is available before decoding the next
Fixes use of uninitialized memory
Fixes out of array read
Fixes assertion failure
Fixes part of cb307d24befbd109c6f054008d6777b5/asan_static-oob_124a175_1445_cov_2355279992_DBLK_D_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-07 04:40:37 +01:00
Michael Niedermayer
a18f111582 avcodec/hevc: clear tab_slice_address of ctb on error.
This allows us to detect which areas have failed to decode

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-07 04:30:31 +01:00
Michael Niedermayer
56985d26d7 avcodec/hevc: clear tab_slice_address in hevc_frame_start()
Fixes inconsistencies
Fixes use of uninitilaized memory
Fixes part of  cb307d24befbd109c6f054008d6777b5/asan_static-oob_124a175_1445_cov_2355279992_DBLK_D_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-07 03:47:49 +01:00
Michael Niedermayer
a0d5204cd9 Merge commit 'b25e84b7399bd91605596b67d761d3464dbe8a6e'
* commit 'b25e84b7399bd91605596b67d761d3464dbe8a6e':
  hevc: check that the VCL NAL types are the same for all slice segments of a frame

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-04 13:44:39 +01:00
Anton Khirnov
b25e84b739 hevc: check that the VCL NAL types are the same for all slice segments of a frame
Fixes possible invalid memory access for mismatching skipped/non-skipped
slice segments.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Sample-Id: 00001533-google
2014-02-04 10:39:07 +01:00
Michael Niedermayer
d9bb7fc3cb Merge commit '816e5b997028c8215c804b1e58b2388592ed612b'
* commit '816e5b997028c8215c804b1e58b2388592ed612b':
  hevc: Reject impossible slice segment

Conflicts:
	libavcodec/hevc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-02 05:40:30 +01:00
Michael Niedermayer
96c4ba2392 avcodec/hevc: propagate error code from hls_coding_quadtree()
Fixes use of uninitialized memory
Fixes out of array read
Fixes: asan_static-oob_123cee5_2630_cov_1869071233_PICSIZE_A_Bossen_1.bin
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-02 02:35:11 +01:00
Luca Barbato
816e5b9970 hevc: Reject impossible slice segment
A dependent slice cannot have address 0.
Prevent an out of array bound load in ff_hevc_cabac_init().

Sample-Id: 00001406-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-02-01 17:04:49 +01:00
Michael Niedermayer
a69dd1163b Merge commit 'e22ebd04bcab7f86548794556c28ecca46d9c2ac'
* commit 'e22ebd04bcab7f86548794556c28ecca46d9c2ac':
  hevc: Bound check cu_qp_delta

Conflicts:
	libavcodec/hevc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 15:42:51 +01:00
Michael Niedermayer
58f437c3f6 Merge commit '0d999333f96a34903448579bf13a3209deaee9da'
* commit '0d999333f96a34903448579bf13a3209deaee9da':
  hevc: Bound check slice_qp

Conflicts:
	libavcodec/hevc.c

See: aead772b58
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 15:38:29 +01:00
Michael Niedermayer
5b93b2722d Merge commit '838740e6420538ad45982da6b1d3aa3ae91307f5'
* commit '838740e6420538ad45982da6b1d3aa3ae91307f5':
  hevc: Prevent some integer overflows

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 15:31:04 +01:00
Luca Barbato
e22ebd04bc hevc: Bound check cu_qp_delta
The T-REC-H.265-2013044 page 91 states it has to be in the range
[-(26 + s->sps->qp_bd_offset / 2), (25 + s->sps->qp_bd_offset / 2)].
2014-01-21 11:59:54 +01:00
Luca Barbato
0d999333f9 hevc: Bound check slice_qp
The T-REC-H.265-2013044 page 79 states it has to be in the range
[-s->sps->qp_bd_offset, 51].

Sample-Id: 00001386-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-01-21 11:59:30 +01:00
Luca Barbato
838740e642 hevc: Prevent some integer overflows
get_ue_golomb_long() returns an unsigned.

Sample-Id: 00001541-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-01-21 11:59:18 +01:00
Michael Niedermayer
d5c15ebeaf hevc: Fix modulo operations
Keep qp fields within the range.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-01-21 11:57:43 +01:00
Michael Niedermayer
22bfb4be28 avcodec/hevc: Check entry point arrays for malloc failure
Fixes null pointer dereference
Fixes: signal_sigsegv_e1d3b6_2192_DBLK_F_VIXS_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 05:04:13 +01:00
Luca Barbato
aead772b58 hevc: Bound check slice_qp
The T-REC-H.265-2013044 page 79 states they have to be into the range
[-s->sps->qp_bd_offset, 51].

Fixes: asan_stack-oob_eae8e3_9522_WP_MAIN10_B_Toshiba_3.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 05:04:13 +01:00
Luca Barbato
48a5b15543 hevc: Reject impossible dependent tile
The tile 0 cannot depend on a previous one.
Prevent an out of array bound load in ff_hevc_cabac_init().

Fixes: asan_heap-oob_e3a924_1630_DBLK_A_MAIN10_VIXS_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

Reviewed-by: Guillaume Martres <smarter@ubuntu.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-13 02:51:53 +01:00
Michael Niedermayer
0999f1613b avcodec/hevc: use av_mallocz() for allocating tab_ipm
Fixes use of uninitialized memory and out of stack array read
Fixes: signal_sigsegv_ecc526_7846_WPP_C_ericsson_MAIN_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-11 20:30:22 +01:00
Michael Niedermayer
c1362ca047 Merge commit 'e588615d938f8581f0d6f3771662d08cadfc00de'
* commit 'e588615d938f8581f0d6f3771662d08cadfc00de':
  hevc: fix decoding of one PU wide files

Conflicts:
	libavcodec/hevc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-09 15:30:50 +01:00
Guillaume Martres
e588615d93 hevc: fix decoding of one PU wide files
For those the block size may be larger than the source linesize (if the
edges are not allocated).

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-01-09 09:46:13 +01:00
Michael Niedermayer
4ced5d7780 avcodec/hevc: Fix modulo operations
Fixes qp fields becoming out of range
Fixes: asan_static-oob_e393a3_6998_WPP_A_ericsson_MAIN10_2.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 04:32:37 +01:00
Michael Niedermayer
21a2fb7e05 avcodec/hevc: clear HEVClc when its deallocated in hevc_decode_free()
Fixes reading freed memory
Fixes: asan_heap-uaf_1abf8ef_3987_NUT_A_ericsson_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-31 15:19:59 +01:00
Michael Niedermayer
d6b7ea06f1 Merge commit '2a41826bea3833895dc06939831b7f35ca1f597e'
* commit '2a41826bea3833895dc06939831b7f35ca1f597e':
  lavc: add HEVC profiles names

Conflicts:
	libavcodec/hevc.c
	libavcodec/version.h

See: fb7d70c1cd
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-21 00:24:43 +01:00
Mickaël Raulet
5c3fa74b94 hevc: support luma bit depth != chroma bit depth for PCM coding units
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-20 19:39:53 +01:00
Gildas Cocherel
321cb8b048 hevc: store profile and level in AVCodecContext
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-20 19:36:24 +01:00
Gildas Cocherel
2a41826bea lavc: add HEVC profiles names
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-20 19:35:27 +01:00
Guillaume Martres
ecb21d2437 hevc: rename ptl structs and variables
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-14 03:01:56 +01:00
Guillaume Martres
17a10d51b8 hevc: set time_base when possible
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-11 20:39:54 +01:00
Gildas Cocherel
33452aede6 hevc: store the VPS list as an AVBufferRef, just like the others *PS
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-11 20:39:54 +01:00
Michael Niedermayer
b769cf4b44 hevc: do not dereference pointer before NULL check in verify_md5()
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-12-11 20:39:54 +01:00
Guillaume Martres
f90281ca97 hevc: Correctly set time_base
- Try reading the time_base information from the VPS too, not just the VUI
- Only set time_base when an SPS is activated, not when it's decoded.
- Reduce the fraction before setting it.
- Don't set anything if the fraction is invalid (because the VUI is not
  present or because the encoded value is invalid).

Conflicts:
	libavcodec/hevc_ps.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-10 16:20:53 +01:00