generic-library/ffmpeg
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

rtpenc: fix overflow checking in avc_mp4_find_startcode()

Browse Source 
The check `start + res < start' is broken since pointer overflow is
undefined behavior in C.  Many compilers such as gcc/clang optimize
away this check.

Use `res > end - start' instead.  Also change `res' to unsigned int
to avoid signed left-shift overflow.

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2f014567cf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This commit is contained in:
Xi Wang
2013-01-22 20:58:07 -05:00
committed by Michael Niedermayer
parent b63dbe2220
commit a31be9dd06
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
libavformat/rtpenc_h264.c
View File

@@ -31,14 +31,14 @@
static const uint8_t *avc_mp4_find_startcode(const uint8_t *start, const uint8_t *end, int nal_length_size) static const uint8_t *avc_mp4_find_startcode(const uint8_t *start, const uint8_t *end, int nal_length_size)
{ {
int res = 0; unsigned int res = 0;
if (end - start < nal_length_size) if (end - start < nal_length_size)
return NULL; return NULL;
while (nal_length_size--) while (nal_length_size--)
res = (res << 8) | *start++; res = (res << 8) | *start++;
if (start + res > end || res < 0 || start + res < start) if (res > end - start)
return NULL; return NULL;
return start + res; return start + res;