generic-library/vpx
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

vp9_quantize_ssse3_x86_64: fix out of bounds write

Browse Source 
eob is a pointer to a uint16_t. previously the code would store 64-bits
causing a crash or test failure with the right stack layout.

Change-Id: Ibd653baf323db114f2444951b9d8b00c596bf15a
This commit is contained in:
James Zern 2017-12-21 16:51:40 -05:00
parent 7a245adb18
commit 84a7263d4c
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vp9/encoder/x86/vp9_quantize_ssse3_x86_64.asm
View File

@ -169,7 +169,7 @@ cglobal quantize_%1, 0, %2, 15, coeff, ncoeff, skip, round, quant, \
pshuflw m7, m8, 0x1
pmaxsw m8, m7
pextrw r6, m8, 0
mov [r2], r6
mov [r2], r6w
RET
%endmacro