generic-library/vpx
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
51e770deb1
vpx/vp9/encoder/x86/vp9_sad_sse3.asm

961 lines
29 KiB
NASM
Raw Normal View History

Initial WebM release
2010-05-18 17:58:33 +02:00
;
Use WebM in copyright notice for consistency Changes 'The VP8 project' to 'The WebM project', for consistency with other webmproject.org repositories. Fixes issue #97. Change-Id: I37c13ed5fbdb9d334ceef71c6350e9febed9bbba
2010-09-09 14:16:39 +02:00
; Copyright (c) 2010 The WebM project authors. All Rights Reserved.
Initial WebM release
2010-05-18 17:58:33 +02:00
;
cosmetics: trim trailing whitespace When the license headers were updated, they accidentally contained trailing whitespace, so unfortunately we have to touch all the files again. Change-Id: I236c05fade06589e417179c0444cb39b09e4200d
2010-06-18 18:39:21 +02:00
; Use of this source code is governed by a BSD-style license
LICENSE: update with latest text Change-Id: Ieebea089095d9073b3a94932791099f614ce120c
2010-06-04 22:19:40 +02:00
; that can be found in the LICENSE file in the root of the source
; tree. An additional intellectual property rights grant can be found
cosmetics: trim trailing whitespace When the license headers were updated, they accidentally contained trailing whitespace, so unfortunately we have to touch all the files again. Change-Id: I236c05fade06589e417179c0444cb39b09e4200d
2010-06-18 18:39:21 +02:00
; in the file PATENTS. All contributing project authors may
LICENSE: update with latest text Change-Id: Ieebea089095d9073b3a94932791099f614ce120c
2010-06-04 22:19:40 +02:00
; be found in the AUTHORS file in the root of the source tree.
Initial WebM release
2010-05-18 17:58:33 +02:00
;
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
%include "vpx_ports/x86_abi_support.asm"
Remove stack shadowing for x86-64 x86-64 passes most arguments in registers. There is no need to push them to the stack before using them. Change-Id: I13c683f1358782682ecafaf1df3fb0af23b978ea
2010-10-15 02:10:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%macro STACK_FRAME_CREATE_X3 0
%if ABI_IS_32BIT
%define src_ptr rsi
%define src_stride rax
%define ref_ptr rdi
%define ref_stride rdx
%define end_ptr rcx
%define ret_var rbx
%define result_ptr arg(4)
%define max_err arg(4)
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
%define height dword ptr arg(4)
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
push rbp
mov rbp, rsp
push rsi
push rdi
push rbx
mov rsi, arg(0) ; src_ptr
mov rdi, arg(2) ; ref_ptr
movsxd rax, dword ptr arg(1) ; src_stride
movsxd rdx, dword ptr arg(3) ; ref_stride
%else
%ifidn __OUTPUT_FORMAT__,x64
modify SAVE_XMM for potential 64bit use the win64 abi requires saving and restoring xmm6:xmm15. currently SAVE_XMM and RESTORE XMM only allow for saving xmm6:xmm7. allow specifying the highest register used and if the stack is unaligned. Change-Id: Ica5699622ffe3346d3a486f48eef0206c51cf867
2011-04-15 16:05:20 +02:00
SAVE_XMM 7, u
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%define src_ptr rcx
%define src_stride rdx
%define ref_ptr r8
%define ref_stride r9
%define end_ptr r10
%define ret_var r11
modify SAVE_XMM for potential 64bit use the win64 abi requires saving and restoring xmm6:xmm15. currently SAVE_XMM and RESTORE XMM only allow for saving xmm6:xmm7. allow specifying the highest register used and if the stack is unaligned. Change-Id: Ica5699622ffe3346d3a486f48eef0206c51cf867
2011-04-15 16:05:20 +02:00
%define result_ptr [rsp+xmm_stack_space+8+4*8]
%define max_err [rsp+xmm_stack_space+8+4*8]
Specify size for argument pushed to stack The change fixes building error on Win64. Change-Id: I63d25b26220c4da8a98ca2e36530cbb802468e6b
2011-07-25 17:30:45 +02:00
%define height dword ptr [rsp+xmm_stack_space+8+4*8]
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%else
%define src_ptr rdi
%define src_stride rsi
%define ref_ptr rdx
%define ref_stride rcx
%define end_ptr r9
%define ret_var r10
%define result_ptr r8
%define max_err r8
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
%define height r8
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%endif
%endif
%endmacro
%macro STACK_FRAME_DESTROY_X3 0
%define src_ptr
%define src_stride
%define ref_ptr
%define ref_stride
%define end_ptr
%define ret_var
%define result_ptr
%define max_err
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
%define height
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%if ABI_IS_32BIT
pop rbx
pop rdi
pop rsi
pop rbp
%else
%ifidn __OUTPUT_FORMAT__,x64
Add save/restore xmm registers in x86 assembly code Went through the code and fixed it. Verified on Windows. Where possible, remove dependencies on xmm[67] Current code relies on pushing rbp to the stack to get 16 byte alignment. This broke when rbp wasn't pushed (vp8/encoder/x86/sad_sse3.asm). Work around this by using unaligned memory accesses. Revisit this and the offsets in vp8/encoder/x86/sad_sse3.asm in another change to SAVE_XMM. Change-Id: I5f940994d3ebfd977c3d68446cef20fd78b07877
2011-04-07 19:17:22 +02:00
RESTORE_XMM
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%endif
%endif
ret
%endmacro
%macro STACK_FRAME_CREATE_X4 0
%if ABI_IS_32BIT
%define src_ptr rsi
%define src_stride rax
%define r0_ptr rcx
%define r1_ptr rdx
%define r2_ptr rbx
%define r3_ptr rdi
%define ref_stride rbp
%define result_ptr arg(4)
push rbp
mov rbp, rsp
push rsi
push rdi
push rbx
push rbp
mov rdi, arg(2) ; ref_ptr_base
LOAD_X4_ADDRESSES rdi, rcx, rdx, rax, rdi
mov rsi, arg(0) ; src_ptr
movsxd rbx, dword ptr arg(1) ; src_stride
movsxd rbp, dword ptr arg(3) ; ref_stride
xchg rbx, rax
%else
%ifidn __OUTPUT_FORMAT__,x64
modify SAVE_XMM for potential 64bit use the win64 abi requires saving and restoring xmm6:xmm15. currently SAVE_XMM and RESTORE XMM only allow for saving xmm6:xmm7. allow specifying the highest register used and if the stack is unaligned. Change-Id: Ica5699622ffe3346d3a486f48eef0206c51cf867
2011-04-15 16:05:20 +02:00
SAVE_XMM 7, u
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%define src_ptr rcx
%define src_stride rdx
%define r0_ptr rsi
%define r1_ptr r10
%define r2_ptr r11
%define r3_ptr r8
%define ref_stride r9
modify SAVE_XMM for potential 64bit use the win64 abi requires saving and restoring xmm6:xmm15. currently SAVE_XMM and RESTORE XMM only allow for saving xmm6:xmm7. allow specifying the highest register used and if the stack is unaligned. Change-Id: Ica5699622ffe3346d3a486f48eef0206c51cf867
2011-04-15 16:05:20 +02:00
%define result_ptr [rsp+xmm_stack_space+16+4*8]
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
push rsi
LOAD_X4_ADDRESSES r8, r0_ptr, r1_ptr, r2_ptr, r3_ptr
%else
%define src_ptr rdi
%define src_stride rsi
%define r0_ptr r9
%define r1_ptr r10
%define r2_ptr r11
%define r3_ptr rdx
%define ref_stride rcx
%define result_ptr r8
LOAD_X4_ADDRESSES rdx, r0_ptr, r1_ptr, r2_ptr, r3_ptr
%endif
%endif
%endmacro
%macro STACK_FRAME_DESTROY_X4 0
%define src_ptr
%define src_stride
%define r0_ptr
%define r1_ptr
%define r2_ptr
%define r3_ptr
%define ref_stride
%define result_ptr
%if ABI_IS_32BIT
pop rbx
pop rdi
pop rsi
pop rbp
%else
%ifidn __OUTPUT_FORMAT__,x64
pop rsi
Add save/restore xmm registers in x86 assembly code Went through the code and fixed it. Verified on Windows. Where possible, remove dependencies on xmm[67] Current code relies on pushing rbp to the stack to get 16 byte alignment. This broke when rbp wasn't pushed (vp8/encoder/x86/sad_sse3.asm). Work around this by using unaligned memory accesses. Revisit this and the offsets in vp8/encoder/x86/sad_sse3.asm in another change to SAVE_XMM. Change-Id: I5f940994d3ebfd977c3d68446cef20fd78b07877
2011-04-07 19:17:22 +02:00
RESTORE_XMM
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%endif
%endif
ret
%endmacro
%macro PROCESS_16X2X3 5
%if %1==0
movdqa xmm0, XMMWORD PTR [%2]
lddqu xmm5, XMMWORD PTR [%3]
lddqu xmm6, XMMWORD PTR [%3+1]
lddqu xmm7, XMMWORD PTR [%3+2]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw xmm5, xmm0
psadbw xmm6, xmm0
psadbw xmm7, xmm0
%else
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movdqa xmm0, XMMWORD PTR [%2]
lddqu xmm1, XMMWORD PTR [%3]
lddqu xmm2, XMMWORD PTR [%3+1]
lddqu xmm3, XMMWORD PTR [%3+2]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw xmm1, xmm0
psadbw xmm2, xmm0
psadbw xmm3, xmm0
paddw xmm5, xmm1
paddw xmm6, xmm2
paddw xmm7, xmm3
%endif
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movdqa xmm0, XMMWORD PTR [%2+%4]
lddqu xmm1, XMMWORD PTR [%3+%5]
lddqu xmm2, XMMWORD PTR [%3+%5+1]
lddqu xmm3, XMMWORD PTR [%3+%5+2]
%if %1==0 || %1==1
lea %2, [%2+%4*2]
lea %3, [%3+%5*2]
%endif
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw xmm1, xmm0
psadbw xmm2, xmm0
psadbw xmm3, xmm0
paddw xmm5, xmm1
paddw xmm6, xmm2
paddw xmm7, xmm3
%endmacro
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%macro PROCESS_8X2X3 5
%if %1==0
movq mm0, QWORD PTR [%2]
movq mm5, QWORD PTR [%3]
movq mm6, QWORD PTR [%3+1]
movq mm7, QWORD PTR [%3+2]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw mm5, mm0
psadbw mm6, mm0
psadbw mm7, mm0
%else
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movq mm0, QWORD PTR [%2]
movq mm1, QWORD PTR [%3]
movq mm2, QWORD PTR [%3+1]
movq mm3, QWORD PTR [%3+2]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw mm1, mm0
psadbw mm2, mm0
psadbw mm3, mm0
paddw mm5, mm1
paddw mm6, mm2
paddw mm7, mm3
%endif
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movq mm0, QWORD PTR [%2+%4]
movq mm1, QWORD PTR [%3+%5]
movq mm2, QWORD PTR [%3+%5+1]
movq mm3, QWORD PTR [%3+%5+2]
%if %1==0 || %1==1
lea %2, [%2+%4*2]
lea %3, [%3+%5*2]
%endif
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw mm1, mm0
psadbw mm2, mm0
psadbw mm3, mm0
paddw mm5, mm1
paddw mm6, mm2
paddw mm7, mm3
%endmacro
%macro LOAD_X4_ADDRESSES 5
mov %2, [%1+REG_SZ_BYTES*0]
mov %3, [%1+REG_SZ_BYTES*1]
mov %4, [%1+REG_SZ_BYTES*2]
mov %5, [%1+REG_SZ_BYTES*3]
%endmacro
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%macro PROCESS_16X2X4 8
%if %1==0
movdqa xmm0, XMMWORD PTR [%2]
lddqu xmm4, XMMWORD PTR [%3]
lddqu xmm5, XMMWORD PTR [%4]
lddqu xmm6, XMMWORD PTR [%5]
lddqu xmm7, XMMWORD PTR [%6]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw xmm4, xmm0
psadbw xmm5, xmm0
psadbw xmm6, xmm0
psadbw xmm7, xmm0
%else
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movdqa xmm0, XMMWORD PTR [%2]
lddqu xmm1, XMMWORD PTR [%3]
lddqu xmm2, XMMWORD PTR [%4]
lddqu xmm3, XMMWORD PTR [%5]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw xmm1, xmm0
psadbw xmm2, xmm0
psadbw xmm3, xmm0
paddw xmm4, xmm1
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
lddqu xmm1, XMMWORD PTR [%6]
Initial WebM release
2010-05-18 17:58:33 +02:00
paddw xmm5, xmm2
paddw xmm6, xmm3
psadbw xmm1, xmm0
paddw xmm7, xmm1
%endif
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movdqa xmm0, XMMWORD PTR [%2+%7]
lddqu xmm1, XMMWORD PTR [%3+%8]
lddqu xmm2, XMMWORD PTR [%4+%8]
lddqu xmm3, XMMWORD PTR [%5+%8]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw xmm1, xmm0
psadbw xmm2, xmm0
psadbw xmm3, xmm0
paddw xmm4, xmm1
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
lddqu xmm1, XMMWORD PTR [%6+%8]
Initial WebM release
2010-05-18 17:58:33 +02:00
paddw xmm5, xmm2
paddw xmm6, xmm3
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%if %1==0 || %1==1
lea %2, [%2+%7*2]
lea %3, [%3+%8*2]
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
lea %4, [%4+%8*2]
lea %5, [%5+%8*2]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
lea %6, [%6+%8*2]
%endif
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw xmm1, xmm0
paddw xmm7, xmm1
%endmacro
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%macro PROCESS_8X2X4 8
%if %1==0
movq mm0, QWORD PTR [%2]
movq mm4, QWORD PTR [%3]
movq mm5, QWORD PTR [%4]
movq mm6, QWORD PTR [%5]
movq mm7, QWORD PTR [%6]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw mm4, mm0
psadbw mm5, mm0
psadbw mm6, mm0
psadbw mm7, mm0
%else
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movq mm0, QWORD PTR [%2]
movq mm1, QWORD PTR [%3]
movq mm2, QWORD PTR [%4]
movq mm3, QWORD PTR [%5]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw mm1, mm0
psadbw mm2, mm0
psadbw mm3, mm0
paddw mm4, mm1
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movq mm1, QWORD PTR [%6]
Initial WebM release
2010-05-18 17:58:33 +02:00
paddw mm5, mm2
paddw mm6, mm3
psadbw mm1, mm0
paddw mm7, mm1
%endif
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movq mm0, QWORD PTR [%2+%7]
movq mm1, QWORD PTR [%3+%8]
movq mm2, QWORD PTR [%4+%8]
movq mm3, QWORD PTR [%5+%8]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw mm1, mm0
psadbw mm2, mm0
psadbw mm3, mm0
paddw mm4, mm1
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movq mm1, QWORD PTR [%6+%8]
Initial WebM release
2010-05-18 17:58:33 +02:00
paddw mm5, mm2
paddw mm6, mm3
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%if %1==0 || %1==1
lea %2, [%2+%7*2]
lea %3, [%3+%8*2]
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
lea %4, [%4+%8*2]
lea %5, [%5+%8*2]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
lea %6, [%6+%8*2]
%endif
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw mm1, mm0
paddw mm7, mm1
%endmacro
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;void int vp9_sad16x16x3_sse3(
Initial WebM release
2010-05-18 17:58:33 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *ref_ptr,
; int ref_stride,
; int *results)
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_sad16x16x3_sse3)
sym(vp9_sad16x16x3_sse3):
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_CREATE_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
PROCESS_16X2X3 0, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_16X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_16X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_16X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_16X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_16X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_16X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_16X2X3 2, src_ptr, ref_ptr, src_stride, ref_stride
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
mov rcx, result_ptr
Initial WebM release
2010-05-18 17:58:33 +02:00
movq xmm0, xmm5
psrldq xmm5, 8
paddw xmm0, xmm5
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
;-
movq xmm0, xmm6
psrldq xmm6, 8
paddw xmm0, xmm6
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx+4], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
;-
movq xmm0, xmm7
psrldq xmm7, 8
paddw xmm0, xmm7
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx+8], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_DESTROY_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;void int vp9_sad16x8x3_sse3(
Initial WebM release
2010-05-18 17:58:33 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *ref_ptr,
; int ref_stride,
; int *results)
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_sad16x8x3_sse3)
sym(vp9_sad16x8x3_sse3):
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_CREATE_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
PROCESS_16X2X3 0, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_16X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_16X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_16X2X3 2, src_ptr, ref_ptr, src_stride, ref_stride
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
mov rcx, result_ptr
Initial WebM release
2010-05-18 17:58:33 +02:00
movq xmm0, xmm5
psrldq xmm5, 8
paddw xmm0, xmm5
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
;-
movq xmm0, xmm6
psrldq xmm6, 8
paddw xmm0, xmm6
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx+4], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
;-
movq xmm0, xmm7
psrldq xmm7, 8
paddw xmm0, xmm7
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx+8], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_DESTROY_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;void int vp9_sad8x16x3_sse3(
Initial WebM release
2010-05-18 17:58:33 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *ref_ptr,
; int ref_stride,
; int *results)
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_sad8x16x3_sse3)
sym(vp9_sad8x16x3_sse3):
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_CREATE_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
PROCESS_8X2X3 0, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_8X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_8X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_8X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_8X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_8X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_8X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_8X2X3 2, src_ptr, ref_ptr, src_stride, ref_stride
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
mov rcx, result_ptr
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
punpckldq mm5, mm6
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movq [rcx], mm5
movd [rcx+8], mm7
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_DESTROY_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;void int vp9_sad8x8x3_sse3(
Initial WebM release
2010-05-18 17:58:33 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *ref_ptr,
; int ref_stride,
; int *results)
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_sad8x8x3_sse3)
sym(vp9_sad8x8x3_sse3):
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_CREATE_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
PROCESS_8X2X3 0, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_8X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_8X2X3 1, src_ptr, ref_ptr, src_stride, ref_stride
PROCESS_8X2X3 2, src_ptr, ref_ptr, src_stride, ref_stride
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
mov rcx, result_ptr
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
punpckldq mm5, mm6
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movq [rcx], mm5
movd [rcx+8], mm7
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_DESTROY_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;void int vp9_sad4x4x3_sse3(
Initial WebM release
2010-05-18 17:58:33 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *ref_ptr,
; int ref_stride,
; int *results)
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_sad4x4x3_sse3)
sym(vp9_sad4x4x3_sse3):
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_CREATE_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm0, DWORD PTR [src_ptr]
movd mm1, DWORD PTR [ref_ptr]
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm2, DWORD PTR [src_ptr+src_stride]
movd mm3, DWORD PTR [ref_ptr+ref_stride]
Initial WebM release
2010-05-18 17:58:33 +02:00
punpcklbw mm0, mm2
punpcklbw mm1, mm3
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm4, DWORD PTR [ref_ptr+1]
movd mm5, DWORD PTR [ref_ptr+2]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm2, DWORD PTR [ref_ptr+ref_stride+1]
movd mm3, DWORD PTR [ref_ptr+ref_stride+2]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw mm1, mm0
punpcklbw mm4, mm2
punpcklbw mm5, mm3
psadbw mm4, mm0
psadbw mm5, mm0
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
lea src_ptr, [src_ptr+src_stride*2]
lea ref_ptr, [ref_ptr+ref_stride*2]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm0, DWORD PTR [src_ptr]
movd mm2, DWORD PTR [ref_ptr]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm3, DWORD PTR [src_ptr+src_stride]
movd mm6, DWORD PTR [ref_ptr+ref_stride]
Initial WebM release
2010-05-18 17:58:33 +02:00
punpcklbw mm0, mm3
punpcklbw mm2, mm6
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm3, DWORD PTR [ref_ptr+1]
movd mm7, DWORD PTR [ref_ptr+2]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw mm2, mm0
paddw mm1, mm2
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm2, DWORD PTR [ref_ptr+ref_stride+1]
movd mm6, DWORD PTR [ref_ptr+ref_stride+2]
Initial WebM release
2010-05-18 17:58:33 +02:00
punpcklbw mm3, mm2
punpcklbw mm7, mm6
psadbw mm3, mm0
psadbw mm7, mm0
paddw mm3, mm4
paddw mm7, mm5
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
mov rcx, result_ptr
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
punpckldq mm1, mm3
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movq [rcx], mm1
movd [rcx+8], mm7
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_DESTROY_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;unsigned int vp9_sad16x16_sse3(
Initial WebM release
2010-05-18 17:58:33 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *ref_ptr,
; int ref_stride,
; int max_err)
;%define lddqu movdqu
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_sad16x16_sse3)
sym(vp9_sad16x16_sse3):
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_CREATE_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Improve vp8_sad16x16_sse3 function In real-time mode, vp8_sad16x16 function is called heavily in motion search part. Improvement of this function gives 1.2% encoding performance gain (real-time mode, tulip clip). Change-Id: I23c401fc40c061f732a9767e8d383737a179bd58
2011-02-14 22:23:49 +01:00
mov end_ptr, 4
pxor xmm7, xmm7
Initial WebM release
2010-05-18 17:58:33 +02:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
.vp9_sad16x16_sse3_loop:
Improve vp8_sad16x16_sse3 function In real-time mode, vp8_sad16x16 function is called heavily in motion search part. Improvement of this function gives 1.2% encoding performance gain (real-time mode, tulip clip). Change-Id: I23c401fc40c061f732a9767e8d383737a179bd58
2011-02-14 22:23:49 +01:00
movdqa xmm0, XMMWORD PTR [src_ptr]
movdqu xmm1, XMMWORD PTR [ref_ptr]
movdqa xmm2, XMMWORD PTR [src_ptr+src_stride]
movdqu xmm3, XMMWORD PTR [ref_ptr+ref_stride]
Initial WebM release
2010-05-18 17:58:33 +02:00
Improve vp8_sad16x16_sse3 function In real-time mode, vp8_sad16x16 function is called heavily in motion search part. Improvement of this function gives 1.2% encoding performance gain (real-time mode, tulip clip). Change-Id: I23c401fc40c061f732a9767e8d383737a179bd58
2011-02-14 22:23:49 +01:00
lea src_ptr, [src_ptr+src_stride*2]
lea ref_ptr, [ref_ptr+ref_stride*2]
Initial WebM release
2010-05-18 17:58:33 +02:00
Improve vp8_sad16x16_sse3 function In real-time mode, vp8_sad16x16 function is called heavily in motion search part. Improvement of this function gives 1.2% encoding performance gain (real-time mode, tulip clip). Change-Id: I23c401fc40c061f732a9767e8d383737a179bd58
2011-02-14 22:23:49 +01:00
movdqa xmm4, XMMWORD PTR [src_ptr]
movdqu xmm5, XMMWORD PTR [ref_ptr]
movdqa xmm6, XMMWORD PTR [src_ptr+src_stride]
Initial WebM release
2010-05-18 17:58:33 +02:00
Improve vp8_sad16x16_sse3 function In real-time mode, vp8_sad16x16 function is called heavily in motion search part. Improvement of this function gives 1.2% encoding performance gain (real-time mode, tulip clip). Change-Id: I23c401fc40c061f732a9767e8d383737a179bd58
2011-02-14 22:23:49 +01:00
psadbw xmm0, xmm1
Initial WebM release
2010-05-18 17:58:33 +02:00
Improve vp8_sad16x16_sse3 function In real-time mode, vp8_sad16x16 function is called heavily in motion search part. Improvement of this function gives 1.2% encoding performance gain (real-time mode, tulip clip). Change-Id: I23c401fc40c061f732a9767e8d383737a179bd58
2011-02-14 22:23:49 +01:00
movdqu xmm1, XMMWORD PTR [ref_ptr+ref_stride]
Initial WebM release
2010-05-18 17:58:33 +02:00
Improve vp8_sad16x16_sse3 function In real-time mode, vp8_sad16x16 function is called heavily in motion search part. Improvement of this function gives 1.2% encoding performance gain (real-time mode, tulip clip). Change-Id: I23c401fc40c061f732a9767e8d383737a179bd58
2011-02-14 22:23:49 +01:00
psadbw xmm2, xmm3
psadbw xmm4, xmm5
psadbw xmm6, xmm1
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
lea src_ptr, [src_ptr+src_stride*2]
lea ref_ptr, [ref_ptr+ref_stride*2]
Initial WebM release
2010-05-18 17:58:33 +02:00
Improve vp8_sad16x16_sse3 function In real-time mode, vp8_sad16x16 function is called heavily in motion search part. Improvement of this function gives 1.2% encoding performance gain (real-time mode, tulip clip). Change-Id: I23c401fc40c061f732a9767e8d383737a179bd58
2011-02-14 22:23:49 +01:00
paddw xmm7, xmm0
paddw xmm7, xmm2
paddw xmm7, xmm4
paddw xmm7, xmm6
Initial WebM release
2010-05-18 17:58:33 +02:00
Improve vp8_sad16x16_sse3 function In real-time mode, vp8_sad16x16 function is called heavily in motion search part. Improvement of this function gives 1.2% encoding performance gain (real-time mode, tulip clip). Change-Id: I23c401fc40c061f732a9767e8d383737a179bd58
2011-02-14 22:23:49 +01:00
sub end_ptr, 1
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
jne .vp9_sad16x16_sse3_loop
Initial WebM release
2010-05-18 17:58:33 +02:00
Improve vp8_sad16x16_sse3 function In real-time mode, vp8_sad16x16 function is called heavily in motion search part. Improvement of this function gives 1.2% encoding performance gain (real-time mode, tulip clip). Change-Id: I23c401fc40c061f732a9767e8d383737a179bd58
2011-02-14 22:23:49 +01:00
movq xmm0, xmm7
psrldq xmm7, 8
paddw xmm0, xmm7
movq rax, xmm0
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_DESTROY_X3
Initial WebM release
2010-05-18 17:58:33 +02:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;void vp9_copy32xn_sse3(
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *dst_ptr,
; int dst_stride,
; int height);
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_copy32xn_sse3)
sym(vp9_copy32xn_sse3):
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
STACK_FRAME_CREATE_X3
Use local labels for jumps/loops in x86 assembly. Prepend . to local labels in assembly code. This allows non unique labels within a file. Also makes profiling information more informative by keeping the function name with the loop name. Change-Id: I7a983cb3a5ba2413d5dafd0a37936b268fb9e37f
2011-08-23 00:29:41 +02:00
.block_copy_sse3_loopx4:
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
lea end_ptr, [src_ptr+src_stride*2]
movdqu xmm0, XMMWORD PTR [src_ptr]
movdqu xmm1, XMMWORD PTR [src_ptr + 16]
movdqu xmm2, XMMWORD PTR [src_ptr + src_stride]
movdqu xmm3, XMMWORD PTR [src_ptr + src_stride + 16]
movdqu xmm4, XMMWORD PTR [end_ptr]
movdqu xmm5, XMMWORD PTR [end_ptr + 16]
movdqu xmm6, XMMWORD PTR [end_ptr + src_stride]
movdqu xmm7, XMMWORD PTR [end_ptr + src_stride + 16]
lea src_ptr, [src_ptr+src_stride*4]
lea end_ptr, [ref_ptr+ref_stride*2]
movdqa XMMWORD PTR [ref_ptr], xmm0
movdqa XMMWORD PTR [ref_ptr + 16], xmm1
movdqa XMMWORD PTR [ref_ptr + ref_stride], xmm2
movdqa XMMWORD PTR [ref_ptr + ref_stride + 16], xmm3
movdqa XMMWORD PTR [end_ptr], xmm4
movdqa XMMWORD PTR [end_ptr + 16], xmm5
movdqa XMMWORD PTR [end_ptr + ref_stride], xmm6
movdqa XMMWORD PTR [end_ptr + ref_stride + 16], xmm7
lea ref_ptr, [ref_ptr+ref_stride*4]
sub height, 4
cmp height, 4
Use local labels for jumps/loops in x86 assembly. Prepend . to local labels in assembly code. This allows non unique labels within a file. Also makes profiling information more informative by keeping the function name with the loop name. Change-Id: I7a983cb3a5ba2413d5dafd0a37936b268fb9e37f
2011-08-23 00:29:41 +02:00
jge .block_copy_sse3_loopx4
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
;Check to see if there is more rows need to be copied.
cmp height, 0
Use local labels for jumps/loops in x86 assembly. Prepend . to local labels in assembly code. This allows non unique labels within a file. Also makes profiling information more informative by keeping the function name with the loop name. Change-Id: I7a983cb3a5ba2413d5dafd0a37936b268fb9e37f
2011-08-23 00:29:41 +02:00
je .copy_is_done
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
Use local labels for jumps/loops in x86 assembly. Prepend . to local labels in assembly code. This allows non unique labels within a file. Also makes profiling information more informative by keeping the function name with the loop name. Change-Id: I7a983cb3a5ba2413d5dafd0a37936b268fb9e37f
2011-08-23 00:29:41 +02:00
.block_copy_sse3_loop:
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
movdqu xmm0, XMMWORD PTR [src_ptr]
movdqu xmm1, XMMWORD PTR [src_ptr + 16]
lea src_ptr, [src_ptr+src_stride]
movdqa XMMWORD PTR [ref_ptr], xmm0
movdqa XMMWORD PTR [ref_ptr + 16], xmm1
lea ref_ptr, [ref_ptr+ref_stride]
sub height, 1
Use local labels for jumps/loops in x86 assembly. Prepend . to local labels in assembly code. This allows non unique labels within a file. Also makes profiling information more informative by keeping the function name with the loop name. Change-Id: I7a983cb3a5ba2413d5dafd0a37936b268fb9e37f
2011-08-23 00:29:41 +02:00
jne .block_copy_sse3_loop
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
Use local labels for jumps/loops in x86 assembly. Prepend . to local labels in assembly code. This allows non unique labels within a file. Also makes profiling information more informative by keeping the function name with the loop name. Change-Id: I7a983cb3a5ba2413d5dafd0a37936b268fb9e37f
2011-08-23 00:29:41 +02:00
.copy_is_done:
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
STACK_FRAME_DESTROY_X3
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;void vp9_sad16x16x4d_sse3(
Initial WebM release
2010-05-18 17:58:33 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *ref_ptr_base,
; int ref_stride,
; int *results)
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_sad16x16x4d_sse3)
sym(vp9_sad16x16x4d_sse3):
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_CREATE_X4
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
PROCESS_16X2X4 0, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_16X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_16X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_16X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_16X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_16X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_16X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_16X2X4 2, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%if ABI_IS_32BIT
Initial WebM release
2010-05-18 17:58:33 +02:00
pop rbp
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%endif
mov rcx, result_ptr
Initial WebM release
2010-05-18 17:58:33 +02:00
movq xmm0, xmm4
psrldq xmm4, 8
paddw xmm0, xmm4
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
;-
movq xmm0, xmm5
psrldq xmm5, 8
paddw xmm0, xmm5
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx+4], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
;-
movq xmm0, xmm6
psrldq xmm6, 8
paddw xmm0, xmm6
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx+8], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
;-
movq xmm0, xmm7
psrldq xmm7, 8
paddw xmm0, xmm7
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx+12], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_DESTROY_X4
Initial WebM release
2010-05-18 17:58:33 +02:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;void vp9_sad16x8x4d_sse3(
Initial WebM release
2010-05-18 17:58:33 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *ref_ptr_base,
; int ref_stride,
; int *results)
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_sad16x8x4d_sse3)
sym(vp9_sad16x8x4d_sse3):
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_CREATE_X4
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
PROCESS_16X2X4 0, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_16X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_16X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_16X2X4 2, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%if ABI_IS_32BIT
Initial WebM release
2010-05-18 17:58:33 +02:00
pop rbp
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%endif
mov rcx, result_ptr
Initial WebM release
2010-05-18 17:58:33 +02:00
movq xmm0, xmm4
psrldq xmm4, 8
paddw xmm0, xmm4
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
;-
movq xmm0, xmm5
psrldq xmm5, 8
paddw xmm0, xmm5
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx+4], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
;-
movq xmm0, xmm6
psrldq xmm6, 8
paddw xmm0, xmm6
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx+8], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
;-
movq xmm0, xmm7
psrldq xmm7, 8
paddw xmm0, xmm7
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd [rcx+12], xmm0
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_DESTROY_X4
Initial WebM release
2010-05-18 17:58:33 +02:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;void int vp9_sad8x16x4d_sse3(
Initial WebM release
2010-05-18 17:58:33 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *ref_ptr,
; int ref_stride,
; int *results)
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_sad8x16x4d_sse3)
sym(vp9_sad8x16x4d_sse3):
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_CREATE_X4
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
PROCESS_8X2X4 0, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_8X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_8X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_8X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_8X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_8X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_8X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_8X2X4 2, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%if ABI_IS_32BIT
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
pop rbp
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%endif
mov rcx, result_ptr
Remove stack shadowing for x86-64 x86-64 passes most arguments in registers. There is no need to push them to the stack before using them. Change-Id: I13c683f1358782682ecafaf1df3fb0af23b978ea
2010-10-15 02:10:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
punpckldq mm4, mm5
punpckldq mm6, mm7
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movq [rcx], mm4
movq [rcx+8], mm6
STACK_FRAME_DESTROY_X4
Initial WebM release
2010-05-18 17:58:33 +02:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;void int vp9_sad8x8x4d_sse3(
Initial WebM release
2010-05-18 17:58:33 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *ref_ptr,
; int ref_stride,
; int *results)
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_sad8x8x4d_sse3)
sym(vp9_sad8x8x4d_sse3):
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_CREATE_X4
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
PROCESS_8X2X4 0, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_8X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_8X2X4 1, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
PROCESS_8X2X4 2, src_ptr, r0_ptr, r1_ptr, r2_ptr, r3_ptr, src_stride, ref_stride
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%if ABI_IS_32BIT
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
pop rbp
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%endif
mov rcx, result_ptr
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
punpckldq mm4, mm5
punpckldq mm6, mm7
Revert "Remove stack shadowing for x86-64" This reverts commit 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851. Change-Id: Ia640be8cbc134432914849c1750f62575ea084e6
2010-11-11 17:19:37 +01:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movq [rcx], mm4
movq [rcx+8], mm6
STACK_FRAME_DESTROY_X4
Initial WebM release
2010-05-18 17:58:33 +02:00
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
;void int vp9_sad4x4x4d_sse3(
Initial WebM release
2010-05-18 17:58:33 +02:00
; unsigned char *src_ptr,
; int src_stride,
; unsigned char *ref_ptr,
; int ref_stride,
; int *results)
Change encoder vp8_ and vp8cx_ public symbol prefixes to vp9_. Change-Id: Ie2e3652591b010ded10c216501ce24fd95d0aec5
2012-10-30 20:58:42 +01:00
global sym(vp9_sad4x4x4d_sse3)
sym(vp9_sad4x4x4d_sse3):
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_CREATE_X4
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm0, DWORD PTR [src_ptr]
movd mm1, DWORD PTR [r0_ptr]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm2, DWORD PTR [src_ptr+src_stride]
movd mm3, DWORD PTR [r0_ptr+ref_stride]
Initial WebM release
2010-05-18 17:58:33 +02:00
punpcklbw mm0, mm2
punpcklbw mm1, mm3
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm4, DWORD PTR [r1_ptr]
movd mm5, DWORD PTR [r2_ptr]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm6, DWORD PTR [r3_ptr]
movd mm2, DWORD PTR [r1_ptr+ref_stride]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm3, DWORD PTR [r2_ptr+ref_stride]
movd mm7, DWORD PTR [r3_ptr+ref_stride]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw mm1, mm0
punpcklbw mm4, mm2
punpcklbw mm5, mm3
punpcklbw mm6, mm7
psadbw mm4, mm0
psadbw mm5, mm0
psadbw mm6, mm0
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
lea src_ptr, [src_ptr+src_stride*2]
lea r0_ptr, [r0_ptr+ref_stride*2]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
lea r1_ptr, [r1_ptr+ref_stride*2]
lea r2_ptr, [r2_ptr+ref_stride*2]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
lea r3_ptr, [r3_ptr+ref_stride*2]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm0, DWORD PTR [src_ptr]
movd mm2, DWORD PTR [r0_ptr]
Initial WebM release
2010-05-18 17:58:33 +02:00
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm3, DWORD PTR [src_ptr+src_stride]
movd mm7, DWORD PTR [r0_ptr+ref_stride]
Initial WebM release
2010-05-18 17:58:33 +02:00
punpcklbw mm0, mm3
punpcklbw mm2, mm7
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm3, DWORD PTR [r1_ptr]
movd mm7, DWORD PTR [r2_ptr]
Initial WebM release
2010-05-18 17:58:33 +02:00
psadbw mm2, mm0
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%if ABI_IS_32BIT
Initial WebM release
2010-05-18 17:58:33 +02:00
mov rax, rbp
pop rbp
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
%define ref_stride rax
%endif
mov rsi, result_ptr
Initial WebM release
2010-05-18 17:58:33 +02:00
paddw mm1, mm2
movd [rsi], mm1
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm2, DWORD PTR [r1_ptr+ref_stride]
movd mm1, DWORD PTR [r2_ptr+ref_stride]
Initial WebM release
2010-05-18 17:58:33 +02:00
punpcklbw mm3, mm2
punpcklbw mm7, mm1
psadbw mm3, mm0
psadbw mm7, mm0
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
movd mm2, DWORD PTR [r3_ptr]
movd mm1, DWORD PTR [r3_ptr+ref_stride]
Initial WebM release
2010-05-18 17:58:33 +02:00
paddw mm3, mm4
paddw mm7, mm5
movd [rsi+4], mm3
punpcklbw mm2, mm1
movd [rsi+8], mm7
psadbw mm2, mm0
paddw mm2, mm6
movd [rsi+12], mm2
Remove stack shadowing for x86-x64 for SAD functions. x86-64 passes arguments in registers. There is no need to push them to the stack before using them. This fixes 15acc84f10cefd98b2f8dbd2eac2cc92c5a3f851 where ebx was not getting preserved on x86. Change-Id: I1214b5f818a0201f75ab6ad7d5c6f448e09b16c2
2010-11-11 19:15:58 +01:00
STACK_FRAME_DESTROY_X4
Preload reference area to an intermediate buffer in sub-pixel motion search In sub-pixel motion search, the search range is small(+/- 3 pixels). Preload whole search area from reference buffer into a 32-byte aligned buffer. Then in search, load reference data from this buffer instead. This keeps data in cache, and reduces the crossing cache- line penalty. For tulip clip, tests on Intel Core2 Quad machine(linux) showed encoder speed improvement: 3.4% at --rt --cpu-used =-4 2.8% at --rt --cpu-used =-3 2.3% at --rt --cpu-used =-2 2.2% at --rt --cpu-used =-1 Test on Atom notebook showed only 1.1% speed improvement(speed=-4). Test on Xeon machine also showed less improvement, since unaligned data access latency is greatly reduced in newer cores. Next, I will apply similar idea to other 2 sub-pixel search functions for encoding speed > 4. Make this change exclusively for x86 platforms. Change-Id: Ia7bb9f56169eac0f01009fe2b2f2ab5b61d2eb2f
2011-06-28 15:14:13 +02:00
Reference in New Issue Copy Permalink