Commit Graph

308 Commits

Author SHA1 Message Date
Greg Tucker
dca9dd221e igzip: Use unaligned load on static header to fix usan
Clang with sanitizer on was catching on cast of static header.
Switching to uload64 macro for better general solution.

Change-Id: I495d440407bb1773841e2f7cdc48bd95fc1a2df4
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2020-11-04 12:40:08 -07:00
Greg Tucker
269df8a67d igzip: Fix order of args check in new dictionary function
In the newly added function isal_deflate_process_dict(), a null check
was added to the dictionary struct but was ineffectual because of the
order.

Change-Id: I3b3e70997210794de102b1348e1467295871cee2
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2020-11-03 08:50:30 -07:00
Greg Tucker
79143208ac test: Add testing for new dictionary functions
Change-Id: I0b0a151374acfe9b44c7a2be4bb959df59356d97
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2020-10-28 17:28:43 -07:00
Greg Tucker
19035917f4 igzip: Add new functions for faster dictionary compression
Change-Id: Id55728fea286d144f8a11192ab02ccc8503d7b25
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2020-10-21 18:09:49 -07:00
Greg Tucker
438ecd8187 Update custom hufftable tool for saving histogram
Change-Id: I515217b19373b8f996ff887268862cf2b102f3a4
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2020-10-21 18:09:49 -07:00
Greg Tucker
89f7c46cd5 Change igzip_file_perf to accept 0 time
Change-Id: Ie2edf8e742d0bcdd9a008704f997006f8f5009ac
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2020-10-21 18:09:49 -07:00
Greg Tucker
9968e7a032 Change gen cust hufftables to accept dictionary
Change-Id: I4eed03bdb91030b16b3ecfd8076adc890e4f59a2
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2020-10-21 18:09:49 -07:00
Greg Tucker
63dffab948 igzip: Change pre-gen inflate table to multi-symbol
Change-Id: I4b0dac1e5aa2796be17644b893e3b6c7aed05876
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2020-10-21 18:09:49 -07:00
Greg Tucker
d7927673ba igzip: Inflate detect pre-gen header and use pre-expanded
Performance improvement for inflate to skip the time-consuming process of decode
table expansion when the header matches a known common dymanic one such as
produced by level 0 compression.

Change-Id: Ia2550b812a062b7cc2eb1b72bcb609f1a631e40b
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2020-10-21 18:09:49 -07:00
Greg Tucker
93049d0d1f igzip: Fix read header for correct null checking and init
Issue with reading header only appears when combined with new feature in cli of
multiple concatenated gzip files.

Change-Id: Id8df9150c6f27d8b22e810b511291f3fcf136723
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2020-08-13 15:21:10 -07:00
H.J. Lu
cd888f01a4 x86: Add ENDBR32/ENDBR64 at function entries for Intel CET
To support Intel CET, all indirect branch targets must start with
ENDBR32/ENDBR64.  Here is a patch to define endbranch and add it to
function entries in x86 assembly codes which are indirect branch
targets as discovered by running testsuite on Intel CET machine and
visual inspection.

Verified with

$ CC="gcc -Wl,-z,cet-report=error -fcf-protection" CXX="g++ -Wl,-z,cet-report=error -fcf-protection" .../configure x86_64-linux
$ make -j8
$ make -j8 check

with both nasm and yasm on both CET and non-CET machines.

Change-Id: I9822578e7294fb5043a64ab7de5c41de81a7d337
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
2020-05-26 09:16:49 -07:00
Greg Tucker
ede04f0a1f build: Fix for windows to allow nasm use
Previously windows build could only use yasm because some procedural items such
as proc_start were not supported by nasm.  This adds a few macros and fixes so
nasm can be used to build on windows.

Change-Id: Ia05dc3ff482f33b0f915bb1be3c7df5e4a753b3a
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2020-03-17 18:05:46 -07:00
Hong Bo Peng
180c74aefd enable VSX SIMD in ISA-L for ppc64le
1) Implement the ErasureCode function in Altivec Intrinsics
  2) Coding style update

Change-Id: I2c81d035f4083e9b011dbf3b741f628813b68606
Thanks-to: Daniel Axtens <dja@axtens.net>
Signed-off-by: Hong Bo Peng <penghb@cn.ibm.com>
2020-02-20 09:40:43 -07:00
Zhang Jinde
a3d5cd8642 igzip: Fix clang error on dep generation
Clang errors when generating dependencies due to a stray semicolon following a
function definition.

Change-Id: Iefb4aca988b643bb62a69bbbaf197aca20a2d085
Signed-off-by: Zhang Jinde <zjd5536@163.com>
2020-01-17 10:25:32 -07:00
Zhang Jinde
163b6cd934 igzip: Fix for deflate logic buffer management
Fixes invalid logic that attempted to eliminate unnecessary copy of input to the
history buffer in cases where it is not required. Correction should improve
performance and not change functionality.

Change-Id: Ife24dcc9d920ce220b1a394031e971321737a171
Signed-off-by: Zhang Jinde <zjd5536@163.com>
2020-01-08 09:46:16 -07:00
Jerry Yu
fc69e8fc79 igzip: fix deflate hash bug
if next_in equal end_in, the function should
return.

Change-Id: I59e631bb1f24835fd43f943a3736e016c4e2d0ac
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
2019-12-31 13:15:35 -07:00
Jerry Yu
936d05fc4f igzip:Add decode huffman code for aarch64
Change-Id: If26cc4fd97b078b5f3b02e5f6f121a12ec73f671
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
2019-12-19 16:10:04 +08:00
Zhiyuan Zhu
2b8cc393af igzip: implement gen_icf_map with assembly
Change-Id: I74e6200a732acfaac44b7f5a82bd4a2215ba1535
Signed-off-by: Zhiyuan Zhu <zhiyuan.zhu@arm.com>
2019-12-13 07:54:12 +00:00
Zhiyuan Zhu
f430953f0a igzip: cleanup perf test related code
This patch addresses some cppcheck issues.
And some minor changes to maintain code consistency.

- Cleanup cppcheck issues.
  [log][igzip/igzip_perf.c] (error) Shifting signed 32-bit value by 31 bits is undefined behaviour
  [log][igzip/igzip_hist_perf.c:132]: (error) Memory leak: outbuf

- Some minor changes to maintain code consistency.
  igzip/igzip_build_hash_table_perf.c
  igzip/igzip_hist_perf.c
  igzip/igzip_semi_dyn_file_perf.c

- delete unused variable
  outbuf and outbuf_size from igzip/igzip_hist_perf.c

Change-Id: Icbbd8f70de689931c8a844d89e457af8d97c6793
Signed-off-by: Zhiyuan Zhu <zhiyuan.zhu@arm.com>
2019-12-06 15:33:20 +08:00
Zhiyuan Zhu
683364c47b igzip: implement encode_deflate_icf with assembly
Change-Id: I90b12da2d2a96bfdb47d29ab329648247a756585
Signed-off-by: Zhiyuan Zhu <zhiyuan.zhu@arm.com>
2019-11-29 14:45:45 -07:00
Jerry Yu
ce9e56054a igzip:implement deflate hash with assembly
Change-Id: I39b3a37cd291c40f597750839c27db2a6a571fe5
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
2019-11-01 14:41:46 -07:00
Jerry Yu
5d7724898d build: fix wrong use the register name
The third parameter must be 32bit register . Those assmebly
put 64bit register here , it is wrong .

Change-Id: Iebe17516b555a6a9b94ea7baa4778ad4b9dd0878
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
2019-11-01 18:11:00 +08:00
Zhiyuan Zhu
6b70da5051 igzip: implement set_long_icf_fg with assembly
Change-Id: I21ac55985a56c2b7b0a684934c076600d90f8b0a
Signed-off-by: Zhiyuan Zhu <zhiyuan.zhu@arm.com>
2019-10-31 11:02:54 -07:00
Zhiyuan Zhu
be4d035227 igzip: Optimize isal update histogram with arm64
Change-Id: I944f9497d990e831de5e066055a21ea7e8d6693b
Signed-off-by: Zhiyuan Zhu <zhiyuan.zhu@arm.com>
2019-10-11 09:59:47 -07:00
Zhiyuan Zhu
290456231c igzip: Implement deflate icf body/finish with assembly
Change-Id: I40e4a9be2ae654c881460056de9730176d3d097c
Signed-off-by: Zhiyuan Zhu <zhiyuan.zhu@arm.com>
2019-10-11 09:59:40 -07:00
Jerry Yu
f3bb041799 igzip: Implement deflate body/finish with assembly
Change-Id: I556af7976294f31abd72ac49366f7259e3baf399
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
2019-10-11 09:59:30 -07:00
Roy Oursler
d3caab9c3a build: Avoid requiring AVX512 define when using dispatch functions
Change-Id: I76af2d6ab7eb61ae531bbc7427650d08737c20ab
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-09-14 16:01:28 -07:00
Jerry Yu
5f45f3f310 igzip: Optimize adler32 with arm neon
Change-Id: I9b8932eb02ed6bc44756f6505e7efbfad1706b46
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
2019-08-29 10:11:06 +08:00
Jerry Yu
a2005c1fd6 igzip: enable multibinary interfaces
- Add dispatcher layer
- Alias functions with assmebly

Change-Id: I84da1be539d890db0df64e5ea989b2fd1f276949
Signed-off-by: Jerry Yu <jerry.h.yu@arm.com>
2019-08-29 10:08:58 +08:00
Jun He
b721db98e5 igzip: optimize convert_dist_to_dist_sym to branchless
convert_dist_to_dist_sym uses long if/else branch to get look back distance.
The distance calculation is well formed for each distance range, so it could
be optimized for a branchless version.

Change-Id: I4e1e5170f8b3238631f3048087f95acc53e4498e
Signed-off-by: Jun He <jun.he@arm.com>
2019-08-13 11:02:53 +08:00
Greg Tucker
e2997062fb igzip: Optimize routine to find msb
Change-Id: I40e7898e2139c04f261980ca10886debc917842a
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2019-08-12 14:28:33 -07:00
Greg Tucker
25374814c9 Format only changes for new indent version
Change-Id: I2b2a5caf1b31ad56665081145d5e7089fd34d0ab
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2019-07-03 15:17:02 -07:00
Roy Oursler
9c91a18c6e igzip: Fix igzip_rand_test to test on a single file
Change-Id: I21cb27f6012094adee6496f811792d6e3b11a8bc
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-05-21 13:52:41 -07:00
Greg Tucker
f1252a9e79 build: Add missing files for distcheck
Change-Id: I644d59eece8a9eae9d467f42d668b13d8fae0d81
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2019-05-01 16:48:10 -07:00
Roy Oursler
db59b1082f igzip: Remove undefined behavior in igzip.c
Remove unaligend data access in
write_deflate_header_unaligned_stateless

Change-Id: I7defa5621d8dc188edc51d22d29155ed3687c49d
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-04-29 16:22:07 -07:00
Greg Tucker
fc9f7493a0 igzip: Fix help message in perf test
Change-Id: I2d4fd7c0eee176570d79bebf5f6e453f3b7dbba6
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2019-04-18 10:59:50 -07:00
Greg Tucker
8f06ac6973 Add Adler32 performance test
Change-Id: I511dd34235c9a4ce2f3596d63236800cbf06703b
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2019-03-22 14:29:42 -07:00
Yibo Cai
57eed2f02b aarch64: Cleanup build issues
This patch addresses one build failure and fixes several build warnings
for Arm (some for x86 too).

- Fix dynamic relocation link failure of ld.bfd 2.30 on Arm
  [log] relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `xor_gen_neon' which may bind externally can not be used when making a shared object

- Add arch dependent "other_tests" to exclude x86 specific tests on Arm
  [log] isa-l/erasure_code/gf_2vect_dot_prod_sse_test.c:181: undefined reference to `gf_2vect_dot_prod_sse'

- Check "fread" return value to fix gcc warnings on Arm and x86
  [log] warning: ignoring return value of ‘fread’, declared with attribute warn_unused_result [-Wunused-result]
        fread(in_buf, 1, in_size, in_file);
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Fix issue of comparing "char" with "int" on Arm. "char" is unsigned
  on Arm by default, an unsigned char will never equal to EOF(-1).
  [Log] programs/igzip_cli.c:318:31: warning: comparison is always true due to limited range of data type [-Wtype-limits]
        while (tmp != '\n' && tmp != EOF)
                                  ^~

- Include <stdlib.h> to several files to fix build warnings on Arm
  [log] igzip/igzip_inflate_perf.c:339:5: warning: incompatible implicit declaration of built-in function ‘exit’
        exit(0);
        ^~~~

Change-Id: I82c1b63316b634b3d398ffba2ff815679d9051a8
Signed-off-by: Yibo Cai <yibo.cai@arm.com>
2019-03-20 10:15:40 +08:00
Greg Tucker
3c009347b1 Fix a few c99isms in unit tests
Change-Id: Iea9ba619e337d5abea7ee791ddf3dd27e0f3e60f
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2019-03-19 15:02:40 -07:00
Roy Oursler
31eca5035f igzip: Modify last byte retrieval method
Change-Id: I3ba7e9bd007277be543ba7a6299d5acc5c848bd2
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-03-07 09:28:04 -07:00
Roy Oursler
67c4e26580 igzip: Remove unneeded generation of k register
Change-Id: I79bfb3b3a3feeb969a0c0ec92b7ae0633f6be1b0
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-03-07 09:28:04 -07:00
Roy Oursler
28da992ad2 igzip: Reduce data used in encode_df_06.asm
Change-Id: I83dbca452840c07b0fd77faaf9d35c46065f8a08
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-03-07 09:28:04 -07:00
Roy Oursler
863e72189b igzip: Modify igzip_rand_test to optionally use getopt
Change-Id: I8ad8e7f18f292b54158f1cda2eef9aec3919d175
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-03-07 09:28:04 -07:00
Roy Oursler
35e90e73d5 igzip: Write out compressed data from igzip_perf
Change-Id: Iefea3e314e277112858874f826f54bdfa0172e04
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-03-07 09:28:04 -07:00
Roy Oursler
d90220d935 igzip: Implement window size for igzip_perf
Change-Id: I6f5e9453aaff980b44c3e6d56b113da7625ec36e
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-03-07 09:28:04 -07:00
Roy Oursler
699bb5bd3f all: Revamp performance testing to be time based
Change-Id: I6260d28e4adc974d8db0a1c770e3eb922d87f8e4
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-03-07 09:28:04 -07:00
Roy Oursler
bde3fc5ff1 igzip: Remove igzip_stateless_file_perf
Remove igzip_stateless_file_perf as all the functionality is included in
igzip_perf

Change-Id: Icfd4dfd25af1a3a6c16fa2c3299d277c18f204d9
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-03-07 09:28:04 -07:00
Roy Oursler
4ac2b7864b igzip: Remove igzip_inflate_perf
Remove igzip_inflate_perf as all the functionality is duplicated in igzip_perf

Change-Id: I510cc4643a3949e2fa8f30309b5d45a249320b9e
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-03-07 09:28:04 -07:00
Roy Oursler
623d2f0dc0 igzip: Bitbuf improvements
Update Bitbuf to use stdmac and decrease register dependencies by replaces a sub
with an and.

Change-Id: Iaadf3c6ef7f533540a7adb57a418e9e80e9b8503
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-03-07 09:28:04 -07:00
Roy Oursler
53b92e83f4 igzip: Avoid UB pointer arithmetic underflow for virtual file start
Change-Id: I95c0e6f004eaf70227a6419fc14bf0958d1f4538
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2019-03-07 09:27:50 -07:00