Commit Graph

77 Commits

Author SHA1 Message Date
Greg Tucker
e19101f5de doc: Add detail of internal checksum value in gzip/zlib mode
Change-Id: I8f7fdcec40371e61eb19248cb24c9837d0845a0c
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2018-12-04 10:11:40 -07:00
Greg Tucker
2e212f28fa build: Fix for mac nasm lack of symbol types
Change-Id: I9ee86a3e32876d3860477c8365fc459d94a8920e
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2018-11-29 13:54:36 -07:00
Greg Tucker
06b926fbb6 igzip: Fix portability issue when bad window size passed
If a user passes an invalid size for window bits it could have triggered an
undefined shift by larger than variable size.

Change-Id: Ib2999b094af075596be3333418667ae9b498e2ae
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2018-10-25 14:43:27 -07:00
John Kariuki
6e2013391a mem: Add zero detect memory functions
This patch introduces the base, avx and sse optimized zero detect memory function.
The zero detect memory function tests if a memory region is all zeroes. If all the
bytes in the memory region are zero, the function return a zero. Otherwise, if the
memory region has non zero bytes, the zero detect function returns a 1.

Change-Id: If965badf750377124d0067d09f888d0419554998
Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
2018-09-25 14:33:31 -07:00
Roy Oursler
8e4f1a1a38 igzip: Create generic deflate performance utility
Change-Id: Idf180660797f97a492550fb557652f036cd55509
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2018-09-20 11:12:02 -07:00
Roy Oursler
ff1928e8ec igzip: Create functions to write gzip/zlib headers
Change-Id: If5aaa277a01214bd36406ee11680df0904ad12f7
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2018-09-20 11:12:02 -07:00
Roy Oursler
43374f6776 igzip: Implement gzip/zlib header/trailer parsing
Change-Id: I3fe8653f2286212a9d6c6ecfa3b78752b2cac8ef
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2018-09-20 11:12:02 -07:00
Roy Oursler
11542000f0 igzip: Implement limited window size for inflate
Change-Id: Ib7fce6a51db99fc7e11f06f5916c2b755bfc5c67
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2018-09-20 11:12:02 -07:00
Roy Oursler
03bef684a4 igzip: Setup for variable hash mask
Change-Id: I3be94dbc40c2e02dcff4f89e5a9df8ed1f744f02
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2018-09-18 14:27:25 -07:00
Roy Oursler
6317ce2b78 igzip: Setup for variable lookback distance
Change-Id: Idd52c9392113dfc54feea3c66916a7f5aa128bef
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2018-09-18 14:27:25 -07:00
Roy Oursler
bb3c6c28c8 igzip: Remove need for total_in_start
Change-Id: Ie9ab3e702ce07a5ba8d6fb3275da98e03c25822b
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2018-08-21 13:18:05 -07:00
Greg Tucker
a1869430c7 build: Fix warnings on mac for objects defining no symbols
Change-Id: I13ef334ae23a3370cbf2a5409974fa0dc9fba7a5
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2018-06-21 14:39:08 -07:00
Greg Tucker
da1aee8714 igzip: Remove references to types.h
Previously included just for struct alignment but all restrictions
have been removed.

Change-Id: I3fa7cbab86fce419b3b3bfccb48d9129bd77cf64
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2018-06-21 14:39:08 -07:00
Roy Oursler
2de5a0fd88 igzip: Swap length code lookup with length lookup
Change-Id: I9f1c3ea5353f2c2fa98bab1d0cb1eb3c7b7397f6
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2018-06-14 15:30:14 -07:00
Roy Oursler
222a68f760 igzip: Implement multibyte decode
Change-Id: I923a57a01f696f2082945fafcc2773655b9a5beb
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2018-06-14 15:30:14 -07:00
Roy Oursler
7a1dc55c27 igzip: Increase size of large short code lookup
Change-Id: I05a564d1759ae417a966f3a12621799db0edf80a
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2018-06-14 15:30:14 -07:00
Daniel Verkamp
99b45db17e types.h: remove [U]INT{8,16,32,64} typedefs
These can be replaced with the <stdint.h> types.

Additionally, the existing definitions weren't correct on some platforms
(e.g. IA-32, where 'long', used for INT64/UINT64, is only 32 bits).

Change-Id: I1d9235c693ca2dc0c51d085128cecc4effc165fd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-29 09:35:25 -07:00
Roy Oursler
48119c5c87 igzip: Increase long_code_lookup struct size to fix buffer overflow
Change-Id: I6546dcb7ffcd5895292d06fdc748c3cf279a4542
2017-12-19 10:24:23 -07:00
Greg Tucker
491035d956 crc: Add t10dif+copy function
Change-Id: Ic6c424a0aa746aa06643575f7fcc8d6944cbfc0e
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2017-12-18 15:59:17 -07:00
Roy Oursler
8965584ae3 igzip: Implement large hash table method as level 2
Move current level 2 compression to be level 3 and create a level 2
compression which is the same algorithm as level 1 but with a larger
hashtable.

Change-Id: I398a6017c2ff218dded24e229fb51c23ea69f314
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-12-15 14:27:14 -07:00
Roy Oursler
7a12bcb2a8 igzip: Separate concept of level and compression method
Change-Id: I82a5fbeb93adc77057893c643e044e311e4f393c
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-12-15 14:27:14 -07:00
Roy Oursler
fe68f02dac igzip: Move hash_table and symbol histogram for icf compression
Change-Id: I50df9c8915ff3e1af450aeb8e4c0cc3baf9624ae
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-12-15 14:27:14 -07:00
Roy Oursler
06cd70481c igzip: Separate defines for LVL1 and LVL0 hash tables
Change-Id: I19bdec8d2d0c74083bc1695763c9630516995885
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-12-15 14:27:14 -07:00
Roy Oursler
4ae2d1be29 igzip: Implement optimized level 2 compression
Change-Id: I8cf5bcd56f290d17205ac36dc2828c8acfc66947
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-12-15 14:27:14 -07:00
Roy Oursler
49156c3568 igzip: Implement block buffer
Change-Id: I9d7942740557e4ffaf8e223e190f4bd4e4f47b1e
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-12-15 14:27:14 -07:00
Roy Oursler
6049ce3ca7 igzip: Modify state to record total_in on deflate call
Change-Id: I13e5878a227732545aee5a762bf5a9a75ce73f02
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-12-15 14:27:14 -07:00
Roy Oursler
59b9990a39 igzip: Remove file_start from zstate
Change-Id: Ia4cb6dc86da54cc771f25a6d958bea730caa4801
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-12-15 14:27:14 -07:00
Roy Oursler
cad5e7d479 ec: Fix doxygen comment format for gen_rs_matrix
Change-Id: I505155b0d57576814d876cd2d3595e522e03c469
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-10-17 10:18:45 -07:00
Roy Oursler
80bfbb33df igzip: Remove DECLARE_ALIGNED and optimize structure layout
Change-Id: I95bc3b8e2e30aff0d596c743158337400c4eb486
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-08-18 14:35:44 -07:00
Xiaodong Liu
3ab8239097 multibinary: move WRT_OPT macro to common header
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
2017-08-18 14:24:57 -07:00
Roy Oursler
34c341db35 igzip: Add reset functions for both deflate and inflate.
Change-Id: I8677a4365ac5c2343751660176f3b2eb4746ddfe
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-06-26 04:14:44 -04:00
Xiaodong Liu
7137c4a5be crc: release crc32_gzip_refl code outside
Merge crc32_gzip_refl function definitions, base code, multi-binary
code into crc32.h, crc32_base,c and crc_multibinary.asm in order to
keep consistency. Add crc32_gzip_refl files into crc/Makefile.am
Original crc32_gzip_refl removed NOT operation, re-add it.

Change-Id: Ib0cbbeb1ab3c9fcafec324b392596d2514202424
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
2017-06-26 04:14:27 -04:00
Roy Oursler
ed15402f5b igzip: Add stateful dictionary support
Change-Id: I75dbac947787bc0041674468c88d0aa41b8b082f
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-06-26 04:11:48 -04:00
Roy Oursler
82a6ac65dc ec: Determine exact conditions where gf_gen_rs_matrix works
Add a program calculating some of the exact conditions where gf_gen_rs_matrix
works, add comments stating these bounds to gf_gen_rs_matrix, and fix erasure
code test that violates the bounds.

Change-Id: I1d0010b09fea97731bfd24f4f76e24609538b24f
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-06-26 04:11:12 -04:00
Greg Tucker
fc1467deb2 Format only patch from iindent and remove_whitespace
Change-Id: I114bfcfa8750c7ba3a50ad2be9dd9e87cb7a1042
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2017-06-26 04:10:47 -04:00
Greg Tucker
278a51979a Fix test helper for windows and gcc7 issues
Change-Id: Idb61d32d928536918dd243df825060c1b5bc484d
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2017-06-26 04:06:22 -04:00
Roy Oursler
5a55e3096c igzip: Avx512 version for encdode_df
Change-Id: I1625a3d7e016805791cfd09e31909562f432fd71
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-06-21 05:56:05 -04:00
Roy Oursler
64143a741e igzip: Implement type 0 blocks for level 1 compress
Change-Id: If55ab161623d29fa6fb08df3bc813e654918e592
2017-06-21 05:46:38 -04:00
Roy Oursler
87652b4489 igzip: Modify inflate to optionally calculate adler32 hash
Change-Id: I314617b89b59d53608e464c7d2cf299faa3528b5
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-06-21 05:42:12 -04:00
Roy Oursler
4259169107 igzip: Implement zlib compression format
Change-Id: I3d3cca425a494ac629cea230de74e3d32fcaea79
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-06-21 05:41:05 -04:00
Greg Tucker
c68e15dc53 Fix test helper function for mingw builds
Change-Id: Ic24b3ba89bf03bfbc829a78d2cb8f820885ada7e
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2017-06-21 05:33:00 -04:00
Hailiang Wang
52f644d3ff igzip:use fgetpos() to replace ftell() to get file's length.
Change-Id: Ia1f3c06e92c01da9d22b3d70b2b05fe4808c9f2c
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
2017-06-21 05:22:07 -04:00
Roy Oursler
871ad43fad igzip: Fix bug in isal_deflate where processed is not calculated correctly.
Change-Id: I61e15a18ebe3130e73010337d2d41f59e2227f08
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-03-30 17:20:32 -07:00
Roy Oursler
bad6569acc igzip: Document and add some defines for higher level compressions
Change-Id: I4d9fc9d7f697e721e247a64b2522ef2793c2d07f
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-03-30 17:18:55 -07:00
Roy Oursler
e38ed4b54e igzip: Increase isal_mod_hist size to stop histogram overflow
Change-Id: I1c651c9625d0fb543cd89e53e2b78b391176ef68
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-03-30 17:11:51 -07:00
Roy Oursler
01dfbcc484 igzip: implement igzip two pass
Change-Id: I9564b2da251a02197b39cab5f141e7aff1ae8439
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2017-03-30 16:50:48 -07:00
Greg Tucker
d549db38e5 doc: Fix igzip header description for isal_create_hufftables
Signed-off-by: Greg Tucker <greg.b.tucker@intel.com>
2017-01-11 10:31:44 -07:00
Roy Oursler
7fefb53938 igzip: Add a hufftable for the static Huffman code in Deflate
Change-Id: If8aadce05bcb705e608fe3ed6028be9a7448fa59
Signed-off-by: Roy Oursler <roy.j.oursler@intel.com>
2016-12-06 16:15:57 -07:00
Xiaodong Liu
3d66317189 crc64: add jones and iso format, crc64 code clean
1. Add normal and reflected bits order functions for ISO format and
   Jones coefficients format.
2. Add a multi-binary macro for crc64 functions.
3. In order to decrease number of repeated test.c and perf.c files,
   using crc64_funcs_test.c and cr crc64_funcs_perf.c.
4. Add crc64_example.c to take the demonstration role.

Change-Id: Icb8c14f1a84cd98f58eb12206ca605dea8a2cefb
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
2016-12-06 13:48:13 -07:00
Xiaodong Liu
90f0ea90de crc64: add crc64 ecma polynomial
crc64_ecma_norm is used for the normal format.
crc64_ecma_refl is used for the refleced format.

Change-Id: I8fa8aad48ed995ea7edcdb8e123e1a5f1a1f01ad
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
2016-12-06 13:27:39 -07:00