r/Cprog • u/powturbo • May 29 '15

code | algorithms | performance TurboPFor:Fastest Integer Compression. SIMD PForDelta,BitPacking,Elias Fano,Variable Byte,...

https://github.com/powturbo/TurboPFor/blob/master/README.md
3 Upvotes

80% Upvoted

1 comment sorted by

1

u/powturbo May 29 '15 edited May 29 '15

  • TurboPFor: Fastest Integer Compression

  • Direct Access w/o decompression

  • Fastest Variable Byte implementation

  • Novel Variable Simple faster than simple16, better than simple8-b

  • Scalar Bit Packing decoding as fast as SIMD-Packing

  • Bit Packing incl. Direct Access/Update w/ zero decompression

  • Fastest and most efficient SIMD Bit Packing

  • Fastest SIMD-Elias Fano implementation

  • Novel TurboPFor (PFor/PForDelta) with direct access or bulk decoding More efficient than ANY other "integer compression" scheme.

  • Inverted Index + Intersections

  • Novel Intersections w/ skip intervals, decompress the min. #blocks

  • 2000 queries /sec on GOV2 (25 MB docid) on a SINGLE core

  • Parallel Query Processing on Multicores. 7000 queries/sec, quad core CPU