Professional Code Healer 🤲 Unicorn Hunter 🦄 Selfless and brave ⚔️ Pre-order my book 📕working @MinIO 🦋 klauspost on bsky
Denmark
Joined January 2008
- Tweets1.5K
- Following395
- Followers993
- Likes3K
Preview: New SIMD #compression mode in #golang MinLZ that uses pivco entropy coding on top of LZ matching.
Compression like zstd with 2-5x faster decompression. Currently implemented with AVX2, AVX512 and NEON (untested). Independent blocks, giving seekable streams.
Silesia, 1t
Upgrade the #golang compression library for zstd possible DOS fix: github.com/klauspost/compres…
v1.20.2: Minimal upgrade (bugfixes only)
v1.20.3: Full upgrade (improvements)
Back in the day Ultra Compressor II (DOS) trickled my interest in compression and data recovery as I was learning about computers.
With the knowledge I have now, it was fascinating to look through it with an LLM and also quite nostalgic re-creating the UI I only vaguely remember
So with an LLM I ported the original Borland to #golang. I tried to make it usable in a modern context, adding utf-8 and 64 bit sizes as an extension.
I have crosscompiled binaries for more than 30 platforms.
Archives produced with it should remain compatible with DOS versions (except for 2GB+ files of course). I even got it to work with Total Commander as a drop-in replacement for the executable.
Repo & Binaries: github.com/klauspost/uc2
Working on a #golang compression benchmark comparison site. Deflate speed improvements from latest compression library release... (JSON stream)
Why is #golang now a non-deterministic language with the 1.27 FMA changes? Pure Go code producing different output on different platforms. Avoiding that is why I *moved* to Go.
I will be going to tasteremote.com/nomadvillage before going to @golab_conf - looks fun - maybe someone wants to join?
Search Compressed Data for byte sequences with selective decompression using MinLZ v1.2.0 - available as cli and #golang API. Overhead is a fractional bit per byte, fully customizable and with fixed specification.
Ported "Radsort - a stable LSD radix sort with O(√n) space overhead" by Robert Clausecker and Florian Schintke to #golang. Good, specialized sorting for big arrays of dense keys (uint64, etc).
Very excited to be back at @golab_conf - super chill and well organised event! I will give a talk on the state of #golang data compression. See you there!
