)]}' { "log": [ { "commit": "48700052e5e9b5cbb3525e788bb678e15d1328a2", "tree": "35b5bb4461c9df44168687cf45dc77e7c6befa74", "parents": [ "3c8f44eab7d6dfa844fa60e762c09c599b9b50a4" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Thu Oct 29 09:22:18 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 29 09:22:18 2026" }, "message": "flate: keep match history across small flushes (#1282)\n\n* flate: keep match history across small flushes\n\nAt levels 1-6 and custom window sizes, a Flush with fewer than 128\nbytes buffered wrote the block as stored or Huffman-only and reset the\nencoder, discarding the match history. Data written after the flush\ncould not reference data written before it, so streams that flush\noften compressed poorly.\n\nPass the window to Encode on every flush, since it adds the window to\nthe history even when it is too small to search. Skip Encode only for\na window under 128 bytes on Close with no history, so one-shot small\npayloads are unchanged, while a small last write or a small payload\nafter a dictionary is matched against the history.\n\nSee https://go.dev/issue/81542.\n\n* flate: test small flush history with a custom window" }, { "commit": "3c8f44eab7d6dfa844fa60e762c09c599b9b50a4", "tree": "186ad6a6ddc78962896c3bb41da656e20a2dc68c", "parents": [ "4bc1f08dc7c8043e74df2d7e6b2d0f1984bbb908" ], "author": { "name": "Raghuvardhan-Reddy78", "email": "raghuvardhan@labs.digiscrypt.com", "time": "Thu Oct 29 09:21:15 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 29 09:21:15 2026" }, "message": "gzhttp: don\u0027t pool a decoder while a Read is still using it (#1288)\n\nTransport\u0027s gzipReader and zstdReader returned their decoder to the pool\nfrom Close, even when a Read on the same body was still blocked inside\nit. net/http allows Close to be called during a Read to abort it.\n\nFor zstd, the blocked Read then called Reset on the nil decoder and\npanicked. For gzip, the next response took the same gzip.Reader from the\npool, and the first Read\u0027s connection error ended up on that unrelated\nresponse.\n\nThe decoder now goes back to the pool once both Close and any Read in\nprogress are done, and a Read after Close returns an error." }, { "commit": "4bc1f08dc7c8043e74df2d7e6b2d0f1984bbb908", "tree": "547b8710f7bca08a2045a6ee504c4f9a365b2a7d", "parents": [ "7cfdd8fb9c59ef8d6407801a284439036ab02a9f" ], "author": { "name": "Wleeaf", "email": "ts.solidarity@gmail.com", "time": "Mon Oct 26 13:12:09 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 26 13:12:09 2026" }, "message": "gzhttp: keep Vary: Accept-Encoding when the handler sets Vary (#1284)\n\nThe wrapper adds \"Vary: Accept-Encoding\" before calling the handler, so a\nhandler that sets its own Vary with Header().Set(\"Vary\", ...) replaces it.\nThe response is then compressed without saying that it depends on\nAccept-Encoding, and a shared cache can serve the gzip body to a client\nthat does not accept it (RFC 9110 section 12.5.5).\n\nWhen compression starts, add Accept-Encoding to Vary unless it, or \"*\",\nis already listed.\n\nCo-authored-by: Wleeaf \u003c179125857+wleeaf@users.noreply.github.com\u003e" }, { "commit": "7cfdd8fb9c59ef8d6407801a284439036ab02a9f", "tree": "5f95314e0af5308bc691c3c0f5e2d9a9e1498a51", "parents": [ "62d757cd8e6fa111ee3e445071cb2273791372ce" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Sun Oct 25 12:57:50 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 25 12:57:50 2026" }, "message": "huff0: promote the 4X loop\u0027s input pointers to registers on arm64 (#1277)\n\n* huff0: keep 4X input pointers in stack slots so arm64 promotes them\n\nThe 4X loops kept the four input pointers and the inner-loop limit in\nthe context struct, so every iteration loaded and stored each pointer\n(4 loads, 4 stores) and loaded the limit, on arm64 as well as amd64.\nThey now live in stack slots, ctx is re-read from the argument outside\nthe inner loop (freeing its register for a separate reload pointer\ntemporary), and huff0 is generated with -arm64-promote-stack-slots.\nOn arm64 the slots become registers and each reload folds to one SUB\nplus the window load: the 4X main inner loop goes from 151 to 142\ninstructions and from 13 to 4 non-table memory operations. amd64 keeps\nthe same memory operations plus one MOVQ per iteration for the limit\ncompare. The 1X loops are unchanged.\n\nNeoverse N1, 12 alternating rounds, PCALIGN-pinned, one core: huff0\n4X loops -1.18% geomean (18/19 cells faster, none slower); zstd\nDecodeAll -0.30% geomean, none slower.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* huff0: compare the 4X inner-loop limit against its slot directly\n\nLoading the limit into a register before the compare added one\ninstruction per iteration on amd64 and cost Zen 4 about 0.6% in the 4X\nloops. The arm64 promotion accepts the CMPQ read of the slot, so the\narm64 output is unchanged and amd64 is back to master\u0027s instruction\ncount.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "62d757cd8e6fa111ee3e445071cb2273791372ce", "tree": "357c9e3c0fda30a5ee58a4ce24c9b9d1b9bc06a4", "parents": [ "35c6f98a5cb955b838d313596dd86c882a9f6d3f" ], "author": { "name": "Jille Timmermans", "email": "jille@hexon.nl", "time": "Wed Oct 21 15:58:24 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 21 15:58:24 2026" }, "message": "zstd: Don\u0027t retain pointers to decoded inputs in DecodeAll (#1273)\n\nI have a global *Decoder that\u0027s used for all decoding. The compressed\ninput is part of huge memory allocations which are kept alive with these\npointers.\n\nWritten by gpt-5.6-sol" }, { "commit": "35c6f98a5cb955b838d313596dd86c882a9f6d3f", "tree": "40312d004e768021fd76bc850e8e3d7e646dafe9", "parents": [ "c2cacb4dcf9fee5ca36f095ff2bc0314ead739ae" ], "author": { "name": "Andreï Basdereff", "email": "abasdereff@gmail.com", "time": "Wed Oct 21 12:41:03 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 21 12:41:03 2026" }, "message": "gzhttp: flush and hijack through http.ResponseController (#1272)" }, { "commit": "c2cacb4dcf9fee5ca36f095ff2bc0314ead739ae", "tree": "e04381e5898eee041be89410696ea3e9f17a297b", "parents": [ "6ce0477ba82ea7c89aa643767c98f2f589f7a2f9" ], "author": { "name": "Yang Bai", "email": "yangbai2026@gmail.com", "time": "Mon Oct 19 15:13:49 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 19 15:13:49 2026" }, "message": "gzip: backport two standard library fixes (truncated header EOF, Write count) (#1270)\n\n* gzip: return io.ErrUnexpectedEOF for a header truncated in FNAME or FCOMMENT\n\nreadHeader passed readString\u0027s io.EOF through unchanged, so a header cut\noff inside the file name or comment looked like a clean end of input:\nNewReader returned io.EOF, as for an empty stream, and in multistream\nmode Read and WriteTo returned the earlier members\u0027 data with no error.\nThe other header fields already use noEOF.\n\nBackport of https://go.dev/cl/389034 (golang/go#51417).\n\n* gzip: do not count header bytes in Writer.Write\n\nWhen the underlying writer failed while the header was being written,\nWrite returned the number of header bytes written, which can be more\nthan len(p). io.Copy from a strings.Reader or bytes.Reader then panics\n(\"invalid WriteString count\"), and bufio.Writer\u0027s buffered count goes\nnegative. Return 0 until p itself is written.\n\nBackport of https://go.dev/cl/103861 (golang/go#24625).\n\n* gzip: return io.ErrShortWrite for a short header write\n\n* gzip: check for short writes in the header helpers\n\n---------\n\nCo-authored-by: ybai08 \u003c271969568+ybai08@users.noreply.github.com\u003e" }, { "commit": "6ce0477ba82ea7c89aa643767c98f2f589f7a2f9", "tree": "66a1c54082bf89e9b0c495223fc2a79f4f0bda61", "parents": [ "2033f3fd935ffe2fc68fecfa9033619a6ca367bf" ], "author": { "name": "Yang Bai", "email": "yangbai2026@gmail.com", "time": "Mon Oct 19 11:31:06 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 19 11:31:06 2026" }, "message": "zip: accept 0xFFFFFFFF sizes and offsets without a Zip64 extra (#1267)\n\nInfo-ZIP writes a 32-bit compressed size or local header offset of\nexactly 0xFFFFFFFF as a literal value and adds no Zip64 extra field,\nsince the value fits. The reader returned ErrFormat for such entries,\nso a stored file of 4GiB-1 bytes, or an entry whose header starts at\noffset 0xFFFFFFFF, could not be opened.\n\nUse the 32-bit value when no Zip64 extra overrides it, as Go 1.27\u0027s\narchive/zip does. The two goldens come from the standard library\u0027s\ntestdata/zip64.\n\nCo-authored-by: ybai08 \u003c271969568+ybai08@users.noreply.github.com\u003e" }, { "commit": "2033f3fd935ffe2fc68fecfa9033619a6ca367bf", "tree": "88962781669c70a79f4e06c470c8744ee6e3c9da", "parents": [ "920bb4bcc0188e341ce7655cd2bc96b164cd6c56" ], "author": { "name": "Spicy Shark", "email": "321699413+spicysharkco@users.noreply.github.com", "time": "Mon Oct 19 10:13:32 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 19 10:13:32 2026" }, "message": "gzip: write and read a zero MTIME for a zero ModTime (#1268)\n\nHeader.ModTime defaults to the zero time.Time. Writer encoded it as\nuint32(time.Time{}.Unix()), which is 0x886e0900, so a stream written\nwithout a ModTime carried a modification time in the year 2042. Reader\ndid the reverse: an MTIME of 0 was reported as the Unix epoch instead\nof the zero time.\n\nRFC 1952 section 2.3.1 says an MTIME of 0 means the time is not set.\nGo 1.8 changed compress/gzip to write 0 for a zero ModTime and to read\n0 as a zero ModTime. Do the same here.\n\nTestEmpty now also checks that the header read back from a stream\nwritten without a ModTime is Header{OS: 255}, as the standard library\ntest does." }, { "commit": "920bb4bcc0188e341ce7655cd2bc96b164cd6c56", "tree": "ff6222e50771158c5ae9df0ccb7cb45463ad8841", "parents": [ "6de8e38778cd78c5dcebf645a29e017ab2e84b6d" ], "author": { "name": "Grivn", "email": "grivn.wang@gmail.com", "time": "Fri Oct 16 10:41:45 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 16 10:41:45 2026" }, "message": "zstd: preserve stream state when resetting options (#1266)\n\n(1) Preserve the current stream while resetting encoder options (e470023b55bf).\n\nWith concurrent blocks enabled, adding a dictionary in `ResetWithOptions` disables the job shutdown path before the old jobs finish. Those jobs can then write compressed blocks to the new destination, producing a stream that fails to decode with `invalid input: magic number mismatch`.\n\nAn option error can also leave partial changes behind. For example, `WithEncoderCRC(false)` followed by a rejected concurrency change removes the checksum from a frame whose header still requires it.\n\nApply options to a copy first, then finish pending compression and writes before publishing the new options or replacing the encoder. Failed option validation leaves the current stream unchanged.\n\nRegression tests:\n\n- `zstd/enc_jobs_test.go`: block the old destination while resetting, then check worker shutdown and round-trip the new stream, with and without parallel jobs and a dictionary.\n- `zstd/encoder_options_test.go`: reject a concurrency change after a CRC change, then finish and decode the original stream.\n\nBoth failure cases reproduce before the fix. Validation on macOS arm64:\n\n- Targeted race tests on Go 1.25.4, 1.26.8 and 1.27.2; 20 repeated runs on Go 1.27.2.\n- Go 1.27.2: `go test ./...`, `go test -tags\u003dnoasm ./...`, and `go test -short -race -tags\u003dnounsafe ./...` with `-cpu\u003d1` and `-cpu\u003d4`.\n- `go vet ./...`, `go fix -diff ./...`, `gofmt` and `git diff --check`.\n\n\n\u003c!-- This is an auto-generated comment: release notes by coderabbit.ai --\u003e\n\n## Summary by CodeRabbit\n\n* **Bug Fixes**\n * Resetting a stream now waits for pending writes and encoding work to finish before switching configurations, preventing data from the old stream from leaking into the new one.\n * If reset options are invalid, the existing stream remains usable and its data is preserved.\n\n\u003c!-- end of auto-generated comment: release notes by coderabbit.ai --\u003e" }, { "commit": "6de8e38778cd78c5dcebf645a29e017ab2e84b6d", "tree": "63d722c98de9cb4dde36cc827301875f69ffb069", "parents": [ "b43b8dcc07965f93d76839638cf7bd8bfaae86de" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Wed Oct 14 12:02:59 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 14 12:02:59 2026" }, "message": "flate: Fix stateless EOF on large writes. (#1265)\n\nFixes `StatelessDeflate(w, data, true, dict)` with non-empty data long enough to be split into more than one internal block (more than 32 KB, or about 24 KB with a dictionary) missing EOF blocks.\n\nReported by OSS-fuzz" }, { "commit": "b43b8dcc07965f93d76839638cf7bd8bfaae86de", "tree": "1d1a7f648f0b7d0ff61de6e1cf32a791db7c00b2", "parents": [ "f9c34280cd4de53e41ed3cf61c11efb6a1370794" ], "author": { "name": "Shai", "email": "66793913+Shaisolaris@users.noreply.github.com", "time": "Wed Oct 14 12:02:40 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 14 12:02:40 2026" }, "message": "gzhttp: let \"*\" in Accept-Encoding select gzip or zstd (#1263)\n\n* gzhttp: let \"*\" in Accept-Encoding select gzip or zstd\n\nA request with \"Accept-Encoding: *\" was served uncompressed, because\nonly entries naming gzip or zstd were looked at. RFC 9110 section\n12.5.3 says \"*\" matches any coding not listed in the field.\n\nUse the quality of \"*\" when the coding is not named. An entry that\nnames the coding still takes precedence, in either order, so\n\"*, zstd;q\u003d0\" selects gzip. parseEncodingGzip now shares\nparseEncodingQValue.\n\n* gzhttp: put Accept-Encoding \"*\" handling behind AcceptWildcards option\n\n---------\n\nCo-authored-by: Shai \u003cshai@Shais.local\u003e" }, { "commit": "f9c34280cd4de53e41ed3cf61c11efb6a1370794", "tree": "57b8d128a0a3fac69c4cd9327d2a9d3bf6c111c7", "parents": [ "1974577b19ef1edd312803ff61b11cf86f870b33" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Tue Oct 13 08:35:39 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 13 08:35:39 2026" }, "message": "flate: build unrestricted Huffman trees before limiting lengths (#1219)\n\n* flate: build unrestricted Huffman trees before limiting lengths\n\nBackport of https://go.dev/cl/829569 (golang/go#81398).\n\nbitCounts computes optimal length-limited code lengths directly, but it\ncosts O(symbols * maxBits) with poorly predicted branches, and for the\nliteral alphabet it accounts for a large share of Huffman-only encoding\ntime.\n\nInstead, first build an ordinary Huffman tree over the frequency-sorted\nsymbols using the linear time two-queue method and count the leaf\ndepths. The result is optimal and, for the block sizes used here,\nalmost never exceeds the 15-bit (or 7-bit, for the code length\nalphabet) limit. Only when it does, fall back to bitCounts.\n\nThis adds ~2.3KB of scratch space to each huffmanEncoder.\n\nNOTE: this is the only CL in the series that changes the output. The\ngenerated code lengths can differ when several optimal trees exist, so\nthe golden test files are regenerated. Measured over e.txt, Tom Sawyer,\ngettysburg.txt, pi.txt, sharnd.out and the huffman-* test inputs at\nevery level, total compressed size goes from 3748412 to 3748348 bytes\n(-0.0017%); the worst single regression is +1 byte on a 768 byte file\n(gettysburg.txt at level 4) and the best improvement is -3 bytes on a\n240 byte file.\n\nAdd tests checking that the generated codes are complete prefix codes\nwithin the length limit, and that they are as short as those from the\nlength-limited construction whenever the tree construction is used.\n\n```\ngoos: windows\ngoarch: amd64\npkg: github.com/klauspost/compress/flate\ncpu: AMD Ryzen 9 9950X 16-Core Processor\n │ master │ cl-829569 │\n │ sec/op │ sec/op vs base │\nPR/Digits/Huffman-32 804.3µ ± 1% 740.8µ ± 1% -7.89% (p\u003d0.000 n\u003d15)\nPR/Digits/L1-32 4.731m ± 1% 4.664m ± 1% -1.42% (p\u003d0.009 n\u003d15)\nPR/Digits/L4-32 5.518m ± 2% 5.576m ± 2% ~ (p\u003d0.098 n\u003d15)\nPR/Digits/L6-32 7.879m ± 5% 7.926m ± 1% ~ (p\u003d0.775 n\u003d15)\nPR/Digits/L7-32 14.54m ± 3% 14.76m ± 2% ~ (p\u003d0.148 n\u003d15)\nPR/Digits/L9-32 32.37m ± 3% 32.38m ± 1% ~ (p\u003d0.389 n\u003d15)\nPR/Twain/Huffman-32 1050.6µ ± 3% 832.4µ ± 3% -20.77% (p\u003d0.000 n\u003d15)\nPR/Twain/L1-32 4.726m ± 2% 4.613m ± 2% -2.40% (p\u003d0.000 n\u003d15)\nPR/Twain/L4-32 5.944m ± 1% 5.884m ± 1% -1.00% (p\u003d0.037 n\u003d15)\nPR/Twain/L6-32 7.773m ± 3% 7.713m ± 2% ~ (p\u003d0.174 n\u003d15)\nPR/Twain/L7-32 17.70m ± 2% 17.65m ± 2% ~ (p\u003d0.838 n\u003d15)\nPR/Twain/L9-32 36.68m ± 1% 36.68m ± 1% ~ (p\u003d0.967 n\u003d15)\ngeomean 6.898m 6.704m -2.81%\n\n │ B/s │ B/s vs base │\nPR/Digits/Huffman-32 1.158Gi ± 1% 1.257Gi ± 1% +8.57% (p\u003d0.000 n\u003d15)\nPR/Digits/L1-32 201.6Mi ± 1% 204.5Mi ± 1% +1.44% (p\u003d0.008 n\u003d15)\nPR/Digits/L4-32 172.8Mi ± 2% 171.0Mi ± 2% ~ (p\u003d0.095 n\u003d15)\nPR/Digits/L6-32 121.0Mi ± 5% 120.3Mi ± 1% ~ (p\u003d0.767 n\u003d15)\nPR/Digits/L7-32 65.58Mi ± 3% 64.60Mi ± 2% ~ (p\u003d0.148 n\u003d15)\nPR/Digits/L9-32 29.47Mi ± 3% 29.45Mi ± 1% ~ (p\u003d0.389 n\u003d15)\nPR/Twain/Huffman-32 907.7Mi ± 3% 1145.6Mi ± 3% +26.21% (p\u003d0.000 n\u003d15)\nPR/Twain/L1-32 201.8Mi ± 2% 206.7Mi ± 2% +2.46% (p\u003d0.000 n\u003d15)\nPR/Twain/L4-32 160.4Mi ± 1% 162.1Mi ± 1% +1.01% (p\u003d0.037 n\u003d15)\nPR/Twain/L6-32 122.7Mi ± 3% 123.6Mi ± 1% ~ (p\u003d0.164 n\u003d15)\nPR/Twain/L7-32 53.87Mi ± 2% 54.03Mi ± 2% ~ (p\u003d0.847 n\u003d15)\nPR/Twain/L9-32 26.00Mi ± 1% 26.01Mi ± 1% ~ (p\u003d0.976 n\u003d15)\ngeomean 138.3Mi 142.3Mi +2.89%\n```\n\nOriginal-Author: Alex Gaynor \u003calex.gaynor@gmail.com\u003e\n\nCo-Authored-By: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* Update tests.\n\n---------\n\nCo-authored-by: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "1974577b19ef1edd312803ff61b11cf86f870b33", "tree": "9301ce3de4394aba4a505714a1501056fa4b05e4", "parents": [ "dd49be439f65a7a8a55c4e0a52f8b387741fb808" ], "author": { "name": "Liz Fong-Jones", "email": "elizabeth@ctyalcove.org", "time": "Mon Oct 12 12:29:55 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 12 12:29:55 2026" }, "message": "zstd, huff0: faster pure-Go decoders (#1256)\n\n* huff0: refill the Go 4X streams without a branch\n\nThe pure-Go Decompress4X loop for table logs 9-11 kept its four\nreaders in an array on the stack and refilled each with a\nconditional 32-bit load every two symbols.\n\nEach stream\u0027s state is now held by value, and refilled once per four\nsymbols with one load and no branch: value \u003d Load64(in, off) \u003c\u003c bits\nafter moving off back by the whole bytes consumed, the refill the\n#1210 1X assembly uses. 57 bits cover four symbols of up to 11 bits,\nand the loop runs while every stream has 8 bytes left, so a refill\nnever moves before the start of its stream.\n\nTable logs up to 8 keep the 8-bit loops: on Zen 3 routing them through\nthis loop was 9% slower.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: copy overlapping matches in doubling spans in Go\n\nThe Go decoders copied matches that overlap their own output one byte\nat a time. On log data that is a large share of the output (half of\nthe bytes of apache.log at -19), and the speed of such a one-byte loop\ndepends on where it lands relative to a 64-byte boundary: once the\nloops around it changed, the multithreaded apache decode was 16-25%\nslower on Zen 3/4 at two of the four 32-byte placements of\nexecuteSimple and unchanged at the other two.\n\noverlapCopy repeats the bytes already written, doubling the span with\neach copy, so a match takes about log2(length/offset) copies. It is\nused by decodeSync, execute and the Go executeSimple.\n\nTestOverlapCopy (seqdec_test.go) checks it against a bytewise copy.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: copy matches and read sequences without calls in the Go decoders\n\nThe pure-Go sequence loops (decodeSync, the two-pass decode, execute\nand executeSimple) appended or copied every literal run and match with\na memmove call, and read and wrote the bit reader, repeat offsets and\nliterals through pointers. Go has no callee-saved registers, so the\ncalls forced every loop variable through the stack on each sequence;\nkeeping the state in locals alone made decodeSync 5% slower. Removing\nthe calls is what lets the locals stay in registers.\n\n- Matches that copy from earlier output they do not overlap, from\n inside the history, or from inside the dictionary are copied in the\n loop. matchSource picks the source and checks the bounds. Sequences\n with at most 16 literal and 16 match bytes and 16 bytes of spare\n capacity use two fixed 16-byte copies, the pure-Go form of the\n assembly\u0027s short-sequence path; longer ones use two copy calls.\n Everything else (overlap, a match spanning history and output,\n growth, errors) goes through executeSeq, the old decodeSync code\n moved out of the loop.\n- The bit reader\u0027s changing fields move into a 24-byte bitReadState\n with by-value methods (bitReader is too big for the compiler to keep\n in registers), held in locals with the repeat offsets and literals\n and written back on success. Every sequence uses the inline read, so\n sequences in the last bytes of the bitstream no longer go through\n next(); next and fillFast are removed.\n\nTests (seqdec_test.go): TestMatchSource checks each bound of\nmatchSource; TestExecuteShortSequences checks execute and executeSimple\nagainst a bytewise reference at every spare capacity from 0 to 32\nbytes, with and without a dictionary, and that an offset before the\ndictionary is still an error; TestDecoderShortSequenceCopies gains\noffsets 1..40 and runs the Go decodeSync.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "dd49be439f65a7a8a55c4e0a52f8b387741fb808", "tree": "8cca69fbe93d26a40a7fa2745609deb7bf78dfa6", "parents": [ "b946d41b75cee10ea255ced85179e5224748a023" ], "author": { "name": "Liz Fong-Jones", "email": "elizabeth@ctyalcove.org", "time": "Mon Oct 12 12:19:22 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 12 12:19:22 2026" }, "message": "zstd, huff0, s2: bump the avo arm64 lowering; keep zstd stack slots in registers (#1248)\n\n* zstd, huff0, s2: bump the avo arm64 lowering\n\nMove all three generators to the same honeycombio/avo commit. zstd and\nhuff0 regenerate byte-identically. s2 was on an older commit and picks\nup the shift source fold: 94 register copies into a shift become\nthree-operand LSL/LSR.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: keep arm64 sequence decoder stack slots in registers\n\nGenerate the arm64 sequence decoders with -arm64-promote-stack-slots:\nthe frame slots the amd64 program uses for want of registers live in\nspare arm64 registers instead, and the loads and stores around them are\nfolded away. amd64 output is unchanged.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* zstd, huff0, s2: move the avo pin to the current lowering\n\nThe new commit drops a jump to the next label across PCALIGN padding.\nNone of these generators emits that shape, so every generated file is\nunchanged; this keeps the pin on the lowering\u0027s current trunk.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "b946d41b75cee10ea255ced85179e5224748a023", "tree": "bfab33ff2ca2f0bd37a3d84c82bbb9b8566a8abf", "parents": [ "748550b77336a1720b51f574d344ab98422c5e2a" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Oct 12 09:19:21 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 12 09:19:21 2026" }, "message": "Revert #1255 (#1261)\n\nRevert \"gzhttp: do not compress 206 responses without Content-Range, read Accept-Encoding across field lines (#1255)\"\n\nTestGzipHandlerMultipleRanges test is unstable.\n\nThis reverts commit d189b4c2eb01cb5bd3d7c4a4b7e8ac9b02e88281." }, { "commit": "748550b77336a1720b51f574d344ab98422c5e2a", "tree": "34e3678a794c3c3bd7c94c1a5110704ae98da294", "parents": [ "dd54d8695696369d9504c9661f3e476a8a69a84a" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Oct 12 08:35:05 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 12 08:35:05 2026" }, "message": "Merge commit from fork\n\n* fse: reject normalized counts that do not fit in int16\n\nreadNCount accepts tableLog 15, where one symbol can be given all 1\u003c\u003c15\nstates. That count was stored in norm as int16 -32768, so buildDtable\nspread no symbols, the position check still passed, and the decoding\ntable kept the cells of the previous table decoded on the same Scratch.\nCells with nbBits 0 then reached nextFast, which read 16 junk bits into\nthe state and indexed past the table.\n\nzstd reaches this through huff0.ReadTable, whose fse Scratch is kept\nbetween blocks and comes back from huffDecoderPool in the next frame, so\ncrafted input panics DecodeAll and the streaming decoder. The streaming\ndecoder panics on its own goroutine, where callers cannot recover.\n\nA symbol filling the whole table cannot be decoded at any tableLog, so\nreject the count before it is truncated. crash4, crash5 and crash6 in\nTestDecompress are this overflow; with a fresh Scratch they failed\nfurther on, crash6 in the newState guard added by #151.\n\nTests: TestDecompressCountOverflow (fse/fse_test.go) and\nTestDecoderHuffWeightsTableLog (zstd/decoder_test.go) panic without\nthe fix.\n\n* huff0: limit the FSE tableLog of Huffman weights to 6\n\nRFC 8878 limits the accuracy log of FSE-compressed Huffman weights to 6\nand the reference decoder rejects larger tables, so no valid stream uses\nthem. Reject them in ReadTable before decoding. A weight table can then\nno longer make fse build a 1\u003c\u003c15 entry table for at most 255 weights.\n\nThe only entry in decompress1x_regression.zip stored its weights with\ntableLog 15. It is re-encoded with raw weights: the Huffman table and\npayload are unchanged, and Decompress1X still fails with \"maximum output\nsize exceeded\".\n\nTest: TestReadTableWeightsTableLog (huff0/decompress_test.go).\n\n* fse: check accepted counts in FuzzDecompress\n\nFuzzDecompress only looked for panics. The int16 count overflow returned\nan error on a fresh Scratch, so inputs that hit it, one of them in the\nseed corpus, never failed. Run readNCount on every input and require the\naccepted counts to fill the table, which buildDtable relies on. Without\nthe readNCount fix this fails on seed#2 of fse_decompress.zip.\n\nvalidateNorm added -v for every negative count, which let a wrapped\n-32768 pass as 32768. Count a negative entry as one cell, as buildDtable\ndoes for -1. Its only callers are FuzzCompress and a disabled check in\nCompress.\n\nTest: FuzzDecompress (fse/fuzz_test.go)." }, { "commit": "dd54d8695696369d9504c9661f3e476a8a69a84a", "tree": "94aa8fed586f1303185c6a8484414b1630aaa63a", "parents": [ "d189b4c2eb01cb5bd3d7c4a4b7e8ac9b02e88281" ], "author": { "name": "Liz Fong-Jones", "email": "elizabeth@ctyalcove.org", "time": "Thu Oct 08 07:50:19 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 08 07:50:19 2026" }, "message": "huff0: reuse Scratch across inputs in the fuzzers (#1257)\n\n* huff0: do not pin each Scratch with its own buffer pool\n\nScratch embedded the sync.Pool holding the decoders\u0027 [4][256]byte\nbuffers. The runtime keeps every pool it has seen until the second GC\nafter its first use, and a pool inside a struct keeps the whole struct\nreachable, so a discarded Scratch and its 256 KiB Decompress1X output\nbuffer stayed live for two GC cycles. Creating a Scratch per block, as\nFuzzCompress does, grew the heap until the fuzz worker was OOM-killed\nunder noasm.\n\nThe buffers carry no per-Scratch state; use one package-level pool.\n\nTestDecoderDoesNotPinScratch (decompress_test.go) fails on the old code\nin both builds and passes now.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* huff0: reuse Scratch in the fuzzers instead of moving the buffer pool\n\nPer review, the per-block Scratch in the fuzz harnesses is the problem,\nnot the library. Restore the per-Scratch decoder pool and have\nFuzzCompress and FuzzDecompress1x keep their Scratches across inputs,\nas zstd does. FuzzCompress clears prevTable per input so the 1X call\nstill starts without a table to reuse.\n\ngo test -tags noasm -fuzz FuzzCompress ./huff0 for 5 minutes: peak\nworker RSS 275 MB, no OOM (previously 2.5-3.1 GB within a minute).\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* Update huff0/fuzz_test.go\n\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "d189b4c2eb01cb5bd3d7c4a4b7e8ac9b02e88281", "tree": "f1075b43cf32a4acb56f45b4990ec77894748d90", "parents": [ "6b9199594fc567d83deacdcea4d0130df249109a" ], "author": { "name": "Suliman Abdulrazzaq", "email": "144490671+SulimanAbdulrazzaq@users.noreply.github.com", "time": "Sun Oct 04 11:55:17 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 04 11:55:17 2026" }, "message": "gzhttp: do not compress 206 responses without Content-Range, read Accept-Encoding across field lines (#1255)\n\n* gzhttp: do not compress 206 responses without Content-Range\n\nA single range is answered with a Content-Range header, and responses that have\none are already passed through uncompressed. The multipart/byteranges answer to\na request for several ranges (as written by http.ServeContent) has no\nContent-Range header, so it was compressed: the client got a 206 with\n\"Content-Encoding: gzip\" and Accept-Ranges removed, although the ranges in its\nparts refer to the uncompressed representation.\n\nTreat the 206 status like the Content-Range header in Write, Flush and Close.\n\n* gzhttp: read Accept-Encoding across field lines\n\nA client may split one Accept-Encoding list over several field lines, which\nmeans the same as the joined comma list (RFC 9110, section 5.3). selectEncoding\nand acceptsGzip used Header.Get, which only returns the first line, so\n\"Accept-Encoding: deflate\" followed by \"Accept-Encoding: gzip\" was answered\nwithout compression, and a later q-value such as \"zstd;q\u003d0\" was ignored.\n\nJoin the field lines before parsing, the same way the request Content-Encoding\nis already read, and share the helper between the two.\n\n* gzhttp: remove the unused acceptsGzip" }, { "commit": "6b9199594fc567d83deacdcea4d0130df249109a", "tree": "24e2021199e8a485ac2a201d4105b56dbc307d98", "parents": [ "402c3a567166a64c1af49735f7f5f40178e48ebd" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Fri Oct 02 18:41:58 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 02 18:41:58 2026" }, "message": "zstd, huff0: keep the encoder bit writers in registers (#1245)\n\n* zstd: add BenchmarkEncoderCorpus\n\nEncodes each file in the directory named by ZSTD_CORPUS with EncodeAll on\none goroutine at every level, CRC off, and reports the compression ratio\nalongside speed. The encoder counterpart of BenchmarkCorpus, for use with\nthe real-world corpora at https://klauspost.com/files/compress/.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: keep the sequence bit writer in registers\n\nblockEnc.encode wrote every FSE state and extra-bits field through the\nbitWriter held in the block (wr :\u003d \u0026b.wr), so the container and bit count\nwere loaded and stored around each addBits, and every flush32 carried an\nappend with a growslice path. On arm64 that was ~20% of SpeedFastest\nencode time.\n\nKeep the container, bit count, output position and the three states in\nlocals, and flush the way libzstd does: store 8 bytes unconditionally and\nadvance by whole bytes. Capacity is checked once per run of sequences\n(12 bytes per sequence plus the store\u0027s slack), so the buffer only grows\nwhen the output would have.\n\nThe bitstream is unchanged: output is byte-identical to before for every\nlevel on the real-world corpora. The rare split for sequences with more\nthan 56 extra bits (offsets over 2^24 with long literal and match\nlengths) is covered by TestEncoderLongOffsetSequence.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* huff0: keep the compress bit writer in registers\n\ncompress1xDo\u0027s bitWriter lived on the stack because flush32 appends with\na growslice path, so the container and bit count were reloaded and\nstored for every four symbols. Keep them in locals and flush with an\nunconditional 8-byte store, checking capacity once per run of symbols.\n\nAfter such a flush at most 7 bits remain, and four symbols add at most\n4*tableLogMax \u003d 44 bits, so the separate two-symbol path for table logs\nabove 8 is no longer needed. Output is byte-identical.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* fixup! zstd: keep the sequence bit writer in registers\n\n* fixup! huff0: keep the compress bit writer in registers\n\n* zstd, huff0: mask the bit writer shifts with regmask\n\nFrom review: use the regmask constants instead of literal masks, so 386\ndoes not pay for masking it does not need. amd64 and arm64 code is\nunchanged.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "402c3a567166a64c1af49735f7f5f40178e48ebd", "tree": "cd0b4507b2079c9e0cd9c67f9e987ad7c01f863b", "parents": [ "29f5f81f681bb2af88ca81956346288d677a648e" ], "author": { "name": "Raghuvardhan-Reddy78", "email": "raghuvardhan@labs.digiscrypt.com", "time": "Fri Oct 02 14:41:11 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 02 14:41:11 2026" }, "message": "s2: Reject short index chunk in RemoveIndexHeaders (#1254)" }, { "commit": "29f5f81f681bb2af88ca81956346288d677a648e", "tree": "c524e9a1da34276d6e43ea59044d3cd7e8c27c1d", "parents": [ "9b8ba6149408b157131b52e424ef97470ced32f7" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Fri Oct 02 09:25:25 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 02 09:25:25 2026" }, "message": "build(deps): bump the github-actions group with 4 updates (#1252)\n\nBumps the github-actions group with 4 updates: [github/codeql-action/init](https://github.com/github/codeql-action), [github/codeql-action/autobuild](https://github.com/github/codeql-action), [github/codeql-action/analyze](https://github.com/github/codeql-action) and [github/codeql-action](https://github.com/github/codeql-action).\n\n\nUpdates `github/codeql-action/init` from 4.37.9 to 4.38.2\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/cdf488f595d80d6e07e03d4674febd5ab45fa938...2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2)\n\nUpdates `github/codeql-action/autobuild` from 4.37.9 to 4.38.2\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/cdf488f595d80d6e07e03d4674febd5ab45fa938...2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2)\n\nUpdates `github/codeql-action/analyze` from 4.37.9 to 4.38.2\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/cdf488f595d80d6e07e03d4674febd5ab45fa938...2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2)\n\nUpdates `github/codeql-action` from 4.37.9 to 4.38.2\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/v4.37.9...v4.38.2)\n\n---\nupdated-dependencies:\n- dependency-name: github/codeql-action/init\n dependency-version: 4.38.2\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: github-actions\n- dependency-name: github/codeql-action/autobuild\n dependency-version: 4.38.2\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: github-actions\n- dependency-name: github/codeql-action/analyze\n dependency-version: 4.38.2\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: github-actions\n- dependency-name: github/codeql-action\n dependency-version: 4.38.2\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: github-actions\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "9b8ba6149408b157131b52e424ef97470ced32f7", "tree": "736587896d5b2bf6d2c686ec163b6a133142c400", "parents": [ "da4df9cb2b3ca4fe6b813acd710711fd7b5c0dbc" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Wed Sep 30 08:46:46 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 30 08:46:46 2026" }, "message": "dict: Stricter input validation (#1250)\n\nReject unusable inputs to dictionary builder." }, { "commit": "da4df9cb2b3ca4fe6b813acd710711fd7b5c0dbc", "tree": "f2f62399fd6aaebe0be35390a7967cad20e52eab", "parents": [ "1971eecb678dfb1ff77c4e5b55787dbc57e9edda" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Wed Sep 30 07:32:21 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 30 07:32:21 2026" }, "message": "zstd: decode at full speed into exact-size destinations (#1246)\n\n* zstd: bound the decodeSync slack check by the block, not the frame\n\nWhen the frame content size is known, useSafeDecodeSync required the\noutput buffer to have compressedBlockOverAlloc bytes of slack beyond the\nwhole rest of the frame (maxSyncLen). A destination sized exactly to the\nframe therefore ran the bounds-exact copies for every block, although\nthe extended copies only need the slack past the block being decoded,\nwhich produces at most min(maxSyncLen, maxBlockSize) bytes. The assembly\nstill checks each sequence against cap-compressedBlockOverAlloc, so a\nframe that lies about its size fails as before.\n\nTest in zstd/seqdec_test.go.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: don\u0027t reallocate the output for missing slack in executeSimple\n\nexecuteSimple grew s.out whenever the block plus compressedBlockOverAlloc\ndid not fit. With a destination sized exactly to the frame that happens\non the last block, and the growth copied the whole decoded frame into a\nnew buffer about twice its size, so DecodeAll also returned a different\nbacking array than the caller supplied. Grow only when the block itself\ndoes not fit, and use the bounds-exact copies when the output lacks the\nslack.\n\nThe two-pass path that calls executeSimple from DecodeAll runs on arm64\nfor windows of 1 MiB and up.\n\nTest in zstd/decoder_test.go.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "1971eecb678dfb1ff77c4e5b55787dbc57e9edda", "tree": "312765823c29ff04f78d2c7360b5335d89d79153", "parents": [ "9abcafbf831ecede1741e14681503a96531783dc" ], "author": { "name": "Raghuvardhan-Reddy78", "email": "raghuvardhan@labs.digiscrypt.com", "time": "Wed Sep 30 07:26:59 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 30 07:26:59 2026" }, "message": "reject zip entries named \".\" in ReadDir (#1249)" }, { "commit": "9abcafbf831ecede1741e14681503a96531783dc", "tree": "c3bc8b973af43a8b0cb68659bba9566209241957", "parents": [ "4e2dcf8a72b0d874708466b3f0e3741c00353b32" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Wed Sep 30 07:13:23 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 30 07:13:23 2026" }, "message": "s2: fix LZ4 block conversion writing past dst (#1244)\n\n* s2: stop LZ4 block conversion from writing past dst\n\nThe literal bound in the S2-output converters ignored the 1-5 byte\nliteral header, leaving too little room for a following copy with a\n5-byte repeat (up to 8 bytes). The assembly wrote a few bytes past\ncap(dst) before returning ErrDstTooSmall; the pure-Go path panicked.\nMatches over ~16.8 MB chained 5-byte repeat chunks with no check, so the\noverrun grew with the match length.\n\nTighten the existing literal bound so a copy always fits, and check the\nlimit only before each extra repeat chunk. The Snappy-output converters\nalready checked every chunk.\n\nAdd an unexported noAsm field so tests reach the pure-Go path without\nbuild tags.\n\n* s2: fuzz LZ4 conversion with under- and oversized dst\n\nFuzzLZ4Block gave the converter 2*len(data)+4096 bytes, which S2 output\nnever comes close to, and asm writes past cap(dst) are invisible without\na guard. Convert again with every variant into need +/- a deterministic\namount up to len(data), with 16 guard bytes after cap, and check the\nguard and that the result agrees with the large-dst run.\n\nOn the unfixed code this found an overrun within minutes; the input is\nadded to the seed corpus.\n\n* s2: fuzz LZ4 assembly only when available\n\nFuzzLZ4Block covers the assembly and pure-Go converters in one run, so\nadd the assembly variants only when hasAsm is set, and run the fuzzer in\nCI without the noasm,nounsafe pass, on both amd64 and arm64." }, { "commit": "4e2dcf8a72b0d874708466b3f0e3741c00353b32", "tree": "39f677897bb81ca2e77ce67284b8780f9b46e05b", "parents": [ "699c61827bcd414e21107724b93c00b1f56ab6b3" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Tue Sep 29 15:21:37 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 29 15:21:37 2026" }, "message": "maint(all): drop legacy build tags and old-Go workarounds (#1242)\n\n* all: drop the appengine build tag and legacy +build lines\n\nThe appengine tag selected pure-Go code for the classic App Engine\nruntime, which no longer exists; noasm and nounsafe cover that need.\nGo has read //go:build since 1.17, so the // +build lines are dead too.\n\nThe zstd and s2 generators also added !noasm twice. Their constraints\nnow match huff0\u0027s single ConstraintExpr, and s2 only adds the\n(amd64 || arm64) term to the run that writes the shared stubs, so the\narm64 file no longer reads arm64 \u0026\u0026 ((amd64 || arm64) \u0026\u0026 ...).\n\nThe compiled file set (go list GoFiles, SFiles, TestGoFiles) is\nidentical to before for amd64, arm64, 386, riscv64 and ppc64le with no\ntags, noasm, nounsafe, noasm,nounsafe and purego. Regenerated assembly\ndiffers only in its header.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* all: drop build constraints every supported Go version satisfies\n\ngo.mod requires Go 1.25, so the go1.16, go1.18 and go1.24 constraints\nalways hold. zstd/simple_go124.go and its test only existed for the\nconstraint and become simple.go and simple_test.go.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* zip, gzhttp: resolve Go-version TODOs\n\nzip carried copies of fs.FormatDirEntry and fs.FormatFileInfo until Go\n1.20 support was dropped; call the identical io/fs functions, as\narchive/zip does. The gzhttp test builds its request with\nhttptest.NewRequest, as its Go 1.6-era TODO asked, and a comment about\npre-1.20 builds goes.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* all: replace stand-ins kept for old Go versions\n\nTestEncodeTo sat in its own file only because of the go1.24 tag; it\nmoves into encoder_test.go as its TODO asked. The flate writer tests\ncopied io.CopyBuffer to support Go versions before 1.5; they now call\nit with the reader and writer wrapped so it still copies through the\ngiven buffer. io/ioutil goes from s2 and the gzhttp examples.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "699c61827bcd414e21107724b93c00b1f56ab6b3", "tree": "3748018c1339975aecd90b4dc1e26dc124546893", "parents": [ "5d45bca0cdb5f144a3f638b31013b170256cfa1d" ], "author": { "name": "Raghuvardhan-Reddy78", "email": "raghuvardhan@labs.digiscrypt.com", "time": "Tue Sep 29 09:49:10 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 29 09:49:10 2026" }, "message": "zstd: reject snappy copy shorter than minimum match (#1235)\n\n* zstd: reject snappy copy shorter than minimum match\n* zstd: drop stray debug println from snappy copy guard\n* go fix\n* zstd: deprecate SnappyConverter\n\n---------\n\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "5d45bca0cdb5f144a3f638b31013b170256cfa1d", "tree": "13ca2ca0c6d948b6e1f8e7d9214ee290fa8bfa82", "parents": [ "951333e537ae02581cf9ff53780e874bcc0aafb0" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Tue Sep 29 07:46:54 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 29 07:46:54 2026" }, "message": "zstd, s2, flate: smoke-test Silesia in CI, add real-corpus benchmarks (#1237)\n\n* zstd: smoke-test Silesia in CI\n\ntestdata/fetch_silesia.sh downloads silesia.tar.zst from\nklauspost.com/files/compress/, checks its SHA-256, and decompresses it to\ntestdata/silesia.tar, whose SHA-256 it also checks. This is the tar the\nexisting Silesia tests already expect. With -cli it also makes two\nencodings with the zstd CLI: -19 --long\u003d27, and four concatenated -3 frames.\n\nNew tests, skipped without testdata/silesia.tar:\n- TestSilesiaRoundTrip: each level through Writer/Reader and\n EncodeAll/DecodeAll (encoder_test.go).\n- TestSilesiaCLIDecode: the encodings not made by this package, through\n DecodeAll and the Reader with one and the default number of goroutines\n (decoder_test.go).\n- TestSilesiaAsmVsGo: the assembly sequence decoder, the Go one and, on\n amd64 with BMI2, the non-BMI2 assembly on the same inputs\n (seqdec_test.go). decodeSyncGoOnly lets the test bypass the assembly.\n\nA new CI job on amd64 and arm64 caches silesia.tar keyed on the pinned\nSHA-256, and the CLI encodings keyed on it plus the zstd CLI version, and\nruns these tests with and without -tags\u003dnoasm.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: add BenchmarkSilesia and BenchmarkCorpus\n\nBenchmarkSilesia encodes testdata/silesia.tar at each level and decodes\nthe result, then decodes the zstd CLI\u0027s encodings of it.\n\nBenchmarkCorpus decodes the files in the directory named by ZSTD_CORPUS,\nsuch as the real-world corpora at https://klauspost.com/files/compress/,\nwith DecodeAll into a preallocated buffer and with the Reader on one and\nthe default number of goroutines. Files that are not .zst are encoded at\nSpeedDefault first.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* s2, flate: smoke-test Silesia too\n\nMove the fetch script to the repository\u0027s testdata directory, next to the\nother shared test files, so every package can use the one silesia.tar.\ninternal/silesiatest reads it once per test binary, falls back to a\npackage-local testdata/silesia.tar, and skips when it is absent or with\n-short.\n\nNew tests, in the same CI job:\n- s2 TestSilesiaRoundTrip (s2_test.go): Encode, EncodeBetter, EncodeBest\n and EncodeSnappy through Decode; Snappy both ways against the reference\n implementation; Writer/Reader at each level.\n- flate TestSilesiaRoundTrip (writer_test.go): Writer/Reader at each level\n from HuffmanOnly to BestCompression, each result also decoded by the\n standard library, and the standard library\u0027s encoding decoded here.\n\nhuff0 and fse are left out: their blocks are at most 128 KiB, and the zstd\ntests already run all of Silesia through them. gzip, zlib and zip wrap\nflate.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "951333e537ae02581cf9ff53780e874bcc0aafb0", "tree": "17d8b5f7a836886942bb62d83982f9f76bf38e0f", "parents": [ "3ea13ece2e321191ac0852fe0cb1d5e5f7473044" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Tue Sep 29 07:44:36 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 29 07:44:36 2026" }, "message": "zstd, huff0: cheaper per-block table setup (#1234)\n\n* zstd: fold the FSE symbol transform into the table build\n\nThe sequence decoder built each FSE table in two passes: buildDtable\nspread the symbols and set nbBits/newState, then transform rewrote every\nstate again to replace the symbol with its addBits and baseline, checking\nthe symbol bound on each entry. readNCount already limits symbols to the\ntable\u0027s range, so that check could not fail.\n\nPrecompute each table\u0027s per-symbol addBits/baseline once, and have the\nbuild loop merge them into the entry it stores, as the reference decoder\ndoes. The asm build loop now does one 8-byte store per state instead of\na byte and a word store. maxBits is the last symbol\u0027s addBits, since\naddBits never decreases with the symbol in any of the three tables;\ninitPredefined checks that.\n\nBenchmarkBuildDtable now includes the work transform did.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* huff0: faster ReadTable weight stats and table fill\n\nWeights repeat, so counting them into a single histogram made each\nincrement wait on the previous store to the same counter. Count into\nfour histograms and merge; the weight total and the too-large check are\ntaken from the merged counts.\n\nFill runs of 4 or more entries 4 at a time.\n\nTestReadTableWeights (decompress_test.go) covers the weight checks inside\nand after the four-at-a-time loop.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: reject FSE counts for symbols past the table\u0027s range\n\nreadNCount\u0027s main loop stops at maxSymbol, but a run of zero counts only\nchecks against 255, so a table description could skip past the last valid\nsymbol and give the next one a count; the check after the loop allowed up\nto 256 symbols. transform used to reject such tables. Without it they got\nstates with zero addBits and baseline, and a maxBits that was too low for\nthe refill paths. Check symbolLen against the table\u0027s own maximum.\n\nTestReadNCountSymbolPastTable in seqdec_test.go gives an offsets table a\ncount for symbol 40 after a zero run.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "3ea13ece2e321191ac0852fe0cb1d5e5f7473044", "tree": "ea155b65301a6343158bdf535938f6b045073c20", "parents": [ "179d9a25ce9dd991c5832f7e768f6d55ff6c2c21" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Tue Sep 29 07:44:12 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 29 07:44:12 2026" }, "message": "tests: Use Go 1.27 \u0026 fixed fuzz time (#1239)\n\n* tests: Use Go 1.27 \u0026 fixed fuzz time\n\nSimplifies fuzz running a bit.\n\nBumps goreleaser as well.\n\n* Skip BMI2 decoder on arm/noasm" }, { "commit": "179d9a25ce9dd991c5832f7e768f6d55ff6c2c21", "tree": "b1d498ccfbca5cf1e76e47764a3b51f5dbf070d7", "parents": [ "8a5e589b1b1ea353ceb1a02a6c671a294bc175eb" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Sep 28 20:13:10 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 28 20:13:10 2026" }, "message": "zstd: align the sequence decode and execute loops to 64 bytes (#1240)\n\n* zstd: move the asm generator to the current avo arm64 fork\n\nUse the same honeycombio/avo pin as huff0/_generate. The fork\u0027s arm64\nlowering passes PCALIGN through, and it rejects a label placed directly on\na PCALIGN, which the amd64 assembler mishandles (golang/go#74648,\ngolang/go#81792). The regenerated assembly is byte-identical. The newer\navo requires newer golang.org/x modules, which require go 1.25.0.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: align the sequence decode and execute loops to 64 bytes\n\nPlace PCALIGN $64 before the decode/decodeSync and executeSimple main\nloops, which also raises each function\u0027s alignment to 64. Without it the\nloop heads land wherever the linker places the function, and the same\ncode measured about ±2% apart between builds that differed only in what\ncame before it. The padding runs once per call.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "8a5e589b1b1ea353ceb1a02a6c671a294bc175eb", "tree": "8c54322030a2167f9c5802f2e05972d0b2388bb4", "parents": [ "0533a6752647c21beafdf426e54b917389537e4d" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Sep 28 20:12:10 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 28 20:12:10 2026" }, "message": "zstd: keep default-encoder hash primes in registers on arm64 (#1241)\n\nApply the better encoder\u0027s treatment to doubleFastEncoder and\ndoubleFastEncoderDict, which hash with the same two primes. The primes\nare loaded once per Encode through hashPrimes8and5, now shared in\nhash.go, and dFastHashL/dFastHashS replace hashLen in the encode loops.\nReset and ResetPrefix keep hashLen.\n\nOutput is byte-identical to master (8 corpora x default/better, arm64\nand amd64). The better encoder\u0027s code generation is unchanged.\n\nDefault level, 4 MiB frames, benchstat MB/s vs master:\n Graviton2 +4.4%, Graviton3 +2.1%, Graviton4 +1.9%, Sapphire Rapids ~\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "0533a6752647c21beafdf426e54b917389537e4d", "tree": "639541a6f9a7b210825fc38c30d60a23d8ae0e51", "parents": [ "33090b91663ee20041c754c34227d102ebbd4bc4" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Sep 28 17:43:06 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 28 17:43:06 2026" }, "message": "huff0: align the decode inner loops to 64 bytes (#1236)\n\n* huff0: move the asm generator to the current avo arm64 fork\n\nThe honeycombio/avo fork adds a PCALIGN pseudo-instruction that the arm64\nlowering passes through. The regenerated assembly is byte-identical. The\nnewer avo requires newer golang.org/x modules, which require go 1.25.0.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* huff0: align the decode inner loops to 64 bytes\n\nPlace PCALIGN $64 before the 4X and 1X inner loops, which also raises\neach function\u0027s alignment to 64. Without it the loop heads land wherever\nthe linker places the function, and the same code measured about ±2%\napart between builds that differed only in what came before it. The\npadding runs once per outer iteration.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* huff0: move the asm generator to avo with the PCALIGN label check\n\nThe honeycombio/avo fork now rejects a label placed directly on a\nPCALIGN, which the amd64 assembler mishandles (golang/go#74648,\ngolang/go#81792). The regenerated assembly is byte-identical.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "33090b91663ee20041c754c34227d102ebbd4bc4", "tree": "2d2e78da71638520b8324fdf7b038ebb9b859090", "parents": [ "f4490a82af6566a02a60e5fffe9a5258e39a5a90" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Sep 28 17:42:44 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 28 17:42:44 2026" }, "message": "docs: Add AGENTS.md (#1238)\n\n* Add AGENTS.md\n\nConventions and traps for people and coding agents changing the\nrepository: generated assembly and the avo fork, the PCALIGN label hazard,\nassembly calls, build tags, checks and fuzzing before sending a change,\nwhere tests go, what counts as evidence for a performance claim, and\npull-request conventions.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* AGENTS.md: swap which build runs first each round\n\nAlternating A and B still gives one of them the same slot every round;\nswapping the order each round keeps warm-up and slot effects off both.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* AGENTS.md: reflow the benchmark alternation bullet\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* AGENTS.md: registers with fixed roles survive calls\n\nGo\u0027s ABI has no callee-saved registers, but the stack and frame pointers\nand the goroutine register keep their roles across a call. Say so, as the\nABI document does, while keeping the rule to spill everything else.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* AGENTS.md: address review\n\n- Drop the bullet on calling Go from assembly; it is too specific here.\n- Vet code for every platform it targets instead of naming arm64, and test\n on every architecture whose assembly changed.\n- Check `go fix -diff ./...`.\n- Move the assembly section to the end; the others are more general.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* gzhttp: apply go fix\n\nUse slices.Backward, so `go fix -diff ./...` reports nothing.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "f4490a82af6566a02a60e5fffe9a5258e39a5a90", "tree": "5323f7e4f54ea256b1e86526b08a79ca8d5f359f", "parents": [ "8219c0c5fb34167e7a53bbb1e11306d5e37854a2" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Sep 28 17:42:12 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 28 17:42:12 2026" }, "message": "zstd: close most of the decode gap to libzstd on real corpora (#1232)\n\n* zstd: keep literal slack for treeless literal sections\n\nA treeless literals section (Huffman table reused from the previous\nblock) was decoded into literalBuf[:0:litRegenSize] and kept that\ncapacity. The compressed-literals case re-slices to literalBuf to regain\nthe compressedBlockOverAlloc bytes past the end; the treeless case did\nnot, so useSafeDecodeSync saw no literal slack and sent every block that\nreused a table to the bounds-exact decodeSync_safe loop.\n\nThe reference encoder reuses tables often. In the zstd -19 files from\nklauspost.com/files/compress, as provided, the safe loop ran for 99% of\ngob-stream\u0027s blocks, 85% of github-ranks-backup\u0027s, 62% of\nsofia-air-quality\u0027s, 42% of cockroach\u0027s, 28% of nyc-taxi\u0027s and 5% of\napache\u0027s. After this change none do. Frames from this package\u0027s encoder\nwere not affected, since it rarely emits treeless sections.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01JsKM9Z6WCqNNFCZhjGHJcM\n\n* zstd: stream synchronously through a history ring instead of sliding\n\nWith WithDecoderConcurrency(1), the Reader kept history in one buffer of\nwindow plus 1 MiB (lowmem, the default) and, once free space fell to a\nblock, moved the whole window down. With an 8 MiB window that is an\n8 MiB memmove per ~1 MiB of output: 22.8 GB moved to stream the 2.5 GB\napache.log from klauspost.com/files/compress, 75% of decode time.\n\nKeep history in a ring of window plus two blocks of space, as the\nreference decoder does. Each block is written at the current position\nwith its output capped to one block of space; when the next block would\nnot fit, the lap just written becomes ext and writing restarts at the\nfront. decodeSync already takes the history preceding out as a separate\nslice, so ext is passed there and matches that start in the previous lap\ncontinue into the current one. Nothing is moved, and the ring is smaller\nthan the lowmem buffer for windows above 256 KiB.\n\nBlock space and the space checks in decodeSyncSimple/useSafeDecodeSync\nuse the frame\u0027s maximum block size, min(window, 128 KiB), rather than\n128 KiB, so small-window streams also reach the assembly.\n\nThe decodeSync \"bigger than current history\" errors now report bytes\nproduced by the current block without the history slice. That is what\nDecodeAll and the Reader both reported before, since the slice was\nalways empty on those paths; TestDecoderRegression compares the two.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01JsKM9Z6WCqNNFCZhjGHJcM\n\n* zstd: copy overlapping matches in 16-byte blocks when the offset allows\n\nThe assembly copied every overlapping match (length \u003e offset) one byte\nper iteration. With an offset of at least 16, each 16-byte block reads\nonly bytes that earlier blocks have already written, so these matches\ncan be copied like non-overlapping ones; the reference decoder does the\nsame and only special-cases offsets below 16.\n\nRepeated content is mostly long overlapping matches at large offsets:\n64% of html_x_4\u0027s output and 27-30% of apache.log\u0027s comes from them,\nwhile shorter overlapping offsets are under 1% of any corpus measured.\nDecodeAll as a share of libzstd\u0027s speed, pinned builds, 200 MB slices of\nklauspost.com/files/compress plus the benchdecoder files:\n\n html_x_4 apache -19 all real (geomean)\n Graviton5 37 -\u003e 99% 59 -\u003e 87% 84 -\u003e 91%\n Graviton4 38 -\u003e 100% 62 -\u003e 85% 82 -\u003e 88%\n Zen 4 37 -\u003e 91% 67 -\u003e 93% 87 -\u003e 94%\n Sapphire Rapids 38 -\u003e 92% 72 -\u003e 89% 85 -\u003e 90%\n Neoverse N1 31 -\u003e 76% 54 -\u003e 77% 71 -\u003e 75%\n\nThe fast variant reuses the 16-byte copy loop and its 15-byte overrun;\nthe bounds-exact variant copies whole blocks and finishes byte by byte.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01JsKM9Z6WCqNNFCZhjGHJcM\n\n* zstd: hash DecodeAll output block by block\n\nrunDecoder decoded the whole frame and then ran xxhash over all of it,\nso for a large frame the checksum pass read the output back from memory\nafter it had left the cache. Hash each block\u0027s output as soon as it is\ndecoded instead, as the streaming decoder and the reference decoder do.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01JsKM9Z6WCqNNFCZhjGHJcM\n\n* zstd: tighten the ring and overlap tests (review)\n\nTestDecodeOverlappingMatches appended tail%period bytes, so a tail equal\nto a multiple of the period appended nothing; append exactly tail bytes.\n\nTestStreamRingWraps only compared output, so it did not show that\nmatches near the full window distance reach the decoder. Add an input\nthat repeats a random chunk of 15/16 of the window: it can only compress\nthrough matches at that distance, and the test now requires it to shrink\nto under half its size (45%, 46% and 15% for 1 KiB, 64 KiB and 1 MiB\nwindows).\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01JsKM9Z6WCqNNFCZhjGHJcM\n\n* zstd: use the history ring in the concurrent stream decoder\n\nWith the default concurrency the Reader\u0027s execute goroutine kept history\nin window + 1 MiB (lowmem) and history.append moved the whole window down\nwhenever free space ran out, the same slide the synchronous path had.\nWith an 8 MiB window that is an 8 MiB memmove per ~1 MiB of output; on\nGraviton5 it was 37% of CPU streaming klauspost.com\u0027s gob-stream and JSON\nfiles, leaving the concurrent Reader slower than the synchronous one.\n\nExecute sequences directly into the history ring, passing the previous\nlap as history as decodeCompressed does, then copy the block\u0027s output to\nits buffer. That keeps one copy per output byte, as before, without the\nslide. Raw and RLE blocks append to the ring too, and the ring is kept\nacross Reset. history.append and allocFrameBuffer are no longer used and\nare removed, and the \"Last block\" message is now debug-only.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01JsKM9Z6WCqNNFCZhjGHJcM\n\n* zstd: test overlapping matches on the concurrent and two-pass paths\n\nTestDecodeOverlappingMatches only reached decodeSync. The same copy change\nis in executeSimple, which the concurrent Reader uses and, since #1212, so\ndoes the two-pass DecodeAll path. Run each input through both, too.\nChanging executeSimple\u0027s threshold from 16 to 8 now fails 30 of the\nsubtests on amd64; before, none failed.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: remove unreachable output allocation from runDecoder\n\nrunDecoder set d.history.b \u003d dst before the block that grew dst when\ncap(dst) \u003c maxSyncLen, so if it ran it copied the output for nothing:\ndecoding kept appending to the old buffer and the copy was dropped.\n\nIt cannot run. DecodeAll, the only caller, grows dst to at least\nlen(dst)+FrameContentSize+compressedBlockOverAlloc under the same\ncondition (known content size, !limitToCap), which is maxSyncLen plus the\nslack. A panic planted in it was never reached by the zstd tests with and\nwithout noasm or by 300000 FuzzDecodeAll executions.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "8219c0c5fb34167e7a53bbb1e11306d5e37854a2", "tree": "e5164fc031b24f757a70e0c157f2ce25ab42a115", "parents": [ "2393e1cb51b2d2b7ecd4459b0d8e263bae509d63" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Sep 28 12:12:49 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 28 12:12:49 2026" }, "message": "zstd: reuse dictionary sequence tables in the encoder (#1233)\n\n* zstd: reuse dictionary sequence tables in the encoder\n\nThe encoder used only the dictionary\u0027s Huffman literal table. Its FSE\ntables for literal lengths, match lengths and offsets were loaded but\nnever offered as the \"previous\" tables, so the first block of every\nframe paid for new or predefined sequence tables. Build encoders for\nthem once when the dictionary is loaded, and copy them into the\nprevious-table slots on each reset, so the first block can use repeat\nmode, as libzstd does.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01UYFUfDhLAWujsNGaXiAwKF\n\n* zstd: count exact costs when reusing a previous sequence table\n\nWith dictionary tables offered for reuse, the chooser picked them where\na predefined or new table was smaller. Two costs were missing from the\ncomparison:\n\n- Each stream ends by flushing its FSE state, tableLog bits. Dictionary\n tables have larger table logs than the predefined ones, which decides\n the choice for messages with few sequences.\n- A new table was charged its maximum header size plus a penalty. That\n keeps new tables from replacing predefined ones for tiny gains, but\n against a previous table, which may fit poorly, the exact header size\n is now used.\n\nOn sofia-air-quality records with a trained dictionary this turns a\n0.3-2.0% size increase into a decrease. Without a dictionary, output\nshrinks 0.01-0.62% on apache, github json, nyc-taxi and gob-stream.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01UYFUfDhLAWujsNGaXiAwKF\n\n---------\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "2393e1cb51b2d2b7ecd4459b0d8e263bae509d63", "tree": "8fa8816b2ff514a63e41548afb7b96fe3edc389c", "parents": [ "258a0cee549b10a1f3b32263fdc9b2405b3ee23f" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Sep 28 09:40:42 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 28 09:40:42 2026" }, "message": "zstd: speed up better encoder hashing, primes in registers on arm64 (#1230)\n\n* zstd: keep better-encoder hash primes in registers on arm64\n\narm64 rebuilds each 64-bit hash prime with MOVD+3xMOVK per use. Load them\nonce per Encode so they stay in registers; other arches keep constants.\nAlso compute the index-loop bound once and store the short table first so\ncv1 dies early, removing per-iteration spills. Output is unchanged.\n\nCo-Authored-By: Claude Opus 5.5 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* zstd: name the per-arch hash prime switch for all encoders\n\nRename betterPrimesInRegs to hashPrimesInRegs, in hash_regs_arm64.go and\nhash_regs_other.go, so other encoders can use the same switch.\nNo code generation change.\n\nCo-Authored-By: Claude Opus 5.5 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* zstd: move better hash test into encoder_test.go\n\nCo-Authored-By: Claude Opus 5.5 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* zstd: select hash prime handling with runtime.GOARCH\n\nReplace the two build-tagged files with a constant in hash.go.\nNo code generation change.\n\nCo-Authored-By: Claude Opus 5.5 (1M context) \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5.5 (1M context) \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "258a0cee549b10a1f3b32263fdc9b2405b3ee23f", "tree": "78a408b46e290729bcffc002c988faee99bbebb3", "parents": [ "e4086c1ba49285cd62b7ea4f64176cbf1e08c94d" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Sep 28 08:30:03 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 28 08:30:03 2026" }, "message": "zstd: two-pass decode with match-source prefetch on arm64 (largest win decoding SpeedBestCompression-class data) (#1212)\n\n* zstd: two-pass decode with match-source prefetch for large windows on arm64\n\nThe synchronous decoder (DecodeAll, or a Reader with concurrency 1)\ndecodes and executes each sequence in one loop. The concurrent decoder\ndecodes a block\u0027s sequences into seqVals first and executes them in a\nsecond pass. That second shape knows every match offset before the\ncopies run, so the execute loop can prefetch match sources from the\nsequence table, and its short iteration lets the core keep several\nmisses in flight. On a Neoverse N1 (oracle) the execute stage of a\nsilesia subset at production settings ran 26% faster with the prefetch\neight sequences ahead, the knee of a distance sweep from 1 to 16.\n\nexecuteSimple\u0027s generated loop now carries that prefetch, switched per\ncall by a new executeAsmContext.prefetch that folds into the loop\u0027s\nbound check (off, the end pointer is the start pointer and every helper\nskips). Nothing new is loop-carried in a register: the copy code already\nuses the whole file and avo cannot spill, so the running output position\nand bounds live in stack slots and the cursor is a displacement off the\nsequence pointer. A match reaching into a separate history buffer is\nredirected there rather than prefetched from below the output buffer,\nwhere each one cost a page walk. The avo fork gains the PRFM lowering\n(honeycombio/avo 6461c92).\n\ndecodeCompressed takes the two-pass path from a 1 MiB window on arm64\nand never on amd64, where it measured 7-15% slower on three parts; the\nper-file spread that motivates the next commit\u0027s gate is -15% to +5%.\n\nDecodeAll on the N1, checksum off, versus the one-pass decoder:\n\n silesia dickens+reymont+xml+ooffice+webster, SpeedDefault -9.2%\n synthetic, offsets 128-768 KiB -28%\n synthetic, offsets 1-2 MiB -51%\n synthetic, offsets under 32 KiB in a 4 MiB window +7.9%\n benchdecoder corpus (windows under 1 MiB, gate off) 0.0%\n\nTestDecodeTwoPassMatchesOnePass forces both paths on any architecture\nand checks every DecodeAll and Reader shape, including a separate\nhistory buffer, plus corrupt and truncated input. Test_seqdec_execute\nruns with the prefetch off and forced on and compares the output, which\nexercises the history-buffer redirect on the small seqs.zip windows.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* zstd: gate the two-pass decode on the block\u0027s offset table\n\nThe window size alone is a coarse proxy for how far a block\u0027s matches\nreach: at an 8 MiB window the two-pass decode wins 8-15% on text but\nloses 4-5% on mozilla, ooffice and xml, whose matches are mostly near.\nThe block header already says which: the offset FSE table\u0027s normalized\ncounts are the encoder\u0027s histogram of the block\u0027s offset codes, and\ncode 17 and up means 128 KiB or more. fseDecoder.codeShare sums that\nmass once per block (a loop over at most 32 entries); the predefined\nand RLE tables carry no distribution and keep the window rule. useTwoPass\ntakes the two-pass path when the window is at least 1 MiB and the share\nis at least 18% (46/256), between the 16% ceiling of every silesia file\nthat lost and the 20% floor of every file that gained; the same decision\nswitches executeSimple\u0027s prefetch. The window condition stays because\na frame that fits in L2 has no misses whatever its offsets: with the\nshare alone, three small-file corpus members that the window excluded\nregressed 3-4%.\n\nThe table share tracks the measured far share within two points on\neleven of twelve silesia files; nci undercounts because repeat-offset\ncodes hide far matches and is still selected on 93% of its sequences.\n\nNeoverse N1, DecodeAll, checksum off, versus one-pass, with the window\ngate in parentheses: silesia subset -9.8% (-8.7%); offsets under 32 KiB\nin a 4 MiB window +0.4% (+6.4%); silesia per-file geomean -3.6% (-2.2%),\nworst cells mozilla +0.8% (+5.6%) and ooffice +0.7% (+4.9%); the\nbenchdecoder corpus is unchanged (+0.1%).\n\nTestSequenceDecsUseTwoPass covers each kind of offset table against a\nsmall and a large window.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* zstd: count a match as far from 1 MiB unless the core is a Neoverse N1\n\nThe gate\u0027s thresholds were calibrated on a Neoverse N1 (oracle), and\nGraviton2 is the same core; on Graviton3, 4 and 5 (Neoverse V1, V2, V3)\nthe two-pass decode loses on real data at those thresholds, 128 KiB\nbeing well inside what those cores hide: silesia subset +0.4%, +8.7%\nand +11.1%, silesia geomean +1.9%, +5.1% and +5.6%. Every arm64 core\nmeasured still gains from 1 MiB (the synthetic 1-2 MiB bucket: -48%\non the N1, -32%, -24% and -17% on V1, V2 and V3), and no silesia file\nhas 18% of its offsets that far, so with the far code at 20 those\ncores keep the far-offset gain and are neutral on real data.\n\ntwoPassFarCode is therefore 20 by default and 17 when the core is an\nN1, read once at init from the MIDR register Linux exposes in sysfs;\nelsewhere the conservative default stands. Per-core gate versus\none-pass, DecodeAll, checksum off: oracle silesia subset -9.0% and\ngeomean -3.8%; Graviton2 -5.9% and -1.9% (mr +2.7%); Graviton3 -1.5%\nand -0.1%; Graviton4 0.0% and +0.3%; Graviton5 +0.6% and -0.5%; the\n1-2 MiB bucket as above; the benchdecoder corpus within 1.2% on every\npart. amd64 is unaffected: there the two-pass decode loses on every\nsilesia file on Sapphire Rapids, Granite Rapids and Zen 4, and on the\n1-2 MiB bucket on Zen 4, so it stays off.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* zstd/_generate: note the avo fork\u0027s upstreaming target in go.mod\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* zstd: drop the Neoverse N1 sysfs threshold override, add a quick reject\n\nKlaus\u0027s review on #1212: a single-arm64-core-model special case doesn\u0027t\nbelong in the package, and it will phase itself out as those cores are\nsunsetted. twoPassFarCode goes back to a plain const at 20 (1 MiB), no\nsysfs read, no per-core branch; seqdec_linux_arm64.go is gone.\n\nOn oracle (Ampere Altra, an N1), farCode\u003d20 on real silesia data is\nnoise-level versus one-pass (geomean -0.33% vs the -3.8% the 17-threshold\ngave up), because the offset-table share on real text rarely reaches 18%\nat the 1 MiB code. The synthetic offset buckets show why: the 128-768 KiB\nbucket the N1 threshold specifically targeted goes from -26.8% to ~0%\n(entirely forgone), while the 1-2 MiB bucket both thresholds already\nqualify for stays at -39.7% (kept). N1 keeps the general-arm64 win and\ngives up only the narrow sub-1MiB extra it needed the override for.\n\nAlso addresses Klaus\u0027s inline comment on useTwoPass: reject before\ncomputing the offset table\u0027s far-code share when the table\u0027s symbolLen\ndoesn\u0027t reach twoPassFarCode at all, so codeShare\u0027s loop is skipped\nwhenever it could only return 0.\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: move two-pass decode tests into seqdec_test.go\n\nCo-Authored-By: Claude Opus 5.5 (1M context) \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e" }, { "commit": "e4086c1ba49285cd62b7ea4f64176cbf1e08c94d", "tree": "46c7c28c837a0c19f8a2472259e5e778d761240a", "parents": [ "5d880f230c38a0fc806b9ca1613103a44feff0ac" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Fri Sep 25 08:43:26 2026" }, "committer": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Fri Sep 25 08:43:26 2026" }, "message": "Update README with new changelog\n" }, { "commit": "5d880f230c38a0fc806b9ca1613103a44feff0ac", "tree": "4ee3eafd7347c32a600732894479d554a8f7e10e", "parents": [ "5ed8a01b0df5d1c6ae5933014b7a901ca5bd1449" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Fri Sep 25 08:00:35 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Sep 25 08:00:35 2026" }, "message": "tests: Pre-release cleanups (#1231)\n\n* tests: Pre-release cleanups\n\nCompress large regression test. Fix flaky fuzz test.\n\n* Added while verifying the narrowed condition is sound: continuing the walk makes a second divergence reachable that\n the old give-up-at-fill predicate happened to cover. The reference\u0027s hi is unbounded — it climbs past 4095 and wraps a\n uint16 after 61441 more codes, at which point hi+1 \u003e\u003d overflow goes false again and it resumes assigning table\n entries from a low hi, which our decoder never does. Its output past that point says nothing about ours, and ours\n needn\u0027t stop with invalid code there, so that case is a distinct result (refHiWrapped) that skips comparison rather\n than asserting the error text. Hence three states instead of a bool." }, { "commit": "5ed8a01b0df5d1c6ae5933014b7a901ca5bd1449", "tree": "14a6edc675c5c12390d4ec05ef35d4d374361266", "parents": [ "0bed72427862f1d6e18fac32d14509659c7932f7" ], "author": { "name": "Rupayon Haldar", "email": "80724680+rupayon123@users.noreply.github.com", "time": "Wed Sep 23 15:06:51 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 23 15:06:51 2026" }, "message": "Report unexpected EOF for truncated Huffman extra bits (#1229)" }, { "commit": "0bed72427862f1d6e18fac32d14509659c7932f7", "tree": "7c4c5d03408efc3d00d049494b1a214514eb36c7", "parents": [ "147b531210dec3320948ca045feb4f75513c6ed2" ], "author": { "name": "Malte Poll", "email": "1780588+malt3@users.noreply.github.com", "time": "Wed Sep 23 14:11:20 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 23 14:11:20 2026" }, "message": "flate: avoid FMA in EstimatedBits for portable rounding (#1224)\n\nEstimatedBits and mFastLog2 had chained multiply-add expressions\nthat the compiler is allowed to contract into fused multiply-adds.\nFMA contraction rounds differently than sequential multiply+add,\nso EstimatedBits previously returned a different result on\narchitectures with hardware FMA (e.g. arm64) than on default amd64,\nmaking EstimatedBits, and the resulting compressed output, GOARCH-dependent.\n\nFixes https://github.com/klauspost/pgzip/issues/72" }, { "commit": "147b531210dec3320948ca045feb4f75513c6ed2", "tree": "179b73a2f638079ba89dae7d2fb56dd638881c47", "parents": [ "58dac78a7837e06b478a286de653ea376a979f06" ], "author": { "name": "Robert Pająk", "email": "pellared@hotmail.com", "time": "Wed Sep 23 14:09:17 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 23 14:09:17 2026" }, "message": "zstd: apply reset options to reused frame decoders (#1226)\n\n* zstd: apply reset options to reused frame decoders\n\n* zstd: simplify decoder option snapshots\n\n* zstd: inline decoder option snapshot access" }, { "commit": "58dac78a7837e06b478a286de653ea376a979f06", "tree": "d9f9be0e7840763e5c3330c99c2234213d7265e3", "parents": [ "9b00a552dd3ceef5cb6079efa17f135aa9783cde" ], "author": { "name": "KBS", "email": "youdie006@naver.com", "time": "Wed Sep 23 14:08:55 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 23 14:08:55 2026" }, "message": "gzhttp: only decompress a request when gzip is the outermost coding (#1223)\n\n* gzhttp: only decompress a request when gzip is the outermost coding\n\ncontentGzip used parseEncodingGzip, which scans a comma list for gzip in any\nposition with q-values. That is an Accept-Encoding parser: Accept-Encoding is\nan unordered preference set, so acceptsGzip is right to use it. Content-Encoding\nlists codings in the order they were applied, so only the last one is the\noutermost coding a decoder may remove.\n\nTwo consequences for a handler wrapped with AllowCompressedRequests(true):\n\n Content-Encoding: gzip, deflate the body is deflate on the outside, but it\n was fed to a gzip reader and the handler\n got \"gzip: invalid header\"\n Content-Encoding: br, gzip gunzipping is correct, but deleting the\n whole header hid that br is still applied\n\nDecide on the last coding and strip only that one, leaving the rest in place.\nThe single \"Content-Encoding: gzip\" case is unchanged.\n\n* gzhttp: read Content-Encoding across field lines\n\nA sender may split one Content-Encoding list over several field lines, which\nmeans the same as the joined comma list. Header.Get returns only the first of\nthem, so \"Content-Encoding: gzip\" followed by \"Content-Encoding: deflate\" was\nread as gzip alone and the body was fed to a gzip reader although deflate is\nthe outermost coding. The reverse pair was left undecoded for the same reason.\n\nJoin the field lines before deciding and before rewriting.\n\nThe three-coding test case declared br and deflate but the fixture applied\nneither, so it asserted nothing about them. Use three gzip layers instead,\nwhich the fixture can really apply, and assert that exactly one is removed.\n\n* gzhttp: make the content-encoding cases round trip\n\nMove the cases into compress_test.go instead of a new file, and encode each\nrequest body with exactly the codings it declares so the handler can decode\nwhatever is left and compare against testBody. A pass now means the payload\nsurvived compression and decompression byte for byte, rather than that one\nheader string matched.\n\nNo test in the package read r.Body before this, so the request-decompression\npath was only covered by a response-length check.\n\n* gzhttp: use strings.SplitSeq in the content-encoding test\n\ngo fix flags strings.Split here; the CI fix gate runs go fix -diff over the\nwhole module." }, { "commit": "9b00a552dd3ceef5cb6079efa17f135aa9783cde", "tree": "2b4afcbc820b4e5e3e7d04e0cdda15d6f5154ed5", "parents": [ "de8f55df9fb3af7aec787942d6b3c71e045bfee9" ], "author": { "name": "Jake Wang", "email": "jakezwang@users.noreply.github.com", "time": "Tue Sep 22 16:30:14 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 22 16:30:14 2026" }, "message": "gzhttp: preserve caller request headers in Transport (#1225)" }, { "commit": "de8f55df9fb3af7aec787942d6b3c71e045bfee9", "tree": "0c518419030ce061fcd52ed6a526507d4863559b", "parents": [ "a1c49c64983e0d7f036ea25c8913255f921df630" ], "author": { "name": "xingyaner", "email": "longfor2025@gmail.com", "time": "Tue Sep 15 09:46:55 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 15 09:46:55 2026" }, "message": "ossfuzz: include zstd seqdec path tests (#1222)\n\nSigned-off-by: xingyaner \u003clongfor2025@gmail.com\u003e" }, { "commit": "a1c49c64983e0d7f036ea25c8913255f921df630", "tree": "2c7f810704654ef64c3baedc65389a61d8297517", "parents": [ "ebd819039813720ce6ef7b8810a8d4d08830694c" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Sep 14 12:00:48 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 14 12:00:48 2026" }, "message": "flate: precompute literal costs for findMatch (#1217)\n\nBackport of https://go.dev/cl/829570 (golang/go#81398).\n\nWhen estimating the gain of a candidate match at level 9, findMatch\nsummed the literal code lengths of the whole match for every candidate\nthat was longer than the best so far, which cost about a tenth of the\nencoding time.\n\nCompute a cumulative literal cost array over the window once per\ndeflateLazy call, right after the literal encoder used for the estimate\nis generated, so that the cost of any range is a subtraction. Also\nhoist the hash chain fields into locals for the chain walk.\n\nUnlike upstream, litBits is a lazily allocated slice rather than an\narray in advancedState, so only level 9 writers pay the 256KB; levels\n7 and 8 never generate the estimate tables.\n\nfindMatch took the gain path for d.chain \u003e\u003d 100 while deflateLazy only\ngenerated the tables it needs for d.chain \u003e 100. No level has a chain\nof exactly 100 so this was unreachable, but the mismatch is now load\nbearing (a nil litBits instead of a nil d.h), so the conditions are\nmade to agree.\n\nbitLengthRaw is no longer used and is removed. The output is\nbyte-identical.\n\n```\ngoos: windows\ngoarch: amd64\npkg: github.com/klauspost/compress/flate\ncpu: AMD Ryzen 9 9950X 16-Core Processor\n │ master │ cl-829570 │\n │ sec/op │ sec/op vs base │\nPR/Digits/Huffman-32 804.3µ ± 1% 801.7µ ± 1% ~ (p\u003d0.305 n\u003d15)\nPR/Digits/L1-32 4.731m ± 1% 4.716m ± 1% ~ (p\u003d0.683 n\u003d15)\nPR/Digits/L4-32 5.518m ± 2% 5.486m ± 2% ~ (p\u003d0.285 n\u003d15)\nPR/Digits/L6-32 7.879m ± 5% 7.875m ± 1% ~ (p\u003d0.267 n\u003d15)\nPR/Digits/L7-32 14.54m ± 3% 14.50m ± 1% ~ (p\u003d0.116 n\u003d15)\nPR/Digits/L9-32 32.37m ± 3% 26.79m ± 6% -17.23% (p\u003d0.000 n\u003d15)\nPR/Twain/Huffman-32 1.051m ± 3% 1.051m ± 2% ~ (p\u003d0.775 n\u003d15)\nPR/Twain/L1-32 4.726m ± 2% 4.757m ± 1% ~ (p\u003d0.595 n\u003d15)\nPR/Twain/L4-32 5.944m ± 1% 5.898m ± 1% ~ (p\u003d0.187 n\u003d15)\nPR/Twain/L6-32 7.773m ± 3% 7.832m ± 2% ~ (p\u003d0.267 n\u003d15)\nPR/Twain/L7-32 17.70m ± 2% 17.50m ± 1% ~ (p\u003d0.202 n\u003d15)\nPR/Twain/L9-32 36.68m ± 1% 34.58m ± 2% -5.72% (p\u003d0.000 n\u003d15)\ngeomean 6.898m 6.745m -2.21%\n\n │ B/s │ B/s vs base │\nPR/Digits/Huffman-32 1.158Gi ± 1% 1.162Gi ± 1% ~ (p\u003d0.305 n\u003d15)\nPR/Digits/L1-32 201.6Mi ± 1% 202.2Mi ± 1% ~ (p\u003d0.683 n\u003d15)\nPR/Digits/L4-32 172.8Mi ± 2% 173.8Mi ± 2% ~ (p\u003d0.281 n\u003d15)\nPR/Digits/L6-32 121.0Mi ± 5% 121.1Mi ± 1% ~ (p\u003d0.263 n\u003d15)\nPR/Digits/L7-32 65.58Mi ± 3% 65.76Mi ± 1% ~ (p\u003d0.116 n\u003d15)\nPR/Digits/L9-32 29.47Mi ± 3% 35.60Mi ± 5% +20.81% (p\u003d0.000 n\u003d15)\nPR/Twain/Huffman-32 907.7Mi ± 3% 907.3Mi ± 2% ~ (p\u003d0.775 n\u003d15)\nPR/Twain/L1-32 201.8Mi ± 2% 200.5Mi ± 1% ~ (p\u003d0.602 n\u003d15)\nPR/Twain/L4-32 160.4Mi ± 1% 161.7Mi ± 1% ~ (p\u003d0.187 n\u003d15)\nPR/Twain/L6-32 122.7Mi ± 3% 121.8Mi ± 2% ~ (p\u003d0.267 n\u003d15)\nPR/Twain/L7-32 53.87Mi ± 2% 54.49Mi ± 1% ~ (p\u003d0.202 n\u003d15)\nPR/Twain/L9-32 26.00Mi ± 1% 27.57Mi ± 2% +6.05% (p\u003d0.000 n\u003d15)\ngeomean 138.3Mi 141.4Mi +2.26%\n```\n\nOriginal-Author: Alex Gaynor \u003calex.gaynor@gmail.com\u003e\n\nCo-authored-by: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "ebd819039813720ce6ef7b8810a8d4d08830694c", "tree": "ab2dbda7f2d728cd93f95494be2d7a2b2ea802b5", "parents": [ "4dc0d3242fe9d429f1da07a582fa023093040d43" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Sep 14 11:46:32 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 14 11:46:32 2026" }, "message": "flate: avoid the hand-rolled sorts when generating Huffman codes (#1214)\n\nBackport of https://go.dev/cl/829568 (golang/go#81398).\n\nSorting the symbols by frequency used a hand-rolled quicksort\n(huffman_sortByFreq.go), and assigning codes sorted every per-length\nchunk by literal again with a second quicksort\n(huffman_sortByLiteral.go). Replace the first with a stable counting\nsort on the 16-bit frequency (insertion sort for small alphabets), and\nthe second with the canonical code assignment from RFC 1951 3.2.2,\nwhich walks the literals in order and needs no sort at all. Both sort\nfiles become dead and are removed.\n\nUpstream replaced slices.SortFunc calls, so the win there is larger\nthan here; this tree already avoided the generic sort. The remaining\nwin is from dropping the O(n log n) second sort entirely and from the\ncheaper counting sort.\n\nThis adds a 1148 byte sortcache to each huffmanEncoder (four per\nwriter, plus the level 9 literal encoder).\n\nThe generated codes are unchanged and the output is byte-identical.\n\n```\ngoos: windows\ngoarch: amd64\npkg: github.com/klauspost/compress/flate\ncpu: AMD Ryzen 9 9950X 16-Core Processor\n │ master │ cl-829568 │\n │ sec/op │ sec/op vs base │\nPR/Digits/Huffman-32 804.3µ ± 1% 805.4µ ± 1% ~ (p\u003d0.870 n\u003d15)\nPR/Digits/L1-32 4.731m ± 1% 4.751m ± 1% +0.42% (p\u003d0.021 n\u003d15)\nPR/Digits/L4-32 5.518m ± 2% 5.509m ± 2% ~ (p\u003d0.412 n\u003d15)\nPR/Digits/L6-32 7.879m ± 5% 7.743m ± 1% -1.72% (p\u003d0.037 n\u003d15)\nPR/Digits/L7-32 14.54m ± 3% 14.69m ± 2% ~ (p\u003d0.461 n\u003d15)\nPR/Digits/L9-32 32.37m ± 3% 32.50m ± 4% ~ (p\u003d0.967 n\u003d15)\nPR/Twain/Huffman-32 1.051m ± 3% 1.011m ± 1% -3.78% (p\u003d0.000 n\u003d15)\nPR/Twain/L1-32 4.726m ± 2% 4.753m ± 1% ~ (p\u003d0.217 n\u003d15)\nPR/Twain/L4-32 5.944m ± 1% 5.838m ± 1% -1.78% (p\u003d0.003 n\u003d15)\nPR/Twain/L6-32 7.773m ± 3% 7.590m ± 1% -2.36% (p\u003d0.001 n\u003d15)\nPR/Twain/L7-32 17.70m ± 2% 17.45m ± 2% ~ (p\u003d0.305 n\u003d15)\nPR/Twain/L9-32 36.68m ± 1% 36.55m ± 1% ~ (p\u003d0.595 n\u003d15)\ngeomean 6.898m 6.845m -0.76%\n\n │ B/s │ B/s vs base │\nPR/Digits/Huffman-32 1.158Gi ± 1% 1.156Gi ± 1% ~ (p\u003d0.870 n\u003d15)\nPR/Digits/L1-32 201.6Mi ± 1% 200.7Mi ± 1% -0.42% (p\u003d0.021 n\u003d15)\nPR/Digits/L4-32 172.8Mi ± 2% 173.1Mi ± 2% ~ (p\u003d0.406 n\u003d15)\nPR/Digits/L6-32 121.0Mi ± 5% 123.2Mi ± 1% +1.76% (p\u003d0.036 n\u003d15)\nPR/Digits/L7-32 65.58Mi ± 3% 64.94Mi ± 1% ~ (p\u003d0.455 n\u003d15)\nPR/Digits/L9-32 29.47Mi ± 3% 29.34Mi ± 4% ~ (p\u003d0.943 n\u003d15)\nPR/Twain/Huffman-32 907.7Mi ± 3% 943.4Mi ± 1% +3.93% (p\u003d0.000 n\u003d15)\nPR/Twain/L1-32 201.8Mi ± 2% 200.7Mi ± 1% ~ (p\u003d0.217 n\u003d15)\nPR/Twain/L4-32 160.4Mi ± 1% 163.4Mi ± 1% +1.81% (p\u003d0.003 n\u003d15)\nPR/Twain/L6-32 122.7Mi ± 3% 125.6Mi ± 1% +2.42% (p\u003d0.001 n\u003d15)\nPR/Twain/L7-32 53.87Mi ± 2% 54.64Mi ± 2% ~ (p\u003d0.305 n\u003d15)\nPR/Twain/L9-32 26.00Mi ± 1% 26.09Mi ± 1% ~ (p\u003d0.588 n\u003d15)\ngeomean 138.3Mi 139.3Mi +0.77%\n```\n\nOriginal-Author: Alex Gaynor \u003calex.gaynor@gmail.com\u003e\n\nCo-authored-by: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "4dc0d3242fe9d429f1da07a582fa023093040d43", "tree": "6aafd00c93588290c321ac61486ebc5372317d49", "parents": [ "e022476498e6a3cc23ba4540dab98378eafc5d74" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Sep 14 11:46:03 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 14 11:46:03 2026" }, "message": "flate: use separate tables in histogramSplit (#1215)\n\nBackport of https://go.dev/cl/829566 (golang/go#81398).\n\nhistogramSplit walked four quarters of the input in parallel, but\nincremented counters in a single shared table. Whenever two of the\nstreams hit the same byte value, the increments were serialized through\nstore-to-load forwarding, which is common for text. Give each stream\nits own table and sum them at the end.\n\nThe in-tree comment claimed this was \"slightly slower\"; it is not, at\nleast not on this CPU. Unlike upstream, the merge step adds into h\nrather than overwriting it, so histogramSplit keeps the accumulating\nsemantics of the non-split path in histogram().\n\nThis only affects Huffman-only encoding and blocks without matches.\nThe output is byte-identical.\n\n```\ngoos: windows\ngoarch: amd64\npkg: github.com/klauspost/compress/flate\ncpu: AMD Ryzen 9 9950X 16-Core Processor\n │ master │ cl-829566 │\n │ sec/op │ sec/op vs base │\nPR/Digits/Huffman-32 804.3µ ± 1% 754.8µ ± 0% -6.16% (p\u003d0.000 n\u003d15)\nPR/Digits/L1-32 4.731m ± 1% 4.720m ± 1% ~ (p\u003d0.624 n\u003d15)\nPR/Digits/L4-32 5.518m ± 2% 5.531m ± 2% ~ (p\u003d0.744 n\u003d15)\nPR/Digits/L6-32 7.879m ± 5% 7.907m ± 2% ~ (p\u003d0.512 n\u003d15)\nPR/Digits/L7-32 14.54m ± 3% 14.55m ± 1% ~ (p\u003d0.436 n\u003d15)\nPR/Digits/L9-32 32.37m ± 3% 32.56m ± 1% ~ (p\u003d1.000 n\u003d15)\nPR/Twain/Huffman-32 1050.6µ ± 3% 989.7µ ± 1% -5.79% (p\u003d0.000 n\u003d15)\nPR/Twain/L1-32 4.726m ± 2% 4.733m ± 1% ~ (p\u003d0.806 n\u003d15)\nPR/Twain/L4-32 5.944m ± 1% 5.960m ± 2% ~ (p\u003d0.870 n\u003d15)\nPR/Twain/L6-32 7.773m ± 3% 7.884m ± 1% ~ (p\u003d0.217 n\u003d15)\nPR/Twain/L7-32 17.70m ± 2% 17.51m ± 3% ~ (p\u003d0.389 n\u003d15)\nPR/Twain/L9-32 36.68m ± 1% 36.64m ± 1% ~ (p\u003d0.838 n\u003d15)\ngeomean 6.898m 6.837m -0.89%\n\n │ B/s │ B/s vs base │\nPR/Digits/Huffman-32 1.158Gi ± 1% 1.234Gi ± 0% +6.56% (p\u003d0.000 n\u003d15)\nPR/Digits/L1-32 201.6Mi ± 1% 202.1Mi ± 1% ~ (p\u003d0.631 n\u003d15)\nPR/Digits/L4-32 172.8Mi ± 2% 172.4Mi ± 2% ~ (p\u003d0.736 n\u003d15)\nPR/Digits/L6-32 121.0Mi ± 5% 120.6Mi ± 2% ~ (p\u003d0.506 n\u003d15)\nPR/Digits/L7-32 65.58Mi ± 3% 65.55Mi ± 1% ~ (p\u003d0.442 n\u003d15)\nPR/Digits/L9-32 29.47Mi ± 3% 29.29Mi ± 1% ~ (p\u003d1.000 n\u003d15)\nPR/Twain/Huffman-32 907.7Mi ± 3% 963.6Mi ± 1% +6.15% (p\u003d0.000 n\u003d15)\nPR/Twain/L1-32 201.8Mi ± 2% 201.5Mi ± 1% ~ (p\u003d0.806 n\u003d15)\nPR/Twain/L4-32 160.4Mi ± 1% 160.0Mi ± 2% ~ (p\u003d0.870 n\u003d15)\nPR/Twain/L6-32 122.7Mi ± 3% 121.0Mi ± 1% ~ (p\u003d0.217 n\u003d15)\nPR/Twain/L7-32 53.87Mi ± 2% 54.46Mi ± 2% ~ (p\u003d0.389 n\u003d15)\nPR/Twain/L9-32 26.00Mi ± 1% 26.03Mi ± 1% ~ (p\u003d0.830 n\u003d15)\ngeomean 138.3Mi 139.5Mi +0.89%\n```\n\nOriginal-Author: Alex Gaynor \u003calex.gaynor@gmail.com\u003e\n\nCo-authored-by: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "e022476498e6a3cc23ba4540dab98378eafc5d74", "tree": "8aec5bcc4b5d464b16c42cd2473ef6ddeb011118", "parents": [ "8cacafd07a27666e0473af16d0218cd5527d0a2a" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Sep 14 11:45:38 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 14 11:45:38 2026" }, "message": "flate: restructure matchLen to avoid bounds checks (#1216)\n\nBackport of https://go.dev/cl/829564 (golang/go#81398).\n\nThe load-helper half of the upstream CL does not apply here:\ninternal/le\u0027s Load32/Load64 already use unsafe pointer arithmetic on\namd64, arm64, ppc64le and riscv64, so there are no per-byte bounds\nchecks to remove, and the !unsafe fallback goes through encoding/binary,\nwhich already compiles to a single combined load.\n\nOnly the matchLen restructuring is carried over: truncate b to the\nlength of a up front and reslice both as the comparison advances, so\nthe loop condition proves every load in bounds and the byte-at-a-time\ntail loses its bounds check on b[i].\n\nThe output is byte-identical.\n\nAs predicted from the above, this is performance neutral on the default\nbuild, where the loads are already unchecked: geomean +0.1%, every\nbenchmark within noise. The table below is the default build.\n\nUnder -tags nounsafe, where the loads really are bounds checked, there\nis a small consistent win at the levels that call matchLen hardest\n(15 interleaved rounds):\n\n```\n │ master │ cl-829564 │\n │ sec/op │ sec/op vs base │\nPR/Digits/Huffman-32 747.7µ ± 1% 749.5µ ± 1% ~ (p\u003d0.539 n\u003d15)\nPR/Digits/L1-32 5.027m ± 1% 5.048m ± 1% ~ (p\u003d0.713 n\u003d15)\nPR/Digits/L4-32 5.818m ± 2% 5.812m ± 2% ~ (p\u003d0.345 n\u003d15)\nPR/Digits/L6-32 8.496m ± 1% 8.399m ± 1% -1.15% (p\u003d0.006 n\u003d15)\nPR/Digits/L7-32 14.61m ± 4% 14.48m ± 3% -0.88% (p\u003d0.008 n\u003d15)\nPR/Digits/L9-32 32.43m ± 2% 31.63m ± 4% -2.46% (p\u003d0.041 n\u003d15)\nPR/Twain/Huffman-32 971.9µ ± 1% 969.7µ ± 2% ~ (p\u003d0.683 n\u003d15)\nPR/Twain/L1-32 4.873m ± 2% 4.904m ± 2% ~ (p\u003d0.744 n\u003d15)\nPR/Twain/L4-32 6.184m ± 1% 6.206m ± 1% ~ (p\u003d0.744 n\u003d15)\nPR/Twain/L6-32 8.284m ± 1% 8.141m ± 2% -1.73% (p\u003d0.009 n\u003d15)\nPR/Twain/L7-32 17.46m ± 1% 17.29m ± 1% ~ (p\u003d0.098 n\u003d15)\nPR/Twain/L9-32 36.27m ± 3% 36.02m ± 1% -0.70% (p\u003d0.026 n\u003d15)\ngeomean 6.987m 6.948m -0.56%\n```\n\n```\ngoos: windows\ngoarch: amd64\npkg: github.com/klauspost/compress/flate\ncpu: AMD Ryzen 9 9950X 16-Core Processor\n │ master │ cl-829564 │\n │ sec/op │ sec/op vs base │\nPR/Digits/Huffman-32 804.3µ ± 1% 807.0µ ± 1% ~ (p\u003d0.653 n\u003d15)\nPR/Digits/L1-32 4.731m ± 1% 4.704m ± 0% ~ (p\u003d0.116 n\u003d15)\nPR/Digits/L4-32 5.518m ± 2% 5.488m ± 2% ~ (p\u003d0.567 n\u003d15)\nPR/Digits/L6-32 7.879m ± 5% 7.840m ± 2% ~ (p\u003d0.285 n\u003d15)\nPR/Digits/L7-32 14.54m ± 3% 14.70m ± 1% ~ (p\u003d0.902 n\u003d15)\nPR/Digits/L9-32 32.37m ± 3% 32.33m ± 3% ~ (p\u003d0.595 n\u003d15)\nPR/Twain/Huffman-32 1.051m ± 3% 1.055m ± 1% ~ (p\u003d1.000 n\u003d15)\nPR/Twain/L1-32 4.726m ± 2% 4.734m ± 1% ~ (p\u003d0.838 n\u003d15)\nPR/Twain/L4-32 5.944m ± 1% 5.932m ± 1% ~ (p\u003d0.486 n\u003d15)\nPR/Twain/L6-32 7.773m ± 3% 7.768m ± 1% ~ (p\u003d0.713 n\u003d15)\nPR/Twain/L7-32 17.70m ± 2% 17.66m ± 2% ~ (p\u003d0.539 n\u003d15)\nPR/Twain/L9-32 36.68m ± 1% 36.53m ± 1% ~ (p\u003d0.539 n\u003d15)\ngeomean 6.898m 6.894m -0.05%\n\n │ B/s │ B/s vs base │\nPR/Digits/Huffman-32 1.158Gi ± 1% 1.154Gi ± 1% ~ (p\u003d0.653 n\u003d15)\nPR/Digits/L1-32 201.6Mi ± 1% 202.7Mi ± 0% ~ (p\u003d0.116 n\u003d15)\nPR/Digits/L4-32 172.8Mi ± 2% 173.8Mi ± 2% ~ (p\u003d0.560 n\u003d15)\nPR/Digits/L6-32 121.0Mi ± 5% 121.6Mi ± 2% ~ (p\u003d0.281 n\u003d15)\nPR/Digits/L7-32 65.58Mi ± 3% 64.86Mi ± 1% ~ (p\u003d0.902 n\u003d15)\nPR/Digits/L9-32 29.47Mi ± 3% 29.50Mi ± 3% ~ (p\u003d0.588 n\u003d15)\nPR/Twain/Huffman-32 907.7Mi ± 3% 904.4Mi ± 1% ~ (p\u003d1.000 n\u003d15)\nPR/Twain/L1-32 201.8Mi ± 2% 201.4Mi ± 1% ~ (p\u003d0.838 n\u003d15)\nPR/Twain/L4-32 160.4Mi ± 1% 160.8Mi ± 1% ~ (p\u003d0.486 n\u003d15)\nPR/Twain/L6-32 122.7Mi ± 3% 122.8Mi ± 1% ~ (p\u003d0.691 n\u003d15)\nPR/Twain/L7-32 53.87Mi ± 2% 54.00Mi ± 2% ~ (p\u003d0.539 n\u003d15)\nPR/Twain/L9-32 26.00Mi ± 1% 26.10Mi ± 1% ~ (p\u003d0.506 n\u003d15)\ngeomean 138.3Mi 138.3Mi +0.05%\n```\n\nOriginal-Author: Alex Gaynor \u003calex.gaynor@gmail.com\u003e\n\nCo-authored-by: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "8cacafd07a27666e0473af16d0218cd5527d0a2a", "tree": "0b3e202e2177d9c593b09fdb3641b6f3dbb003bd", "parents": [ "9b2eb1bd7fa42a4c923a653b69edd5830b668706" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Sep 14 11:44:41 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 14 11:44:41 2026" }, "message": "flate: speed up writeTokens (#1218)\n\nBackport of https://go.dev/cl/829565 (golang/go#81398).\n\nRework the token writing loop to flush whole bytes from the bit\naccumulator unconditionally after every iteration instead of testing\nwhether 48 bits have accumulated after every code. The old test was a\ndata dependent, poorly predicted branch; the new store, shift and add\nare branch free. Since at most 7 bits are pending at the top of each\niteration, up to three literals (max 45 bits) or a complete match\n(max 48 bits) fit into the accumulator without intermediate flushes.\n\nLength codes and their extra bits, and offset codes and their extra\nbits, are now merged into the accumulator with a single shift/or each.\nA combined length table replaces three separate table lookups per\nmatch, and offsetCombined now covers all offset codes so the extra bit\ncheck can go away. lengthCode, lengthExtraBitsMinCode and\noffsetExtraBitsMinCode are no longer used and are removed.\n\nThe output buffer is now written with unconditional 8-byte stores, so\ntie its size to bufferFlushSize and document the slack that is needed.\nThe size is unchanged at 264 bytes, which is already the required\nbufferFlushSize+7+8 rounded up to a multiple of 8.\n\nThe output is byte-identical.\n\nNOTE on the Huffman-only rows below (PR/*/Huffman): those two numbers are a\ncode-layout artifact, not a real improvement, and should be read as 0%.\nwriteTokens is never executed at level -2 -- storeHuff calls only\nwriteBlockHuff, and writeTokens has zero profile samples there. This CL\nshrinks writeTokens from 2112 to 1472 bytes, which pulls writeBlockHuff 416\nbytes earlier in .text and changes its 64-byte alignment. writeBlockHuff\u0027s\ndisassembly is instruction-identical before and after, and the\nhuffmanBitWriter layout is unchanged (1360 bytes, same field offsets), so the\ndelta is purely placement. Inserting dead, never-called padding functions\nbefore writeBlockHuff on master reproduces the whole effect with no source\nchange on the Huffman path; applying only this CL\u0027s table changes while\nkeeping the old writeTokens gives -0.17%.\n\nExcluding those two rows, the geomean is +3.4% rather than +4.6%. The level\n1-7 wins are real: writeTokens is on those paths.\n\nHuffman-only throughput in this package swings 5-10% with code placement, so\ntreat any single-build Huffman-only measurement here below ~10% as unreliable.\n\n```\ngoos: windows\ngoarch: amd64\npkg: github.com/klauspost/compress/flate\ncpu: AMD Ryzen 9 9950X 16-Core Processor\n │ master │ cl-829565 │\n │ sec/op │ sec/op vs base │\nPR/Digits/Huffman-32 804.3µ ± 1% 742.5µ ± 1% -7.69% (p\u003d0.000 n\u003d15)\nPR/Digits/L1-32 4.731m ± 1% 4.460m ± 0% -5.74% (p\u003d0.000 n\u003d15)\nPR/Digits/L4-32 5.518m ± 2% 5.223m ± 1% -5.35% (p\u003d0.000 n\u003d15)\nPR/Digits/L6-32 7.879m ± 5% 7.565m ± 2% -3.99% (p\u003d0.000 n\u003d15)\nPR/Digits/L7-32 14.54m ± 3% 14.42m ± 1% -0.82% (p\u003d0.029 n\u003d15)\nPR/Digits/L9-32 32.37m ± 3% 32.12m ± 1% -0.76% (p\u003d0.045 n\u003d15)\nPR/Twain/Huffman-32 1050.6µ ± 3% 992.1µ ± 2% -5.57% (p\u003d0.000 n\u003d15)\nPR/Twain/L1-32 4.726m ± 2% 4.342m ± 2% -8.12% (p\u003d0.000 n\u003d15)\nPR/Twain/L4-32 5.944m ± 1% 5.595m ± 2% -5.87% (p\u003d0.000 n\u003d15)\nPR/Twain/L6-32 7.773m ± 3% 7.385m ± 2% -5.00% (p\u003d0.000 n\u003d15)\nPR/Twain/L7-32 17.70m ± 2% 17.14m ± 2% -3.20% (p\u003d0.015 n\u003d15)\nPR/Twain/L9-32 36.68m ± 1% 36.50m ± 1% ~ (p\u003d0.412 n\u003d15)\ngeomean 6.898m 6.593m -4.41%\n\n │ B/s │ B/s vs base │\nPR/Digits/Huffman-32 1.158Gi ± 1% 1.254Gi ± 1% +8.33% (p\u003d0.000 n\u003d15)\nPR/Digits/L1-32 201.6Mi ± 1% 213.8Mi ± 0% +6.09% (p\u003d0.000 n\u003d15)\nPR/Digits/L4-32 172.8Mi ± 2% 182.6Mi ± 1% +5.65% (p\u003d0.000 n\u003d15)\nPR/Digits/L6-32 121.0Mi ± 5% 126.1Mi ± 2% +4.15% (p\u003d0.000 n\u003d15)\nPR/Digits/L7-32 65.58Mi ± 3% 66.12Mi ± 1% +0.81% (p\u003d0.029 n\u003d15)\nPR/Digits/L9-32 29.47Mi ± 3% 29.69Mi ± 1% +0.74% (p\u003d0.047 n\u003d15)\nPR/Twain/Huffman-32 907.7Mi ± 3% 961.3Mi ± 2% +5.90% (p\u003d0.000 n\u003d15)\nPR/Twain/L1-32 201.8Mi ± 2% 219.6Mi ± 2% +8.83% (p\u003d0.000 n\u003d15)\nPR/Twain/L4-32 160.4Mi ± 1% 170.4Mi ± 2% +6.23% (p\u003d0.000 n\u003d15)\nPR/Twain/L6-32 122.7Mi ± 3% 129.1Mi ± 2% +5.26% (p\u003d0.000 n\u003d15)\nPR/Twain/L7-32 53.87Mi ± 2% 55.66Mi ± 2% +3.31% (p\u003d0.015 n\u003d15)\nPR/Twain/L9-32 26.00Mi ± 1% 26.13Mi ± 1% ~ (p\u003d0.418 n\u003d15)\ngeomean 138.3Mi 144.6Mi +4.62%\n```\n\nOriginal-Author: Alex Gaynor \u003calex.gaynor@gmail.com\u003e\n\nCo-authored-by: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "9b2eb1bd7fa42a4c923a653b69edd5830b668706", "tree": "e509577fa32ab7f04481b9263fa750efae11fe60", "parents": [ "32dbb2c4038c1aef13849bd51f6bc34deaa4ea5c" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Sep 14 11:43:48 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 14 11:43:48 2026" }, "message": "flate: combine codes in the writeBlockHuff literal loop (#1220)\n\nBackport of https://go.dev/cl/829567 (golang/go#81398).\n\nThroughput of the literal loop is bounded by the serial dependency\nthrough the pending bit count: every code is shifted by the current\ncount and then the count is updated. Assemble the three codes of an\niteration into a separate word first, which only depends on the codes\nthemselves, and merge that word into the accumulator with a single\nshift and or. The flushing structure of the loop is unchanged.\n\nUpstream notes that an earlier version also replaced the conditional\nflush with an unconditional 8-byte store after every group of three\ncodes, which was a further 6% faster on an Apple M1 Max but 12% slower\nthan the original loop on an Intel Xeon E5-2696 v4, so it is not used\nhere either.\n\nThis only affects Huffman-only encoding and blocks without matches.\nThe output is byte-identical.\n\n```\ngoos: windows\ngoarch: amd64\npkg: github.com/klauspost/compress/flate\ncpu: AMD Ryzen 9 9950X 16-Core Processor\n │ master │ cl-829567 │\n │ sec/op │ sec/op vs base │\nPR/Digits/Huffman-32 804.3µ ± 1% 703.6µ ± 2% -12.52% (p\u003d0.000 n\u003d15)\nPR/Digits/L1-32 4.731m ± 1% 4.717m ± 1% ~ (p\u003d0.838 n\u003d15)\nPR/Digits/L4-32 5.518m ± 2% 5.490m ± 1% ~ (p\u003d0.389 n\u003d15)\nPR/Digits/L6-32 7.879m ± 5% 7.851m ± 1% ~ (p\u003d0.539 n\u003d15)\nPR/Digits/L7-32 14.54m ± 3% 14.69m ± 2% ~ (p\u003d0.902 n\u003d15)\nPR/Digits/L9-32 32.37m ± 3% 32.38m ± 3% ~ (p\u003d1.000 n\u003d15)\nPR/Twain/Huffman-32 1050.6µ ± 3% 941.5µ ± 1% -10.39% (p\u003d0.000 n\u003d15)\nPR/Twain/L1-32 4.726m ± 2% 4.738m ± 2% ~ (p\u003d0.744 n\u003d15)\nPR/Twain/L4-32 5.944m ± 1% 5.903m ± 2% ~ (p\u003d0.683 n\u003d15)\nPR/Twain/L6-32 7.773m ± 3% 7.808m ± 2% ~ (p\u003d0.713 n\u003d15)\nPR/Twain/L7-32 17.70m ± 2% 17.52m ± 1% ~ (p\u003d0.436 n\u003d15)\nPR/Twain/L9-32 36.68m ± 1% 36.74m ± 2% ~ (p\u003d0.967 n\u003d15)\ngeomean 6.898m 6.754m -2.09%\n\n │ B/s │ B/s vs base │\nPR/Digits/Huffman-32 1.158Gi ± 1% 1.324Gi ± 2% +14.31% (p\u003d0.000 n\u003d15)\nPR/Digits/L1-32 201.6Mi ± 1% 202.2Mi ± 1% ~ (p\u003d0.814 n\u003d15)\nPR/Digits/L4-32 172.8Mi ± 2% 173.7Mi ± 1% ~ (p\u003d0.383 n\u003d15)\nPR/Digits/L6-32 121.0Mi ± 5% 121.5Mi ± 1% ~ (p\u003d0.532 n\u003d15)\nPR/Digits/L7-32 65.58Mi ± 3% 64.92Mi ± 2% ~ (p\u003d0.902 n\u003d15)\nPR/Digits/L9-32 29.47Mi ± 3% 29.45Mi ± 3% ~ (p\u003d1.000 n\u003d15)\nPR/Twain/Huffman-32 907.7Mi ± 3% 1013.0Mi ± 1% +11.59% (p\u003d0.000 n\u003d15)\nPR/Twain/L1-32 201.8Mi ± 2% 201.3Mi ± 2% ~ (p\u003d0.744 n\u003d15)\nPR/Twain/L4-32 160.4Mi ± 1% 161.6Mi ± 2% ~ (p\u003d0.683 n\u003d15)\nPR/Twain/L6-32 122.7Mi ± 3% 122.1Mi ± 2% ~ (p\u003d0.690 n\u003d15)\nPR/Twain/L7-32 53.87Mi ± 2% 54.43Mi ± 1% ~ (p\u003d0.442 n\u003d15)\nPR/Twain/L9-32 26.00Mi ± 1% 25.96Mi ± 2% ~ (p\u003d0.927 n\u003d15)\ngeomean 138.3Mi 141.2Mi +2.13%\n```\n\nOriginal-Author: Alex Gaynor \u003calex.gaynor@gmail.com\u003e\n\nCo-authored-by: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "32dbb2c4038c1aef13849bd51f6bc34deaa4ea5c", "tree": "7c2cb27c5d2fa5915ee7a49d795efc9bc96c92d2", "parents": [ "7dc45b1e13a2bd5414d1fc3c9a75077d310c337f" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Sep 14 08:27:57 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 14 08:27:57 2026" }, "message": "huff0: regenerate the Decompress1X asm loop with a branch-free refill (#1210)\n\n* huff0: reshape the Decompress1X asm loop around the sentinel bit container\n\nGenerate the single-stream loop from the same sentinel-container decode\nand reload that the Decompress4X loops use: the consumed count is the\ncontainer\u0027s trailing zero count, so per symbol the loop is a shift, a\ntable load, a shift and a store (5 instructions on both arches, down\nfrom 7 plus two fill branches and a byte swap per four symbols), and\nthe reload is branch-free. Three variants decode 5, 7 or 14 symbols per\nreload for tablelog 9..11, 5..8 and \u003c\u003d 4, sharing the 4X constants\n(renamed fastSymbols, fast8bSymbols, fast4bSymbols).\n\nOne stream, so the container, input pointer, output pointer and both\nloop limits stay in registers; the outer loop bounds a batch by the\noutput capacity and by the distance to the stream start exactly as the\n4X loop does, and the Go tail drains the rest. The asm no longer\nreports ErrMaxDecodedSizeExceeded itself: it stops at the capacity and\nthe Go tail reports the error when bits remain, so the error is the\nsame as before. The hand-off reuses canUseAsm, prepareForAsm and\nrestoreFromAsm.\n\nTests: the guard-byte corrupt-input test and the stream-ends-in-0x01\ntest now run over both one and four streams, with the small output\nsizes the 1X entry threshold and batch rounding produce, and the 1X\nroundtrips gain a BMI2-disabled twin.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* huff0: refill the 1X container from below the window instead of a sentinel reload\n\nA single stream is one serial dependency chain (peek, table load,\nshift) with no other stream to hide a reload behind, and the sentinel\nreload (trailing zero count, pointer arithmetic, load, OR, shift) sat on\nthat chain once per group. On Sapphire Rapids that regressed tablelog\n9..11 inputs by 8-14% while still winning on N1.\n\nKeep the consumed count in its own register instead, as the low byte of\na running sum of whole table entries (the symbol halves never carry into\nit), and refill without a branch: OR the consumed whole-byte bits back\nin from the 8 bytes below the window with every shift count computed\nfrom the count alone, so the container chain sees a single OR. The\nright shift by 64-r is two shifts (1, then 63-r, which is 63^r) so that\nr \u003d\u003d 0 contributes nothing. The container invariant is then the Go bit\nreader\u0027s own, so the asm stores off, value and bitsRead directly and\nrestoreFromAsm is not needed for 1X. The 4X code goes back to exactly\nits #1209 form apart from the shared constant names.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* huff0: share the output-bound shift between the 1X and 4X generators\n\nGenerated assembly unchanged. Addresses the CodeRabbit nit on #1210.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e" }, { "commit": "7dc45b1e13a2bd5414d1fc3c9a75077d310c337f", "tree": "54f997e124a24a43d381c778569c9619750bbe51", "parents": [ "b8824569b9662ff2e09e05c68bc6461485177e61" ], "author": { "name": "Jille Timmermans", "email": "jille@hexon.nl", "time": "Fri Sep 11 11:26:55 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Sep 11 11:26:55 2026" }, "message": "s2: Swap s2.Index.appendTo() argument order to match add() (#1213)\n\nI chose add()\u0027s order as the winning order because it aligns with the\nreturn values of Find()." }, { "commit": "b8824569b9662ff2e09e05c68bc6461485177e61", "tree": "a7393a69f12f8fd816f6a2b17c02427cfbe6f882", "parents": [ "e3d11eb15a80d012f33ed6d43a02d228fd9bdf8d" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Fri Sep 11 09:31:21 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Sep 11 09:31:21 2026" }, "message": "huff0: reshape the Decompress4X asm loops around a sentinel bit container (#1209)\n\n* huff0: reshape the Decompress4X asm loops around a sentinel bit container\n\nRewrite the 4X main loops in the shape of zstd\u0027s HUF_decompress4X1 fast\nloop. Each of the four streams keeps its bit container and output pointer\nin a register across iterations; only the input pointers stay in memory,\ntouched once per reload. A sentinel bit sits just below the unread bits,\nso the consumed count is a trailing-zero count and there is no bitsRead\nregister, no per-symbol fill branch, and no AL/AH byte packing of the\noutput (which lowered to BFI chains on arm64). Five symbols per stream are\ndecoded between reloads for tablelog 9..11 and seven for tablelog \u003c\u003d 8;\nthe reload is branch-free. Bounds are hoisted to an outer loop that bounds\niterations by both the output limit and the lowest input pointer, so every\nread stays inside the block even for corrupt input.\n\nThe inner decode is 5 instructions per symbol on both amd64 and arm64\nagainst roughly 9-11 before, and there is now a BMI2 twin on amd64.\n\nThe hand-off back to the Go tail loop is the delicate part. The asm returns\nthe sentinel container and a signed distance from the stream start; a\nstream can legitimately end with its 8-byte window up to 7 bytes below its\nown start (those bytes belong to the previous stream), and the sentinel is\nORed over the window\u0027s lowest data bit, so restoreFromAsm re-reads the\nwindow from memory rather than trusting the register, exactly as reference\nzstd does. prepareForAsm slides the window down a byte for streams whose\nfinal byte is 0x01, which would otherwise start with 8 consumed bits and\nlet a full group shift the sentinel out.\n\nReloads use TZCNT in both twins: avo models BSF\u0027s destination as\nread-write (undefined on zero input), which makes the temporary live\nacross the whole loop and overflows the register budget. TZCNT is a pure\nwrite, and for the non-zero input the sentinel guarantees it runs as BSF\non pre-BMI1 CPUs.\n\nTests: unit tests for the two conversions, a no-BMI2 twin of the 4X\nroundtrip test, and a no-BMI2 leg in FuzzCompress.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* huff0: add a 14-symbol 4X loop variant for tablelog \u003c\u003d 4\n\nWith one bit per symbol the fixed reload cadence of the 7-symbol loop\nreloads every 7 bits where the old loop filled every 32, which showed up\nas +4.7% on the two lowest-entropy benchmark cells on Sapphire Rapids (it\nwas still -39% on Neoverse N1). 14 symbols of at most 4 bits fit the same\n56-bit budget, so this is the same generator with a third symbol count.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* huff0: keep the 4X loops\u0027 input pointers in explicit context fields\n\nGo fills ctx.ip[i] and ctx.ilowest from the bit reader slices, the asm\nadvances ip, and off is written once in the epilogue as the signed\ndistance from the stream start. The loop keeps the context pointer as its\nmemory base instead of the reader array, which also removes the reload of\nthe context pointer from the stack on every inner iteration. No field\nchanges meaning while the call is in flight.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* huff0: bound 4X asm reads by the block start, not stream 0\u0027s start\n\nThe window may reach into the 6-byte jump table below stream 0, which is\ninside src and already handled by restoreFromAsm as consumed bytes. This\nkeeps the asm running about six bytes longer per stream, which for streams\nwith very short codes (pngdata at ~1.1 bits/symbol, tablelog 1 inputs)\nshrinks the Go tail from ~115 to ~70 symbols per stream and removes a\n+9% regression on 10 KB inputs on Sapphire Rapids.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* huff0: bound the 4X asm output batch by the symbols per iteration\n\nThe outer loop divided the remaining output budget by 8 to get the number\nof inner iterations it could run before re-checking bounds. That was only\nvalid while an iteration wrote at most 8 bytes per stream, which the\n14-symbol tablelog \u003c\u003d 4 loop broke: a batch could advance the output\npointers 1.75x past the limit, and with corrupt input (streams whose read\npointers stay high, so the input bound never fires) stream 3 wrote past\nthe end of the output buffer before the Go tail reported the corruption.\nThrough zstd\u0027s literal buffer that is a heap overrun from untrusted input.\nFound by review; valid input was not observed to trigger it but was not\nproven safe either.\n\nDivide the output budget by the next power of two at or above the symbol\ncount instead (5 and 7 -\u003e 8, 14 -\u003e 16), and make the generator panic if a\nfuture symbol count does not fit its shift. Add a guard-byte test that\ndecodes non-draining random streams under tables of every size into\nbuffers with 4 KiB of canary after the capacity, on both amd64 twins; it\nfails on the previous assembly at +0 and passes now.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* huff0: guard-byte roundtrips, a deterministic 0x01-marker case, comment fixes\n\nFrom review of the branch: a canary-checked Decompress4X helper, valid\nroundtrips under every table size with the output capacity exact (the\nallocator\u0027s size classes otherwise hide small overruns from every other\ntest), an end-to-end case for streams whose final byte is 0x01 so the\nwindow slide in prepareForAsm is exercised through the public API, a note\non the generic twins that avo\u0027s \"Requires: BMI\" stamp comes from TZCNT\nwhich runs as BSF without BMI1, and a pointer from the Go-side limit\ncomment to where the batch bound lives.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* huff0: separate the 4X asm entry check from the window slide\n\nReview nit: with the four prepareForAsm calls chained by \u0026\u0026, a late false\nleft the earlier readers already slid (a valid but surprising state).\ncanUseAsm is now a pure check and prepareForAsm only runs once all four\nreaders qualify.\n\nAlso cap the corrupt-input test\u0027s stream lengths at 65535: the jump table\nstores 16-bit lengths, so the 65536 and 262143 cases were encoding a zero\nlength and never reached the assembly.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "e3d11eb15a80d012f33ed6d43a02d228fd9bdf8d", "tree": "82d554554b45da839e39ecd605c8d48cd873bf18", "parents": [ "5e17c92e03c6df623f06797f61cda232661cb131" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Fri Sep 11 08:40:51 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Sep 11 08:40:51 2026" }, "message": "zstd: account for raw and RLE blocks in maxSyncLen (#1211)\n\n* zstd: account for raw and RLE blocks in maxSyncLen\n\nmaxSyncLen bounds how much of the frame is still to come, and\ndecodeSyncSimple compares it with the output buffer\u0027s slack to decide\nwhether the extended 16-byte copies are safe. decodeCompressed subtracts\neach compressed block\u0027s output from it, but decodeBuf\u0027s raw and RLE\ncases did not. A frame that opened with incompressible data therefore\noverstated its remaining size for every block after the raw ones, the\nslack check failed, and the rest of the frame ran the bounds-exact\n\"safe\" decodeSync variant.\n\nSubtract those blocks\u0027 sizes too, landing on zero (the \"no bound\"\nvalue, which selects the conservative paths) if a block meets or\nexceeds the bound; runDecoder\u0027s frame-size checks reject the excess\nafterwards.\n\nOn a Neoverse N1 (oracle), DecodeAll of 70 MB frames whose first 4 MiB\nis random data got 17%, 11% and 7% faster for match offsets in the\n32 KB, 768 KB and 2 MB ranges, entirely from the fast copy variant\nbeing selected again. Frames without raw blocks are unaffected.\n\nThe test walks an encoder-produced frame (raw blocks, then compressed)\nand a hand-built one (raw, then RLE) block by block with DecodeAll\u0027s\nbuffer geometry and checks after each block that maxSyncLen equals the\nbytes still to produce; it fails on the first raw block without the fix.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* zstd: account for literal-only compressed blocks in maxSyncLen too\n\nA compressed block with no sequences returns from decodeCompressed\nbefore the bookkeeping that subtracts its output, so it left maxSyncLen\noverstated the same way raw and RLE blocks did. Subtract its literals\nas well, and add a hand-built raw-then-literals-only frame to the test.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e" }, { "commit": "5e17c92e03c6df623f06797f61cda232661cb131", "tree": "48fb04e2b75b12b8bebed916d4114d40dad88b7b", "parents": [ "fc272ea38376e11bba4d99ddce3035063e99cdf7" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Wed Sep 09 07:56:34 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 09 07:56:34 2026" }, "message": "huff0: seed FuzzDecompress1x with real compressed output, exercise Decompress4X (#1208)\n\n## Summary\n\nSmall rebase of the still-open #893. Most of that PR has already shipped independently: `internal/fuzz.ReturnFromZip` is already on master and is used by `s2/fuzz_test.go` to seed decoder fuzzers from real encoder output, and `FuzzCompress` already round-trips through `Compress1X`/`Decompress1X`/`Compress4X`/`Decompress4X`. What\u0027s left, and still missing on master:\n\n- `FuzzCompress` never seeds `testdata/regression.zip` (it still exists in `huff0/testdata/`, just unused).\n- `FuzzDecompress1x` only ever saw whatever `ReadTable` happened to accept out of a raw zip, and never exercised `Decompress4X` at all — unlike the s2 decoder fuzzers, which seed from real compressor output via `ReturnFromZip`.\n\nThis applies that same `addCompressed`-via-`ReturnFromZip` pattern to `huff0/fuzz_test.go`, and adds `Decompress4X` to the fuzzed paths.\n\n## Test plan\n- [x] `go build ./...`, `go vet ./huff0/...`\n- [x] `go test ./huff0/...` (seed corpus + existing tests pass)\n- [x] `go test -fuzz\u003dFuzzDecompress1x -fuzztime\u003d20s ./huff0` — no crashes, 112 new coverage-interesting inputs found (confirms the new `Decompress4X` path is reached)\n- [x] `go test -fuzz\u003dFuzzCompress -fuzztime\u003d20s ./huff0` — no crashes, 15 new coverage-interesting inputs found\n\n🤖 Generated with [Claude Code](https://claude.com/claude-code)\n\n\u003c!-- This is an auto-generated comment: release notes by coderabbit.ai --\u003e\n\n## Summary by CodeRabbit\n\n* **Tests**\n * Expanded compression and decompression fuzz testing with inputs from existing regression and compressed-data corpora.\n * Added coverage for well-formed compression tables and payloads across supported compression modes.\n * Simplified decompression fuzz execution to exercise decoder behavior regardless of returned errors or output.\n\n\u003c!-- end of auto-generated comment: release notes by coderabbit.ai --\u003e" }, { "commit": "fc272ea38376e11bba4d99ddce3035063e99cdf7", "tree": "b973a9d1241a755b655b18d67f7f8774d61abd33", "parents": [ "ac559c14b27ccf8fd3f96cc6fde6c749e1ab1ed5" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Tue Sep 08 09:06:33 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 08 09:06:33 2026" }, "message": "zstd: copy the first block of short literal runs and matches unconditionally (#1204)\n\n* zstd: copy the first block of short literal runs and matches unconditionally\n\nMeasured on the benchdecoder corpus, 68% of sequences carry no literals,\n99% carry at most 16, and 92% of matches are at most 16 bytes. The\nextended (unsafe) copy paths tested the literal length for zero before\nentering a 16-byte block loop, and ran the match copy as a block loop with\na data-dependent exit. The zero test is taken for two sequences in three on\ntext, so it is a poorly predicted branch, and each block loop\u0027s exit\nbranch flips with the length.\n\nBoth copies now write their first 16-byte block unconditionally and\nbranch only for the long tail (length \u003e 16), which the predictors see as a\nrarely-taken branch. An empty literal run copies one block of slack that\nthe following match copy overwrites; the literal buffer carries\ncompressedBlockOverAlloc readable bytes past its end and the output the\nsame slack past ll+ml, both already required for the unsafe variants\n(useSafeDecodeSync, executeSimple), so no allocation contract changes.\nThe safe variants are untouched. The generic and BMI2 twins share this\ncode, so amd64 and arm64 both change.\n\nDecodeAll on the benchdecoder text files (default level, PCALIGN-pinned\nper-function alignment, -count 8, sec/op):\n\n Neoverse N1 (oracle): Tom Sawyer -10.7%, html -8.0%, pi -7.2%, e -6.9%;\n small files and incompressible data within ±0.7%.\n Sapphire Rapids (c7i): Tom Sawyer -23.3%, pi -8.8%, e -8.7%, html +1.0%;\n everything else within ±0.8%.\n Granite Rapids (c8i): Tom Sawyer -20.9%, pi -8.4%, e -8.3%, html -1.3%;\n everything else within ±0.7%.\n (AWS legs: -count 10 -cpu 1.)\n\nBenchmark_seqdec_decodeSync moves within ±1% on its large blocks except\none at -11% (N1) / -17% (both Xeons); those blocks are literal-heavy (a\nliteral run on nearly every sequence), so the zero test was already well\npredicted there. Its small\nblocks never reach the assembly (they run the Go fallback because the\nbenchmark allocates no output buffer) and shift only by code layout.\n\nTests: zstd and huff0 suites pass on amd64 (Zen 3, Sapphire Rapids,\nGranite Rapids) and arm64 (Neoverse N1). TestDecoderShortSequenceCopies\nround-trips inputs whose literal runs and matches sit at 0, 15, 16 and 17\nbytes through both the DecodeAll and streaming paths; with the match tail\nthreshold deliberately off by one it fails with a CRC error.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* zstd: describe the short-sequence copy test cases by how they are built\n\nReview feedback: the case names claimed exact sequence lengths, but the\nencoder decides the sequences; the inputs only steer them to cluster at\nthe boundary. Name the cases by their construction (word length and\nliteral-run length) and say in the doc comment what is and is not\nguaranteed, including that the clustering is validated by mutation (an\noff-by-one in the generator\u0027s match tail threshold fails the 17-byte\ncases) and that path selection is asserted by Test_seqdec_decodeSync.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* zstd: fold the short-sequence copy test into seqdec_test.go, rewrap a comment\n\nReview feedback: keep sequence decoder tests in seqdec_test.go rather than\nadding a file per feature, and restore the line wrap of the\nuseSafeDecodeSync comment.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e" }, { "commit": "ac559c14b27ccf8fd3f96cc6fde6c749e1ab1ed5", "tree": "f5fea6f806e82bb6c08e1f5da09458dde31699f3", "parents": [ "75489f96c542c09a1a1af296d68c3cbfe4d2bcc0" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Tue Sep 08 08:42:25 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 08 08:42:25 2026" }, "message": "zstd: stream a small buffer when its in-memory decode exceeds the size limit (#1207)\n\nDecoder.Reset decodes a source with Bytes and Len shorter than\nWithDecodeBuffersBelow in one go through DecodeAll, where\nWithDecoderMaxMemory caps the total decoded size. A Reader is documented\nwith the streaming contract, where the option caps the window instead,\nso the same Reader followed one rule or the other depending on input\nlength: a 1740-byte frame with no content size that expands past a\n20 MiB limit failed over a bytes.Buffer and decoded over any other\nio.Reader.\n\nWhen the shortcut\u0027s DecodeAll reports ErrDecoderSizeExceeded, fall\nthrough to the normal streaming decode of the same reader, whose bytes\nthe shortcut has not consumed. Memory stays bounded as documented: the\nin-memory attempt stops at the cap, its buffer is dropped, and streaming\nholds a window.\n\nFound by a differential fuzz harness while fuzzing #1204; the stock\nfuzzers share the threshold on both sides and cannot see it.\n\nFixes #1206\n\nCo-authored-by: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e" }, { "commit": "75489f96c542c09a1a1af296d68c3cbfe4d2bcc0", "tree": "af98410662d88955f02b40cfe972d13d2abaf87c", "parents": [ "dc3d00033ee4d52dc906b6e173d2983ffe513c1f" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Fri Sep 04 11:58:09 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Sep 04 11:58:09 2026" }, "message": "zstd: make the seqdec decodeSync test and benchmark reach the assembly (#1205)\n\n* zstd: make the seqdec decodeSync test and benchmark reach the assembly\n\ndecodeSyncSimple chooses the implementation from the output buffer\ngeometry: with less than a block of capacity it declines and the pure-Go\nloop runs, with exactly a block it runs the assembly with bounds-exact\ncopies, and with compressedBlockOverAlloc slack on top it runs the\nassembly with the extended 16-byte-block copies. Test_seqdec_decodeSync\nand Benchmark_seqdec_decodeSync built their sequenceDecs with a nil\noutput buffer, so both ran the Go loop: the test never covered the\nassembly, and the benchmark timed Go code (or, once append had grown the\nbuffer past a block on later iterations, whichever path the grown\ncapacity happened to select).\n\nThe test now runs each block through all three variants with a fresh\nbuffer of the selecting capacity and requires identical output, making\nthe Go loop the reference for both assembly variants. The benchmark\npre-allocates a block plus slack, the geometry DecodeAll provides, so it\ntimes the extended-copy assembly. Both reset prevOffset per run so every\nrun decodes the same sequences. Verified with panics planted in each\nbranch of decodeSyncSimple that the three variants and the benchmark\nreach the intended paths; under -tags noasm all three run the Go loop\nand the test still passes.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* zstd: assert which decodeSync implementation the test variants select\n\nReview feedback on the differential test: the variant names were only a\nclaim about buffer geometry. The test now asks decodeSyncSimple directly\nwhether it accepted the buffer and checks useSafeDecodeSync before\ndecoding (the answer depends on the capacity left in s.out), so a\ngeometry that does not select what the variant claims fails instead of\nsilently running the Go loop three times. A build-tagged pair of helpers\nlets builds without the assembly fall back to the Go loop. The baseline\nis taken from the first variant by index rather than by a nil check, so\nan empty first output would still be compared against.\n\nMutation-checked: shrinking the unsafe variant to a bare block fails on\n\"safe copies \u003d true, want false\", and giving the Go variant a block fails\non \"accepted a buffer\".\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e" }, { "commit": "dc3d00033ee4d52dc906b6e173d2983ffe513c1f", "tree": "6be9b42f76acf9b531921763bfd9bc7f1e7cbf84", "parents": [ "9d8ccb1d9567304420eb55a88b6f63a2067a8da4" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Fri Sep 04 10:25:19 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Sep 04 10:25:19 2026" }, "message": "zstd/huff0: branch-free arm64 decode reads, avo shift-copy folds, whole-register 4X exhausted counter (#1203)\n\n* zstd: branch-free length reads and fewer memory round trips in the asm decoders\n\nProduction arm64 profiles (Honeycomb\u0027s Lambda workload) put a quarter of\ndecompression time in sequenceDecs_decodeSync_arm64, spread over the\nthree extra-bits reads, the FSE state updates, and the loads and stores\nthat shuttle values through ctx and the frame. This reshapes the generic\n(non-BMI2) generator path, which is what the arm64 lowering starts from,\nwithout changing what any input decodes to.\n\nLength reads (updateLength) lose their three branches. The generic path\ntested addBits for zero and checked for overread before shifting;\nneither branch is data-independent, and the offset one flips with every\nrepeat-offset sequence. The read is now (value\u003c\u003cbitsRead)\u003e\u003e1\u003e\u003e(63-n),\nwhich is bitReader.get32BitsFast\u0027s value for every n the tables can\nhold (addBits is at most 30) and 0 for n \u003d\u003d 0, so it matches the Go\nreference and the BMI2 twin, which never carried the checks: the fill\nbefore each group of reads guarantees the bits, and a stream that runs\ndry is reported at the next fill. The FSE state reads (getBits) take the\nsame shape in place of a rotate and a (1\u003c\u003cn)-1 mask build.\n\nMemory traffic per sequence drops by two ctx dereferences per table\nload, two per iteration-counter access, and the seqSize read-modify-\nwrite: the table base pointers and the counter are copied to frame\nslots at entry (the counter is written back to ctx on every exit of the\ndecode variants, where the caller reads it to locate a failing\nsequence), and decodeSync recovers seqSize once at loop exit from the\noutPosition delta, which is exactly the sum of ll+ml. The copy step\nalso takes ll, mo and ml from the registers the value checks already\nhold instead of reloading the three slots.\n\nOn arm64 this removes 43 instructions from decodeSync\u0027s main loop as\ngenerated, more on the executed path, with the remaining copies of the\nshift count and source to be taken up by the lowering.\n\nValidated: go vet on amd64 and arm64; zstd tests on amd64 including the\nforced non-BMI2 runs; FuzzDecodeAll (5.4M execs, BMI2 versus generic\nbyte-identical) and FuzzDecAllNoBMI2 (9.2M execs); zstd tests on arm64\nunder qemu-user; and a digest of DecodeAll over all 10,091 corpus\nentries identical between amd64 asm, amd64 noasm and arm64 asm.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_015dheXiuzL7dbQM3hjjS4zm\n\n* zstd/huff0: bump avo, regenerate arm64 with the shift-copy folds\n\nPulls in honeycombio/avo@67bf513, whose arm64 lowering absorbs the\nregister copies x86\u0027s shift forms force: the count copy into CL and the\nsource copy before an in-place shift both disappear into arm64\u0027s\nthree-operand shifts where a straight-line scan proves the copy dead,\na SETcc into a freshly zeroed register becomes one CSET, and ADCQ $0\nlowers to a single CSINC.\n\namd64 output is byte-identical; only the arm64 files change, by exactly\nthe matched sites: every peekTopBits in both huff0 4X loops (three\ninstructions to one LSR, eight per iteration of the 16-bit loop), the\nexhausted-flag zero-and-set at the top of those loops, and the count and\nsource copies feeding the shifts in every zstd seqdec function and the\nFSE table builder.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_015dheXiuzL7dbQM3hjjS4zm\n\n* huff0: keep the 4X exhausted counter in a whole register\n\nThe four-stream decoders accumulate \"this stream ran dry\" in a byte:\nXORL to clear, SETGE into the low byte, ADCB $0 per stream, TESTB at the\nloop bottom. The count never exceeds five, so the full register serves\nas well, and the byte-lane forms cost on both sides: on x86 a partial\nwrite, on arm64 a bit-field insert after every CSET and CSINC because\nthe lowering must preserve the bits x86 would. With XORQ, ADCQ $0 and\nTESTQ (SETcc has no wider form, but the lowering now recognizes a SETcc\ninto a register it just saw zeroed) the arm64 loop head is a single\nCSET, each stream\u0027s carry add a single CSINC, and the loop-bottom test a\nplain TST: six instructions fewer per iteration.\n\nValidated: go vet on amd64 and arm64; huff0 tests on both (arm64 under\nqemu-user); FuzzCompress (91s) and FuzzDecompress1x (46s) on amd64.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_015dheXiuzL7dbQM3hjjS4zm\n\n---------\n\nCo-authored-by: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e" }, { "commit": "9d8ccb1d9567304420eb55a88b6f63a2067a8da4", "tree": "243e8bc20cf213b0efb7cdcb76fe118198990856", "parents": [ "f93f23a4bd5434ab55872aff9717f769398fa59e" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Wed Sep 02 12:08:18 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 02 12:08:18 2026" }, "message": "flate: Writer with a preset dictionary emits the dictionary into the output stream (#1201)\n\nBackport of https://github.com/golang/go/issues/80538\n\nAdds dictionary fuzz check. Status:\n\n```\nfuzz: elapsed: 2h12m45s, execs: 98125523 (22332/sec), new interesting: 152 (total: 3769)\nfuzz: elapsed: 2h12m48s, execs: 98203921 (26130/sec), new interesting: 152 (total: 3769)\n```" }, { "commit": "f93f23a4bd5434ab55872aff9717f769398fa59e", "tree": "cebd09790632e634be91c1ade5e86d3118d0d5d8", "parents": [ "42f9d969024a20735148e3577d4b602ed4d0590a" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Wed Sep 02 08:28:05 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 02 08:28:05 2026" }, "message": "build(deps): bump the github-actions group with 6 updates (#1200)\n\nBumps the github-actions group with 6 updates:\n\n| Package | From | To |\n| --- | --- | --- |\n| [actions/checkout](https://github.com/actions/checkout) | `7.0.0` | `7.0.1` |\n| [github/codeql-action/init](https://github.com/github/codeql-action) | `4.37.3` | `4.37.9` |\n| [github/codeql-action/autobuild](https://github.com/github/codeql-action) | `4.37.3` | `4.37.9` |\n| [github/codeql-action/analyze](https://github.com/github/codeql-action) | `4.37.3` | `4.37.9` |\n| [ossf/scorecard-action](https://github.com/ossf/scorecard-action) | `2.4.3` | `2.4.4` |\n| [github/codeql-action](https://github.com/github/codeql-action) | `4.36.2` | `4.37.9` |\n\n\nUpdates `actions/checkout` from 7.0.0 to 7.0.1\n- [Release notes](https://github.com/actions/checkout/releases)\n- [Commits](https://github.com/actions/checkout/compare/v7...v7.0.1)\n\nUpdates `github/codeql-action/init` from 4.37.3 to 4.37.9\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81...cdf488f595d80d6e07e03d4674febd5ab45fa938)\n\nUpdates `github/codeql-action/autobuild` from 4.37.3 to 4.37.9\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81...cdf488f595d80d6e07e03d4674febd5ab45fa938)\n\nUpdates `github/codeql-action/analyze` from 4.37.3 to 4.37.9\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81...cdf488f595d80d6e07e03d4674febd5ab45fa938)\n\nUpdates `ossf/scorecard-action` from 2.4.3 to 2.4.4\n- [Release notes](https://github.com/ossf/scorecard-action/releases)\n- [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md)\n- [Commits](https://github.com/ossf/scorecard-action/compare/v2.4.3...v2.4.4)\n\nUpdates `github/codeql-action` from 4.36.2 to 4.37.9\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/v4.36.2...v4.37.9)\n\n---\nupdated-dependencies:\n- dependency-name: actions/checkout\n dependency-version: 7.0.1\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n- dependency-name: github/codeql-action/init\n dependency-version: 4.37.9\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n- dependency-name: github/codeql-action/autobuild\n dependency-version: 4.37.9\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n- dependency-name: github/codeql-action/analyze\n dependency-version: 4.37.9\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n- dependency-name: ossf/scorecard-action\n dependency-version: 2.4.4\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n- dependency-name: github/codeql-action\n dependency-version: 4.37.9\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: github-actions\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "42f9d969024a20735148e3577d4b602ed4d0590a", "tree": "13f4246e32bd5288f3fa4701ca898116b9b5d7fe", "parents": [ "629c2ea148ae6ddbd3b70bc1e35076aaf94b7143" ], "author": { "name": "GOWTHAM MP", "email": "Gowthamrockerzzz@gmail.com", "time": "Mon Aug 31 13:48:05 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Aug 31 13:48:05 2026" }, "message": "xpress: add Microsoft XPRESS (MS-XCA) decompression (#1195)\n\n* xpress: add Microsoft XPRESS (MS-XCA) decompression\n\n* Fix XPRESS docs: WIM/WOF use the Huffman variant; raise block cap to 32 MiB\n\n* xpress: add fuzzer, fix two panics it found\n\nFuzzXpress runs fuzz input through both decompressors, seeded with the\nMS-XCA test corpus, and asserts output never exceeds the 32 MiB cap.\n\nFixes found by the fuzzer:\n- the Huffman decode table fill could run past the 32768 entries when\n code lengths were oversubscribed (the completeness check ran after the\n fill loop, too late)\n- the mid-stream end-of-data-as-match(3,1) fallback indexed out[-1] when\n the output was still empty\n\n* xpress: spec-correct plain-LZ77 decoding, append APIs, generated corpus\n\nReview follow-up on the XPRESS decompression PR:\n\n- Plain LZ77: fix the shared-nibble and raw-byte length bases to the\n MS-XCA 2.1.1 values (nibble + 10, byte + 25; LE16/LE32 stays + 3) and\n let the pending half-byte survive literals. Both were wrong in the\n initial port and only the self-consistent vectors masked it.\n- Reject LE16/LE32 length values below 22, as the specification does.\n- API: XpressDecompress/XpressHuffmanDecompress become\n AppendDecompressed/AppendHDecompressed, appending to a caller slice\n and returning it unmodified on error (klauspost review).\n- MAX_DECOMPRESSED_FILE becomes the exported MaxSize; errors are\n lowercase with an xpress: prefix.\n- Corpus: xpress_test_data.go is renamed xpress_data_test.go and\n regenerated by the new tests/xpressgen generator, which encodes with\n a spec-faithful MS-XCA 2.1.1 encoder and uses bytes.Repeat for the\n repetitive payloads. The Huffman fixtures are spliced in verbatim\n from huffman_vectors.txt. New vectors cover every nibble length\n 11..24, byte lengths 26..279, the LE32 path, a literal between two\n shared-nibble matches, and the append mode.\n\n* xpress: bound matches to this call\u0027s output and fix negative-test byte order\n\nAppendDecompressed and AppendHDecompressed now reject matches that would read before the caller-provided output base (previously the guards compared against the whole slice, letting a crafted stream copy from the prefix). The hand-built negative-test flag words were byte-reversed - little-endian storage put the match flags in the low bits, so TestXpressExpansionRatio and TestXpressOOB passed on the literal/truncated paths instead of the paths they claim to cover. The flag bytes now place match bits where the decoder looks (bit 31 down), assertions use errors.Is, and the append error path asserts the returned slice equals the original prefix. TestXpressTruncated pins the exact error for each case: mid-match cut (errTruncated), short Huffman table (errTruncated), and oversize declaration (errCorrupt via mid-stream end-of-data).\n\nGenerator: xpressgen output is now passed through go/format, and the match-search cap is raised from 65535 to 128 KiB so a corpus vector can exercise the plain LE32 length branch. A hand-built Huffman fixture (Huff7) covering the LE32 raw-length path is verified against the decoder and spliced into the corpus.\n\n* xpressgen: reserve the flag-group header when a match opens a group\n\nThe encoder reserved the four-byte flag-group header only in the literal path (when flagCount reaches 1), so a match emitted as the first token of a group (after 32 literals, or at the end of the stream) wrote its token bytes where the header belonged and the flags were written at a stale position - corrupting the stream. The match path now reserves the header when flagCount \u003d\u003d 0, mirroring the literal path. New vector Plain10 (32 distinct literals followed by a match) exercises the exact boundary and fails against the previous encoder.\n\n* xpress: address review feedback on tests and fixtures\n\n* xpress: apply go fix integer-range rewrites\n\n* Update xpress/xpress_test.go\n\nCo-authored-by: coderabbitai[bot] \u003c136622811+coderabbitai[bot]@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e\nCo-authored-by: coderabbitai[bot] \u003c136622811+coderabbitai[bot]@users.noreply.github.com\u003e" }, { "commit": "629c2ea148ae6ddbd3b70bc1e35076aaf94b7143", "tree": "5ad678208c4838e99bc2724d979ba35612ebf6df", "parents": [ "ba74977eb2c7002604765bf9c7f3c8a497224d60" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Aug 31 12:10:17 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Aug 31 12:10:17 2026" }, "message": "zstd/huff0: break false partial-register dependencies in seqdec/4X decoders (#1197)\n\n* huff0: break the false output-register dependency in the 4X decoders\n\nThe 4X decoders accumulate decoded symbols in byte lanes of a fixed\noutput register before one coalesced store. The first write used MOVB\ninto AL, which merges into RAX and so depends on whatever the register\nheld before: the previous stream\u0027s value, and across the loop back-edge\nthe previous iteration\u0027s. That serializes the four otherwise-independent\nbitstreams through one register. AL writes are not renamed separately on\nany current x86 core, and the arm64 lowering faithfully reproduces the\nmerge as UBFX into a scratch register plus BFI into the stale\ndestination, with the same chain. Production arm64 profiles show these\ninsert sequences carrying the largest flat sample counts in\ndecompress4x_main_loop_arm64.\n\nEmit a zero-extending MOVBLZX for the first byte instead, making it a\nfresh definition of the full register. The later AH/AL merges then\ndepend only on values produced in the same stream\u0027s block. The lowered\narm64 becomes a fresh-destination UBFX at the same eight sites (both 4X\nloops, all four streams); the lowering itself is unchanged.\n\nValidated: go vet on amd64 and arm64, huff0 tests with and without\nnoasm, zstd tests, 90s huff0 FuzzCompress roundtrip fuzz (1.5M execs),\n60s zstd FuzzDecodeAll differential fuzz (1.4M execs).\n\nCo-Authored-By: Claude Fable 5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01DyCv35JFWbF23UZax6qn2F\n\n* zstd: replace partial-register byte writes with MOVZBL in seqdec (non-BMI2)\n\nIn the offset/match/literal length updates, MOVB AH, CL relied on\nbrBitsRead \u003c\u003d 64 leaving CX\u0027s upper bits zero so the full register could\nbe read back as addBits; in the getBits mask build, MOVB nBits, CL merged\ninto the stale bitsRead sum. Both are partial-register merges: on x86\neach costs a merge uop depending on the stale full register, and the\narm64 lowering renders each as a UBFX+BFI or BFI read-modify-write pair,\nadding serial links next to the FSE state-update critical path that\ndominates the production decodeSync profile.\n\nMOVBLZX writes the full register fresh: unconditionally correct (no\nknown-bits assumption), same instruction count on amd64, and each site\nlowers to a single fresh-destination UBFX or MOVBU on arm64 - 24 sites\nacross the four non-BMI2 functions, dropping 12 instructions and all 24\nread-modify-write chain links from the lowered code. The BMI2 variants\nare byte-identical, so the default path on BMI2-capable amd64 is\nunchanged; the non-BMI2 amd64 variant and its arm64 lowering carry the\nchange.\n\nValidation: go vet and build on amd64+arm64; go test ./zstd with the\nassembly and with -tags\u003dnoasm; FuzzDecAllNoBMI2 2.8M execs (differential:\nthe changed non-BMI2 variant against the untouched BMI2 output,\nbyte-for-byte); FuzzDecodeAll 1.4M execs; BenchmarkDecoder_DecodeAll on\namd64 (BMI2 path) as a no-regression guard. arm64 performance to be\nmeasured on Neoverse hardware.\n\nCo-Authored-By: Claude Fable 5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01DyCv35JFWbF23UZax6qn2F\n\n* zstd: gofmt comment alignment in seqdec generator\n\nNo functional change; fixes build-special gofmt gate.\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01RjoGCCuDTpqAdghumgYWCB\n\n* zstd: fix false loop-carried dependency in overlapped-copy byte loop\n\ncopy_slow_3\u0027s byte-at-a-time overlapping-match copy read each source byte\ninto the low 8 bits of a GP64 declared once outside the loop (MOVB into a\npartial register), the same false-dependency shape #1197 fixed at every\nother site in this file (a partial-register write merges with the\nregister\u0027s stale value from the PREVIOUS iteration, serializing loads\nthat are otherwise independent). One site here was missed because it\u0027s a\nbyte-copy loop rather than a decode step.\n\nFix: MOVBQZX, a fresh full-register def, identical amd64 cost. On arm64\nthis also drops a now-unneeded BFI merge entirely (MOVBU writes the full\ndestination register directly), one instruction fewer per byte copied.\n\nFound via production pprof mining of the same profile that motivated\n#1197 and #1199: copy_slow_3 carries ~0.7-0.9% of decodeSync\u0027s flat CPU\ntime in Honeycomb\u0027s arm64 Lambda workload.\n\n---------\n\nCo-authored-by: Claude \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "ba74977eb2c7002604765bf9c7f3c8a497224d60", "tree": "ef2994fffe1ae99ae00fbb2d26fe38881460eb6f", "parents": [ "f5c1529bc7838087176f39eb85e2495f5292606d" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Aug 31 11:13:32 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Aug 31 11:13:32 2026" }, "message": "ci: test against go1.27.x, drop go1.24.x (#1198)\n\n* ci: test against go1.27.x, drop go1.24.x\n\ngo1.27.0 has shipped stable; per Go\u0027s two-latest-release support\npolicy the currently supported set is 1.26/1.27, so drop 1.24.x from\nthe CI matrix and add 1.27.x. go.mod\u0027s `go 1.24` minimum is left\nuntouched — that\u0027s a separate consumer-compatibility decision.\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01RjoGCCuDTpqAdghumgYWCB\n\n* ci: retrigger to check zlib ExampleNewWriter Windows/1.27 flake reproducibility\n\nNo functional change. build (1.27.x, windows-latest) failed on the\nprevious push with ExampleNewWriter producing a valid-but-different\n(stored vs Huffman-compressed) zlib stream for \"hello, world\\n\". Local\nrepro (isolated test, full ./zlib/..., full ./...) is clean on\namd64/linux, no GOOS-specific code path found in the flate encoder,\nand master\u0027s recent CI history has no record of this failing. Pushing\nan empty commit to see if it reproduces before treating it as a real\nbug versus a runner flake.\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01RjoGCCuDTpqAdghumgYWCB\n\n* zlib: make ExampleNewWriter version-tolerant of the stdlib flate encoder\n\nExampleNewWriter asserts against Go\u0027s standard library compress/zlib\n(not this package), and Go 1.27 changed the stdlib flate encoder\u0027s\ntiny-input heuristic (now stores 13-byte inputs instead of Huffman-\ncompressing them). Exact encoder output isn\u0027t part of Go\u0027s\ncompatibility promise, only decodability is, so a golden-byte\nassertion can\u0027t hold across a CI matrix spanning multiple Go\nversions. Round-trip through a Reader instead, matching the existing\nstyle of ExampleNewReader in the same file.\n\nVerified go1.26.7 and go1.27.0 both pass this test and the full\n`go test ./...` suite.\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01RjoGCCuDTpqAdghumgYWCB\n\n* zlib: check round-trip errors in ExampleNewWriter\n\nPer CodeRabbit review on #1198: io.Copy and r.Close errors were\nunchecked. This matters here specifically because the zlib adler32\nchecksum is only verified at EOF/Close, so a corrupted round-trip\ncould have io.Copy already write the correct-looking bytes before\nClose surfaces the checksum mismatch -- silently defeating the point\nof round-tripping instead of asserting exact encoder bytes.\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01RjoGCCuDTpqAdghumgYWCB\n\n* go.mod: bump minimum Go version to 1.25\n\nPer CodeRabbit review on #1198: go.mod still declared go 1.24 as the\nminimum, but the CI matrix (this PR) no longer tests 1.24, so a\nregression affecting only 1.24 could merge unverified while the\nmodule claimed 1.24 compatibility. Aligning the declared minimum with\nwhat CI actually covers.\n\nThis is a breaking change for any downstream consumer still building\nwith Go 1.24: since Go 1.21 the go directive is enforced as a hard\ntoolchain minimum. Verified `go build ./...`, `go vet ./...`, and the\nfull `go test ./...` suite still pass under both go1.26.7 and\ngo1.27.0.\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01RjoGCCuDTpqAdghumgYWCB\n\n* apply go fix ./... for go1.25\u0027s sync.WaitGroup.Go\n\nBumping go.mod\u0027s minimum to 1.25 unlocked go fix suggestions gated on\nthe declared language version: the Add(1)/go func(){ defer Done() }()\npattern collapses to wg.Go(func(){ ... }), now that WaitGroup.Go\nexists in the stdlib. build-special\u0027s `go fix -diff ./...` gate\nrequires this be applied, not just available.\n\nMechanical, go fix-generated change (verified go build, go vet, gofmt,\nand the full go test ./... suite all pass after applying).\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01RjoGCCuDTpqAdghumgYWCB\n\n* _generate: bump zstd/huff0 generator go.mod to go 1.25\n\nBoth locally replace the root module (replace github.com/klauspost/compress\n\u003d\u003e ../..), so once the root go.mod declared go 1.25, Go\u0027s module graph\nrequired these to declare go \u003e\u003d 1.25 too (\"go: updates to go.mod needed\").\nFixed via go mod tidy in each. s2/_generate doesn\u0027t depend on the root\nmodule and was unaffected.\n\nVerified go generate -v -x in all three _generate dirs produces\nbyte-identical output to what\u0027s committed (git status clean after\nregenerating) -- this only touches go.mod/toolchain metadata, not\ngenerator behavior.\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01RjoGCCuDTpqAdghumgYWCB\n\n* s2/cmd/_s2sx: bump generator go.mod to go 1.25\n\nSame issue as zstd/huff0 _generate: this module locally replaces the\nroot github.com/klauspost/compress module, so it needed go \u003e\u003d 1.25 to\nmatch the root\u0027s bumped directive. This is what build-special\u0027s\ngoreleaser snapshot step exercises (gen.sh -\u003e go generate . in this\ndir), which is why it wasn\u0027t caught by the earlier _generate fix.\n\nVerified go generate . here is still a no-op on tracked source (only\nproduces build artifacts, not committed).\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\nClaude-Session: https://claude.ai/code/session_01RjoGCCuDTpqAdghumgYWCB\n\n---------\n\nCo-authored-by: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "f5c1529bc7838087176f39eb85e2495f5292606d", "tree": "3e0153626afb189fc93b967bc2e83b2d3c3b428b", "parents": [ "73508b25f222ed50fad2ff7540c059f893781c30" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Aug 31 10:22:01 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Aug 31 10:22:01 2026" }, "message": "zstd/huff0: pin avo peephole folds, regenerate arm64 (#1199)\n\nPulls in honeycombio/avo@0a52965 (branch lizf.arm64-peepholes): the\nprinter now folds \"AND $mask x2 + TST\" (x86\u0027s same-register TESTB/\nTESTW idiom) to a single masked TST, and \"MOVQ/SHRQ+MOVBQZX\" self-\nextends to a single UBFX. Regenerated amd64 output is byte-identical\n(these are arm64-only printer changes); arm64 output shrinks by\nexactly the matched sites: the loop-bottom exhausted-flag check in\nboth huff0 4X loops (3 insns -\u003e 1) and the FSE table-build check, and\nthe \"moB\" byte extract in all four zstd seqdec_* functions (4 insns\n-\u003e 1, dropping a dead MOVD to RSP too).\n\nMotivated by production pprof profiling (Honeycomb\u0027s arm64 Lambda\nworkload, Graviton2). Benchmarking to follow before deciding whether\nthis ships." }, { "commit": "73508b25f222ed50fad2ff7540c059f893781c30", "tree": "02ac5510c867b88cad3c624841f6dd0dba9781e6", "parents": [ "0e43f04adae8400cdaf545242467c9910ce9a513" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Aug 17 12:15:05 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Aug 17 12:15:05 2026" }, "message": "s2: arm64 assembly encoders, lowered from the amd64 avo program (#1187)\n\n* s2, zstd: arm64 asm via avo lowering (NOT SHIPPABLE — see benchmarks)\n\nComplete and correct, retained for reference only. Full s2 suite passes on\nNeoverse N1; zstd encode tests pass. Measured against the pure-Go path arm64\nused previously:\n\n s2 Twain default -11.9% time (+13.6% B/s)\n s2 Twain better -11.4%\n s2 snappy-default -12.6%\n s2 Random 1MB default +16% time\n s2 Random 1MB better +156% time (-61% B/s)\n geomean +8.7% SLOWER\n\n zstd matchLen ~24% slower than generic Go at 1-8KB\n\nIncompressible input emits one large literal run, so that benchmark is\nessentially a memmove: Go\u0027s runtime memmove reaches ~11 GB/s on arm64 while\nthe generator\u0027s inlined 128-byte MOVOU loop manages ~4.6 GB/s. matchLen\nloses for a related reason -- the x86 source keeps lengths in 32-bit\nregisters, costing MOVWU truncations, and Go compiles its generic\nle.Load64/TrailingZeros64 loop more tightly than the lowered x86.\n\nCo-Authored-By: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* s2: regenerate arm64 with improved printer (address caching, GOAMD64 evaluation)\n\nRegenerated output only. Still not shippable per the benchmarks in the\nprevious commit; retained so the branch reflects the current printer.\n\nCo-Authored-By: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* s2: regenerate arm64 against the current lowering\n\nThe previous arm64 generation no longer builds: the lowering has since\nstopped interpreting preprocessor directives (that machinery produced\nabout a third of the miscompiles review found) and has grown a guard\nagainst reading a carry flag across DECQ. Both fire on this generator.\n\nNeither is really an arm64 question. The GOAMD64 blocks -- the v4-to-v3\npromotion and the TZCNT/BSF selection -- are instructions to the amd64\nassembler, so arm64 is emitted by a separate pass that omits them and\nuses TZCNTQ outright, which lowers to RBIT+CLZ. Keeping the amd64 pass\nuntouched is also what keeps its output identical, and it is: byte for\nbyte against master, header and build tag aside.\n\nThe carry guard is pointing at something real. genMemMoveLong\u0027s loop\nbranches JA on the carry that a SUBQ two instructions earlier left,\nbecause x86 DECQ deliberately preserves CF where the arm64 SUBS it\nlowers to does not. The previous generation predates the guard and so\nemitted that branch reading a flag its producer no longer set. arm64\nnow gets the same 32 bytes per iteration with the branch reading a CMPQ\nthat immediately precedes it.\n\nFull s2 test suite passes on Neoverse-N1.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: drop the arm64 matchLen from this branch\n\nIt measured ~24% slower than the generic Go loop on Neoverse-N1 (8KB:\n879ns lowered vs 706ns Go), which is the expected direction: the x86\nsource does its length arithmetic in 32 bits, costing truncations once\nlowered, and Go\u0027s le.Load64-xor-TrailingZeros64 loop already compiles to\ntighter arm64 than the translation of the x86 does.\n\nIt rode along on the branch this one descends from. It does not belong\nin a change about the s2 encoders.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\n\n* s2: drop the GOAMD64 guard around TZCNT, per review\n\nTZCNT\u0027s encoding is BSF\u0027s with an F3 prefix, and processors without the\ninstruction ignore the prefix and execute BSF. One encoding is therefore\ncorrect everywhere, and the assembler-level choice this made was never\nneeded.\n\nThe two differ only on a zero input, which cannot reach here -- every\ncaller has established the value is nonzero -- and in flags, which is\nunobservable: all seventy call sites are followed immediately by SARQ,\nwhich overwrites the flags without reading them. Verified by scanning\nthe generated output: seventy sites, seventy SARQ successors, no\nconditional jump within two instructions of any of them.\n\nThe amd64 output changes by exactly the seventy BSFQ lines and their\nsurrounding directives; the instruction stream is otherwise identical\nmodulo avo\u0027s operand alignment. arm64 output is unchanged -- it already\nemitted TZCNTQ unconditionally, which is what prompted the question.\n\nThis removes the per-site GOAMD64 directives, leaving only the v4-to-v3\npromotion block at the top of the file, so the arm64 pass still has to\nbe separate.\n\nSuggested-by: Klaus Post \u003cklauspost@gmail.com\u003e\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "0e43f04adae8400cdaf545242467c9910ce9a513", "tree": "8207863ce42e76cf51247fe8ebd2825aa50fe786", "parents": [ "4005c155a2670b665ddc52205b6e9dacb43cf651" ], "author": { "name": "Alex Sharov", "email": "AskAlexSharov@gmail.com", "time": "Mon Aug 17 09:53:09 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Aug 17 09:53:09 2026" }, "message": "xxhash: bound input of non-preemptive assembly (#1196)\n\n* xxhash: bound the work Digest.Write does in one assembly call\n\nA GC that overlaps a large Digest.Write waits out the whole hash, stopping\nevery goroutine in the process for that window. zstd hits this on every\nframe: framedec hands the entire decoded frame to Digest.Write in one call.\n\nwriteBlocks was NOSPLIT|NOFRAME and Write passed it the whole input, so the\nhash was a single assembly call. The runtime cannot preempt assembly, and\nNOSPLIT removes the stack-growth check that is the only cooperative\npreemption point left.\n\nWrite now feeds writeBlocks 64 KiB at a time, and writeBlocks carries a\nstack frame so each chunk call has that check. Both are needed: measured\nalone, neither moves the stall at all.\n\nSum64 is unchanged. It is also one unbounded call, but no caller here hands\nit more than a block, and wrapping it in Go cost 45% on 4 B inputs.\n\n* xxhash: wait for the hash to start in BenchmarkDigestSTW\n\nThe hash goroutine was not synchronized before runtime.GC(), so a collection\ncould complete before hashing had started and that iteration measured nothing.\nIt overlapped every time in practice only because the hash is far longer than\na collection, which is luck rather than design. The goroutine now signals\nbefore it hashes and the timing waits for that signal.\n\nNumbers are unchanged: DigestSTW/128MB 6429.6us -\u003e 152.4us, DigestSTW/10MB\n533.7us -\u003e 159.6us, and the goloop controls stay flat.\n\n* Update zstd/internal/xxhash/xxhash.go\n\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e\n\n* xxhash: verify the collection overlapped the hash, don\u0027t assume it\n\nA signal sent just before work() begins still leaves a window: the goroutine\ncan be descheduled between the signal and the first Write, and the collection\nthen completes without overlapping anything. Rather than widen the handshake,\nrecord when the work actually began and count the iteration only if that was\nbefore the collection finished.\n\nThis is checkable rather than argued, and it works on both builds. Collecting\nuntil done instead, as a max, would also close the window but reports a tail\nstatistic: measured that way the same benchmark ran at +-15% to +-24% against\n+-1% for the mean.\n\nmissed-overlap is reported when any iteration is discarded, so a run that\nmeasured nothing cannot look clean. Across 6 runs of each build at\n-benchtime\u003d5s, nothing was discarded, and the numbers are unchanged:\n\n DigestSTW/128MB 6449.8us -\u003e 137.4us -97.87%\n DigestSTW/128MB-goloop 217.9us -\u003e 250.0us ~\n DigestSTW/10MB 585.1us -\u003e 166.7us -71.52%\n DigestSTW/10MB-goloop 213.6us -\u003e 212.3us ~\n\n* xxhash: keep the benchmark sizes straddling maxAsmSize\n\nmaxAsmSize is now 128 KiB, so the 128KB case (128e3 \u003d 128000 bytes) sits below\nit and reaches writeBlocks in one call like every smaller size. Adds 256KB so\none size still crosses the bound, and corrects the two comments that said 128KB\nwas over it.\n\n---------\n\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "4005c155a2670b665ddc52205b6e9dacb43cf651", "tree": "7b878b890296c754cc79c66ebf75653754c9af93", "parents": [ "dd6de45e8beb8833684fbafb4ecd06b0062dafa6" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Aug 17 07:46:46 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Aug 17 07:46:46 2026" }, "message": "s2: clamp the Go EncodeBetter skip to match the assembly (#1190)\n\n* s2: clamp the Go EncodeBetter skip to match the assembly\n\nencodeBetterBlockAsm has clamped the adaptive scan skip at 100 since it\nwas written (\"Blocks can be long, limit max skipping\"), and the snappy\nGo encoder in this same file already does the same. encodeBlockBetterGo\ndid not, so after a long stretch with no matches its skip grew\ngeometrically until it strode past the start of the next compressible\nregion and lost the matches there.\n\nMeasured on 4MB of alternating noise and text, Go before -\u003e after:\n\n noise runs compressed size delta\n 16KB 2527580 -\u003e 2521369 -0.25%\n 64KB 2520571 -\u003e 2516749 -0.15%\n 256KB 2518285 -\u003e 2515781 -0.10%\n\nUniformly compressible input is byte-for-byte unchanged, since the skip\ndoes not grow while matches keep arriving. The cost is on incompressible\ninput, where Go now walks the positions the assembly already walked:\nencodeBlockBetterGo on 1MB of random data goes from ~9us to ~39us. That\nis the trade the assembly has always made, and this brings Go into line\nwith it rather than the other way round.\n\nThe Go and assembly outputs are now byte-identical on the \u003e64KB path,\nwhich the new test asserts directly rather than bounding a ratio gap.\n\nNote this leaves blocks of 16383..64KB diverging, in the other\ndirection: the assembly dispatches encodeBetterBlockAsm4MB there, which\nclamps, while Go dispatches encodeBlockBetterGo64K, which does not and\nalso uses skipLog 6 rather than 7. On uniform inputs the two still agree\nbyte for byte; they part company only on mixed content with long noise\nruns inside a small block. Left alone here since it is a different\nconfiguration rather than a missing clamp.\n\nFixes #1186\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\n\n* s2: hoist s out of the skip clamp, per review\n\nKlaus: comparing against the constant rather than against s+maxSkip\nmakes it a constant compare and saves an addition, and may be faster on\nsome platforms.\n\nEquivalent for every value reachable here -- adding s is monotonic, so\nit commutes with min -- and marginally safer besides: the old shape adds\ns to both operands before comparing, which the hoisted form does not, so\nthe comparison cannot overflow a 32-bit int. Not reachable at these\nblock sizes either way.\n\nGo and assembly output remain byte-identical on the \u003e64KB path.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "dd6de45e8beb8833684fbafb4ecd06b0062dafa6", "tree": "051236cf855731b51f27f93450979a020669d97c", "parents": [ "07f8bcdd527893e14cf6cab798217c86a2cf5b66" ], "author": { "name": "Vedant Madane", "email": "6527493+VedantMadane@users.noreply.github.com", "time": "Tue Aug 11 11:04:52 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Aug 11 11:04:52 2026" }, "message": "gzhttp: limit jitter hash to jitterBuffer bytes (#1192)\n\n* gzhttp: limit jitter hash to jitterBuffer bytes\n\nWhen RandomJitter is configured with a buffer smaller than the internal\nwrite buffer (wantBuf \u003d max(minSize, 512)), startCompression hashed the\nentire w.buf for the padding RNG seed. That made the seed depend on how\nthe handler chunked Write() calls, defeating the purpose of jitterBuffer.\n\nClamp the hashed prefix to jitterBuffer for both CRC32 and SHA-256 paths,\nand use the same limit when topping up from remain. Adds a regression\ntest for write-size independence (CRC and paranoid modes).\n\nFixes #1191\n\n* fix: apply go fix range-over-int in jitter test" }, { "commit": "07f8bcdd527893e14cf6cab798217c86a2cf5b66", "tree": "3b7c3c71e26e115252dd795231a22a2f3da01963", "parents": [ "ff8c845b10f76ba9d2fe113e7a4960c772f18b32" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Fri Aug 07 13:45:03 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Aug 07 13:45:03 2026" }, "message": "zstd, huff0: improved arm64 lowering, and narrow two table loads to MOVWQZX (#1175)\n\n* zstd, huff0: regenerate arm64 assembly with the improved avo lowering\n\nNo source or algorithm change: the same generators, re-run against a\nlowering that emits better arm64. Two improvements account for it.\n\n128-bit moves now use FMOVQ, which takes a base+displacement operand\ndirectly, where the previous VLD1/VST1 needed the address materialized into\na scratch register first. And scalar accesses fold base+index into the\ninstruction when arm64 permits it (the index shift must equal log2 of the\naccess width, and no displacement may be present), with a cache so a run of\naccesses sharing a base and index computes the address once. x86 folds\nbase+index+displacement into a single instruction and the generators lean on\nthat heavily, so this recovers much of what the lowering used to pay per\naccess.\n\nseqdec_arm64.s and decompress_arm64.s together shed about 90 lines. The\namd64 assembly is byte-identical.\n\nMeasured on Neoverse N1 (Ampere, go1.26, -count 6, every result p\u003c0.03):\n\n Decoder_DecodeAll geomean -1.53%\n html.zst -2.29%\n plrabn12.txt.zst -2.19%\n lcet10.txt.zst -2.11%\n kppkn.gtb.zst -2.08%\n alice29.txt.zst -2.02%\n huff0 Decompress geomean -0.81%\n Decompress4X/twain -2.81%\n\nInputs that are not sequence-bound (fireworks.jpeg, html_x_4) are unchanged,\nas expected.\n\nCo-Authored-By: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* zstd, huff0: bump the pinned avo to pick up thirteen lowering fixes\n\nGenerated output is unchanged -- regenerating against this pin reproduces\nevery .s file byte for byte, which also confirms none of the fixed bugs was\nreachable from these generators. The pin still matters: it is what the next\nregeneration uses.\n\nThree review passes over the lowering found thirteen ways it could emit\nplausible-looking but wrong arm64. About half are wrong values: byte extends\nreading the wrong field from a high-byte source, a 32-bit conditional move\nlosing x86\u0027s unconditional zero-extension, a negative 32-bit immediate\nsign-extending across all 64 bits, BEXTR clobbering its own staged control\nfield with a memory source, and compare/test/multiply loading eight bytes for\na four-byte memory operand, which can fault at a page boundary.\n\nThe rest are cases where the two architectures disagree about what a flag\nmeans, and the lowering now refuses to translate rather than emit something\nplausible: carry conditions after additions and after logical ops (both\ninvert), carry read across INC/DEC (which preserve it on x86 but not once\nlowered), ADC after anything but a compare or subtract, a signed sub-word\ncompare whose consumer sits past a flag-transparent instruction, and the\nXOR-self zeroing shortcut, which dropped the zero flag x86 sets there.\n\nThe last round also resolves the GOAMD64 conditionals before the flag\nanalyses run, so those see exactly the instructions that will be emitted --\notherwise a producer inside a dead arm could absorb a mark a live one needed.\n\nCo-Authored-By: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* zstd, huff0: narrow two table loads to MOVWQZX, and bump the pinned avo\n\nThe avo bump carries seven lowering fixes from a fourth review pass (see\nthe fork\u0027s log). One of them matters here: a 16-bit load into a register\nlowered to a zero-extending arm64 load, where x86 preserves the upper 48\nbits. Nothing in this repository was miscompiled by that -- at all forty\naffected sites the destination\u0027s upper bits are dead, so zero-extending\nwas harmless as applied -- but the printer cannot know that, and an avo\nprogram that did depend on those bits would have gotten wrong code.\n\nMaking the lowering exact costs an instruction on every 16-bit load, and\nthat instruction reads the destination it writes, which turns an\nindependent load in huff0\u0027s innermost decode loop into a loop-carried\ndependency: measured +11.74% geomean on Decompress4X on Neoverse N1, up\nto +19% (p\u003d0.000, n\u003d12). zstd\u0027s DecodeAll paid a smaller +0.73%.\n\nSo this asks for what the code actually wants. Every one of these sites\nloads a table entry and then reads only its low bits, so none of them\nneeds the surrounding register preserved; MOVWQZX says that, and lowers\nback to a single folded load. This is belt and suspenders -- there was no\nbug here to fix -- it just keeps the general safety fix from costing\nanything where preservation was never wanted.\n\nIt is also simply the better x86 encoding, which is worth having on its\nown: MOVZX avoids the 66h operand-size prefix and the partial-register\nmerge dependency, the same hazard that cost 11.74% on arm64. On amd64\nhuff0 Decompress4X that is -7.59% geomean, up to -14.10% (p\u003d0.000, n\u003d12,\nRyzen 5 5600X).\n\nNet effect on generated code: huff0/decompress_arm64.s and\nzstd/seqdec_arm64.s are byte-identical to before, zstd/fse_decoder_arm64.s\nloses two instructions to folded addressing, and the amd64 files pick up\nthe faster encoding. The two changes land together because the pin bump\nalone would regress arm64 until the generators were narrowed.\n\n* zstd, huff0: bump the pinned avo for the round-five lowering fixes\n\nGenerated assembly is unchanged, byte for byte. None of this round\u0027s\nbugs was reachable from these generators: they emit only GOAMD64\ndirectives, which the lowering resolves and removes, so the state that\nwas leaking across foreign preprocessor arms had nothing to leak past.\nThe pin moves anyway so the committed assembly corresponds to a printer\nwe would be willing to point at other code.\n\n* zstd, huff0: bump the pinned avo for the round-six lowering fixes\n\nGenerated assembly is unchanged, byte for byte. The 64-bit immediate\nsign-extension bug this picks up needs a Q-width immediate with bit 31\nset, which neither of these generators emits -- s2\u0027s does, which is why\nit was worth finding.\n\n* zstd, huff0: bump the pinned avo for the round-seven lowering fixes\n\nGenerated assembly is unchanged, byte for byte. The frame-offset fix this\nround originally carried landed separately as #1176 and is already in the\nbase; regenerating against this pin reproduces it exactly, which is the\ncheck that the two arrived at the same answer rather than merely at\ncompatible ones.\n\nRound seven\u0027s other fixes are not reachable from these generators: the\nthree-operand SHL/SHR forms are rejected (they are SHLD/SHRD, a different\ninstruction whose destination is the third operand); preprocessor\ndirectives are normalized so \"# else\" is recognized, since Go\u0027s assembler\ntokenizes the \u0027#\u0027 separately and a missed #else drops BOTH arms; and the\nmixed-comment check moved into the preprocessor evaluator, deleting a\nsecond stateless opinion about directive ownership that had already\ndisagreed with the authoritative one.\n\n* zstd, huff0: bump the pinned avo for the round-eight lowering changes\n\nThe address cache is gone from the lowering. It let a second access to\nthe same base and index reuse the register the first one computed, and\nit had to be invalidated at every boundary the printer does not\notherwise model -- two of the silent miscompiles found in review were\nmissing invalidations rather than missing lowerings.\n\nMeasured here, it was worth two instructions: seqdec_arm64.s and\nhuff0/decompress_arm64.s are byte-identical without it, and\nfse_decoder_arm64.s gains two ADDs in buildDtable, which runs once per\nblock. Most accesses fold base+index into the instruction and never\nmaterialize an address at all, so there was rarely anything to cache.\n\nRound eight\u0027s review found no new lowering bugs, so the rest is\nhousekeeping: dead code removed, a confusing panic given a real message.\n\n* zstd, huff0: bump the pinned avo for the round-nine lowering fixes\n\nGenerated assembly is unchanged, byte for byte. None of round nine\u0027s\nbugs is reachable from these generators: the CMN boundary case needs a\ncompare against INT32_MIN written as a signed immediate, and the two\ndirective-coupling holes need directives, which neither generator emits.\n\n* zstd, huff0: bump the pinned avo for the round-ten and -eleven changes\n\nGenerated assembly is unchanged, byte for byte, including across the\nremoval of the lowering\u0027s preprocessor-directive support -- neither of\nthese generators emits a directive, which is most of why that support\nwas removed.\n\nAlso picks up shift counts at or above the operand width being masked\nthe way x86 masks them, rather than passing through to an arm64\nimmediate form that rejects them, and 64-bit immediates written unsigned\nwith the top bit set being read as the bit pattern amd64 wraps them to\nrather than refused.\n\n* zstd, huff0: bump the pinned avo for the round-eleven guard fixes\n\nGenerated assembly unchanged, byte for byte. Both fixes are to the guard\nthat refuses preprocessor directives, which neither of these generators\nemits.\n\n* zstd, huff0: bump the pinned avo for the round-twelve guard fixes\n\nThe only change to generated assembly is one build-tag line per file:\nthe original constraint is now bracketed before the arm64 term is\nANDed onto it. That is a no-op for these three -- every constraint here\nis a pure AND chain -- but without it a constraint containing || would\nhave parsed as \"(arm64 \u0026\u0026 ...) || ...\" and pulled the arm64 assembly\ninto non-arm64 builds.\n\nInstructions are byte-identical. Verified building and testing both\narchitectures.\n\n* zstd, huff0: bump the pinned avo for the label-name guard\n\nGenerated assembly unchanged, byte for byte.\n\n* zstd, huff0: bump the pinned avo for the round-thirteen hardening\n\nGenerated assembly unchanged, byte for byte. Round thirteen found no\nmiscompiles; the changes make flag transparency, instruction suffixes,\nbranch targets and the one-line output invariant into properties this\nprinter checks itself rather than ones that held because a separate tool\nhappened to reject the alternative.\n\n* zstd, huff0: bump the pinned avo for the round-fourteen enforcement\n\nGenerated assembly unchanged, byte for byte. The lowering now checks both\ndirections of the flag contract at generation time: an instruction claimed\nflag-neutral must emit no flag-writers, and one marked as a producer must\nemit exactly one.\n\n---------\n\nCo-authored-by: Claude Opus 5 (1M context) \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "ff8c845b10f76ba9d2fe113e7a4960c772f18b32", "tree": "0689332c50d8e80fa15fe4294a38e34bddf9e352", "parents": [ "c3b3439a48196b5082c63252bfb8633d0a2faad4" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Fri Aug 07 13:44:00 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Aug 07 13:44:00 2026" }, "message": "Add lzw reader/writer (#1189)\n\n* Add lzw reader/writer\n\nSee #1188\n\nForked from the Go standard library. Includes https://pkg.go.dev/golang.org/x/image/tiff/lzw as opt-in.\n\n# lzw\n\nThis is a drop-in replacement for the standard library [`compress/lzw`](https://pkg.go.dev/compress/lzw) package,\nwith a **decompressor that is typically 3-4x faster** and a **compressor that is 1.4-2.7x faster**.\n\nBoth are exactly compatible: decoding produces byte for byte identical output and reports the same errors for\ncorrupt or truncated input, and encoding produces byte for byte identical compressed streams, so the\ncompression ratio is unchanged.\n\nOne difference to be aware of: `Read` writes codes out eight bytes at a time, so it may modify up to seven\nbytes of the buffer you pass it beyond the count it returns. It never writes beyond the length of that\nbuffer, so `Read(buf[:k])` will not disturb `buf[k:]`.\n\n* [Godoc documentation](https://pkg.go.dev/github.com/klauspost/compress/lzw)\n\n# usage\n\nReplace imports of `compress/lzw` with `github.com/klauspost/compress/lzw`. Nothing else has to change:\nthe package has the same API, including `Order`, `LSB`, `MSB`, `NewReader`, `NewWriter` and the `Reset`\nmethods.\n\nDecompressing:\n\n```Go\nr :\u003d lzw.NewReader(src, lzw.LSB, 8)\ndefer r.Close()\n\nif _, err :\u003d io.Copy(dst, r); err !\u003d nil {\n return err\n}\n```\n\nCompressing:\n\n```Go\nw :\u003d lzw.NewWriter(dst, lzw.LSB, 8)\nif _, err :\u003d w.Write(data); err !\u003d nil {\n return err\n}\n// Close flushes the final codes. It does not close dst.\nreturn w.Close()\n```\n\nUse `LSB` with a literal width of 8 for GIF, and `MSB` for PDF. The literal width must be the same as the\none used when compressing.\n\n# performance\n\nSingle core, AMD Ryzen 9 9950X. Decompression is measured decoding to a 64KB buffer.\n\nDecompression:\n\n| Input | `compress/lzw` | this package | speedup |\n|------------------------------|---------------:|-------------:|--------:|\n| `Mark.Twain-Tom.Sawyer.txt` | 213 MB/s | 821 MB/s | 3.86x |\n| `html.txt` | 240 MB/s | 818 MB/s | 3.41x |\n| `e.txt` (digits) | 307 MB/s | 881 MB/s | 2.87x |\n| `pngdata.bin` (binary) | 558 MB/s | 1842 MB/s | 3.30x |\n| Tom Sawyer, `MSB` order | 214 MB/s | 844 MB/s | 3.95x |\n| Tom Sawyer, litWidth 5 | 219 MB/s | 875 MB/s | 3.99x |\n| 1KB streams | 332 MB/s | 559 MB/s | 1.69x |\n| `sharnd.out` (random data) | 238 MB/s | 329 MB/s | 1.38x |\n\nCompression:\n\n| Input | `compress/lzw` | this package | speedup |\n|------------------------------|---------------:|-------------:|--------:|\n| `e.txt` (digits) | 130 MB/s | 356 MB/s | 2.73x |\n| 1KB streams | 248 MB/s | 478 MB/s | 1.92x |\n| Tom Sawyer, litWidth 5 | 108 MB/s | 194 MB/s | 1.80x |\n| `html.txt` | 143 MB/s | 230 MB/s | 1.60x |\n| `sharnd.out` (random data) | 160 MB/s | 255 MB/s | 1.60x |\n| `Mark.Twain-Tom.Sawyer.txt` | 132 MB/s | 194 MB/s | 1.48x |\n| Tom Sawyer, `MSB` order | 136 MB/s | 191 MB/s | 1.41x |\n| `pngdata.bin` (binary) | 397 MB/s | 431 MB/s | 1.09x |\n\nDecoding gains most on the compressible data LZW is normally used for; incompressible input, which LZW\nexpands rather than compresses, gains the least since almost every code is a single literal byte. Encoding\ngains most on input drawn from few distinct byte values, such as digits or a small colour palette, and least\non input made of long repeats, where it emits few codes to begin with.\n\nNeither the reader nor the writer allocates while encoding or decoding, and reusing either with `Reset` does\nnot allocate at all — except that a source which is not an `io.ByteReader` is wrapped in a `bufio.Reader` on\neach `Reset`, as it is in the standard library.\n\nRun `go test -bench\u003d\u0027Decode|Encode\u0027 -benchtime\u003d500ms -count\u003d10 .` in this directory to compare on your own\nmachine; `BenchmarkDecodeStd` and `BenchmarkEncodeStd` measure `compress/lzw` in the same process for a fair\ncomparison.\n\n# getting the best speed\n\nNone of these are required for correct use, but each affects how fast the package runs.\n\n**Decoding: read into a buffer of at least 8KB.** Given that much room the reader expands codes straight into\nyour buffer; with less it has to decode into an internal buffer and copy from it. `io.Copy` passes 32KB and so\nneeds no help, but note that a `bufio.Reader` wrapped *around* the reader defaults to 4KB.\n\n**Decoding: prefer a source that can be read a machine word at a time.** The reader is careful never to\nconsume more of the source than the code stream needs, which is what makes it usable for embedded streams such\nas the image data in a GIF. How it achieves that depends on the type passed to `NewReader`:\n\n| Source | How it is read |\n|----------------------------------------------------------------------------------------------------------|--------------------------------------|\n| any `io.Reader` that is not an `io.ByteReader`, e.g. `*os.File` or a network connection | wrapped in a `bufio.Reader`, fast |\n| `*bufio.Reader`, or anything with `Peek`/`Discard`/`Buffered` | peeked, then discarded, fast |\n| `*bytes.Reader`, `*strings.Reader`, or any `io.ByteReader` that is also an `io.ReaderAt` and `io.Seeker` | read at an offset, then seeked, fast |\n| anything else implementing `io.ByteReader` | one byte at a time, slower |\n\nThe last row is the only slow case. It covers types such as `*bytes.Buffer`, and the block readers that\ncontainer formats often use. Wrapping such a source in a `bufio.Reader` is only safe if reading ahead of the\ncompressed stream is acceptable, since a `bufio.Reader` will consume whatever follows it.\n\n**Either direction: reuse readers and writers with `Reset`.** Both hold their dictionary inline — a `Reader` is\naround 56KB and a `Writer` around 68KB — so reusing one across streams, as when handling the frames of an\nanimated GIF, avoids repeated allocation. `Reset` on a `Reader` does not have to clear its tables, so it is\nvery cheap; on a `Writer` it clears the hash table, which is still far cheaper than allocating.\n\n```Go\nr :\u003d lzw.NewReader(frames[0], lzw.LSB, 8).(*lzw.Reader)\ndefer r.Close()\n\nfor _, frame :\u003d range frames {\n r.Reset(frame, lzw.LSB, 8)\n // ... read from r ...\n}\n```\n\n## TIFF and PDF: the Aldus variant\n\nTIFF uses a variant of LZW that increases the code width one code early — the \"off by one\" that Aldus\nimplemented and that libtiff keeps for compatibility. PDF\u0027s `LZWDecode` filter specifies the same variant by\ndefault, through its `/EarlyChange` value of `1`. Decoding either needs one extra call:\n\n```Go\nr :\u003d lzw.NewReader(src, lzw.MSB, 8).(*lzw.Reader)\nr.SetAldusCompatible(true)\ndefer r.Close()\n```\n\nSet it before the first `Read`. It is configuration rather than stream state, so it survives `Reset` and a\npooled reader keeps it. There is no cost to leaving it off: the default path is unaffected.\n\nOnly the reader supports this so far — `Writer` always produces standard streams, which is what GIF wants and\nwhat PDF accepts with `/EarlyChange 0`.\n\n# testing\n\nBoth halves are tested by comparison with `compress/lzw` over every test input, both bit orders and all\nliteral widths.\n\nDecoding must agree on the decoded bytes, on the error returned and on how many bytes of the source were\nconsumed, for each kind of source and a range of read sizes. Encoding must produce identical bytes, whatever\nsizes the input is written in.\n\n`FuzzReader` fuzzes the decoder comparison, including for corrupt and truncated input, `FuzzWriter` the\nencoder comparison, and `FuzzRoundtrip` checks that what the writer produces decodes back to the input.\n\n* Go fix.\n\n* Address feedback\n\n* Cleanup and remove local vars. Was slower anyway." }, { "commit": "c3b3439a48196b5082c63252bfb8633d0a2faad4", "tree": "776d3ec747236df908af7f1fefa658097a13b439", "parents": [ "9874bc9073f350ce462becb84f9a23c3e828d03f" ], "author": { "name": "Sueun Cho", "email": "sueun.dev@gmail.com", "time": "Wed Aug 05 15:56:51 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Aug 05 15:56:51 2026" }, "message": "zstd: don\u0027t clear the registered dictionary when decoding past the window (#1177)\n\n* zstd: don\u0027t clear the registered dictionary when decoding past the window\n\nexecuteSequences set hist.dict.content \u003d nil when the frame history grew\npast the window. hist.dict is the same *dict registered via\nWithDecoderDicts/WithDecoderDictRaw, so this blanked the dictionary for\nevery later decode on the decoder: after one streaming decode of a\ndictionary frame larger than the window, subsequent decodes of a valid\ndictionary frame failed with \u0027match offset (...) bigger than current\nhistory\u0027. The write was dead for the current decode (the streaming path\nreads its own hist.decoders.dict copy) and only ever corrupted reuse.\n\nDrop the mutation and keep the window trim; the shared dictionary is now\nleft intact, matching the sync decode path.\n\nAdd TestDecoderDictReuseAfterLargeStream (fails before, passes after).\n\n* go fix\n\n* Move test into existing file and simplify the test somewhat. Verified that it still reproduces.\n\n---------\n\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "9874bc9073f350ce462becb84f9a23c3e828d03f", "tree": "1c8803d7fec79a47e702a4f80d53c6e8cedc26a0", "parents": [ "71bb6fd9ddbfbb2ca6612542a916c766a866bfb3" ], "author": { "name": "Rayan Salhab", "email": "r.salhab@aiyexpertsolutions.com", "time": "Wed Aug 05 14:34:02 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Aug 05 14:34:02 2026" }, "message": "fix(zstd): handle zero-literal BuildDict corpus (#1178)\n\nCo-authored-by: cyphercodes \u003ccyphercodes@users.noreply.github.com\u003e" }, { "commit": "71bb6fd9ddbfbb2ca6612542a916c766a866bfb3", "tree": "b12ed55ce89fe90924a78652ebd97b7192d87c6a", "parents": [ "3d4dacbaa9faca75caacc35b6d75731a81a92c6b" ], "author": { "name": "zanarelli", "email": "zanarelli.dev@gmail.com", "time": "Wed Aug 05 08:22:54 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Aug 05 08:22:54 2026" }, "message": "zstd: keep BuildDict recent-offsets positive and loadable (#1184)\n\n* zstd: keep BuildDict recent-offsets positive and loadable\n\nHomogeneous corpora with unset Offsets were ranking repeat-code hits as\noffset 0 and writing that into the dictionary, so WithEncoderDict rejected\nthe result with \"invalid offset in dictionary\". Skip zero repeat targets,\npad missing slots with defaults, and validate via loadDict before return.\n\nFixes #1179\n\nSigned-off-by: zanarelli \u003czanarelli.dev@gmail.com\u003e\n\n* zstd: apply go fix to dict_test.go\n\nModernizes the TestBuildDictHomogeneousCorpusValidOffsets sample\ngenerator to strings.Builder and a range-over-int loop, as flagged by\ngo fix and requested in review.\n\nSigned-off-by: zanarelli \u003czanarelli.dev@gmail.com\u003e\n\n---------\n\nSigned-off-by: zanarelli \u003czanarelli.dev@gmail.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "3d4dacbaa9faca75caacc35b6d75731a81a92c6b", "tree": "95f0d416fa14e4836502e7a47a836785d2ef5007", "parents": [ "3ceaa81409aabe39c71821b936155b33471c78d8" ], "author": { "name": "zanarelli", "email": "zanarelli.dev@gmail.com", "time": "Wed Aug 05 07:47:09 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Aug 05 07:47:09 2026" }, "message": "zstd: avoid racing MaxDecodedSize write on shared dict litEnc (#1182)\n\n* zstd: avoid racing MaxDecodedSize write on shared dict litEnc\n\nTreeless literal decode wrote MaxDecodedSize on hist.huffTree, which\naliases dict.litEnc for trained dictionaries. Concurrent DecodeAll then\nraced. Decoder.Decompress* already sizes via cap(dst), so drop the write.\n\nFixes #1180\n\nSigned-off-by: zanarelli \u003czanarelli.dev@gmail.com\u003e\n\n* zstd: apply go fix to dict_test.go\n\nModernizes the alphabet-building loop in\nTestSharedDecoderTrainedDictRace to a range-over-int loop, as flagged\nby go fix and required by the build-special CI check.\n\nSigned-off-by: zanarelli \u003czanarelli.dev@gmail.com\u003e\n\n---------\n\nSigned-off-by: zanarelli \u003czanarelli.dev@gmail.com\u003e" }, { "commit": "3ceaa81409aabe39c71821b936155b33471c78d8", "tree": "d4a0d52c36282d611b323267268c26b3bf374f46", "parents": [ "72cb4d3e8e743bea5d1ba40896ad55214a1844e4" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Tue Aug 04 15:11:45 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Aug 04 15:11:45 2026" }, "message": "build(deps): bump the github-actions group with 5 updates (#1185)\n\nBumps the github-actions group with 5 updates:\n\n| Package | From | To |\n| --- | --- | --- |\n| [actions/checkout](https://github.com/actions/checkout) | `7.0.0` | `7.0.1` |\n| [github/codeql-action/init](https://github.com/github/codeql-action) | `4.36.2` | `4.37.3` |\n| [github/codeql-action/autobuild](https://github.com/github/codeql-action) | `4.36.2` | `4.37.3` |\n| [github/codeql-action/analyze](https://github.com/github/codeql-action) | `4.36.2` | `4.37.3` |\n| [actions/setup-go](https://github.com/actions/setup-go) | `6.5.0` | `7.0.0` |\n\n\nUpdates `actions/checkout` from 7.0.0 to 7.0.1\n- [Release notes](https://github.com/actions/checkout/releases)\n- [Commits](https://github.com/actions/checkout/compare/v7...v7.0.1)\n\nUpdates `github/codeql-action/init` from 4.36.2 to 4.37.3\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/8aad20d150bbac5944a9f9d289da16a4b0d87c1e...e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81)\n\nUpdates `github/codeql-action/autobuild` from 4.36.2 to 4.37.3\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/8aad20d150bbac5944a9f9d289da16a4b0d87c1e...e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81)\n\nUpdates `github/codeql-action/analyze` from 4.36.2 to 4.37.3\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/8aad20d150bbac5944a9f9d289da16a4b0d87c1e...e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81)\n\nUpdates `actions/setup-go` from 6.5.0 to 7.0.0\n- [Release notes](https://github.com/actions/setup-go/releases)\n- [Commits](https://github.com/actions/setup-go/compare/v6.5.0...v7)\n\n---\nupdated-dependencies:\n- dependency-name: actions/checkout\n dependency-version: 7.0.1\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n- dependency-name: github/codeql-action/init\n dependency-version: 4.37.3\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: github-actions\n- dependency-name: github/codeql-action/autobuild\n dependency-version: 4.37.3\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: github-actions\n- dependency-name: github/codeql-action/analyze\n dependency-version: 4.37.3\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: github-actions\n- dependency-name: actions/setup-go\n dependency-version: 7.0.0\n dependency-type: direct:production\n update-type: version-update:semver-major\n dependency-group: github-actions\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "72cb4d3e8e743bea5d1ba40896ad55214a1844e4", "tree": "6e53fd7c3872969e7d7abbe6277768e7b06a8296", "parents": [ "69c9db420ae55bfcdfbef564805e2646206545f7" ], "author": { "name": "zanarelli", "email": "zanarelli.dev@gmail.com", "time": "Tue Aug 04 09:17:56 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Aug 04 09:17:56 2026" }, "message": "chore: add OpenSSF Scorecard GitHub Action (#1183)\n\n* chore: add OpenSSF Scorecard GitHub Action\n\nPublish weekly Scorecard results to the Security panel so posture\nregressions are visible without a manual scan.\n\nFixes #831\n\nSigned-off-by: zanarelli \u003czanarelli.dev@gmail.com\u003e\n\n* chore: use action version tags instead of SHA pins in scorecard workflow\n\nPer review feedback, follow the action version tags directly rather than\ncopying release.yml\u0027s SHA-pinning style.\n\n---------\n\nSigned-off-by: zanarelli \u003czanarelli.dev@gmail.com\u003e" }, { "commit": "69c9db420ae55bfcdfbef564805e2646206545f7", "tree": "a3518bf1813b69a042d9da7575fec6708d68d931", "parents": [ "117430d3b0e3c39c14d32fe7c90652149a78e609" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Thu Jul 30 08:37:10 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Jul 30 08:37:10 2026" }, "message": "zstd: fix arm64 asm locals overwriting the saved link register (#1176)\n\nThe generated arm64 decoder assembly places every function local 8 bytes\ntoo low, so local 0 lands on the saved link register.\n\nx86 and arm64 put the return address in different places. On x86, CALL\npushes it before the callee runs, so it sits above the frame and local 0\nis at SP+0. On arm64, BL leaves it in the link register and the callee\nspills it to the BOTTOM of its own frame, so 0(RSP) is the saved LR and\nlocals start at 8(RSP). The avo arm64 lowering renamed the pseudo stack\npointer to RSP and passed the displacement through unchanged, inheriting\nx86\u0027s layout.\n\nVerified by disassembling probe functions at two frame sizes: for both\nTEXT $8 and TEXT $64 the assembler places pseudo-SP local 0 at 8(RSP),\nwhile the prologue\u0027s MOVD.W R30, -N(RSP) puts the saved LR at 0(RSP).\nThe 8-byte offset does not depend on frame size.\n\nAffected: sequenceDecs_decode_arm64, sequenceDecs_decode_56_arm64,\nsequenceDecs_decodeSync_arm64 and sequenceDecs_decodeSync_safe_arm64,\nall present since v1.19.0.\n\nScope: this does NOT corrupt decoded data and cannot produce wrong\noutput. Every local shifts by the same 8 bytes, so loads and stores\nagree and the arithmetic is self-consistent; that is why the test suite\nand fuzzing never caught it. These are leaf functions, so the link\nregister is still live in R30 and RET works. What breaks is the unwind\npath only: GC stack scanning, panics and profiling read the return\naddress from 0(RSP) and find decoder state there.\n\nThe fix shifts pseudo-SP displacements by 8 in the lowering, so the\nregenerated assembly is a mechanical +8 on every local access and\nnothing else. zstd tests pass on arm64 (Neoverse N1)." }, { "commit": "117430d3b0e3c39c14d32fe7c90652149a78e609", "tree": "b065ebc027fa3772fc22d7ed746fefa7fa22a57b", "parents": [ "c73af0c12cc767386af8388f30d5aa7428e6dfc8" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Tue Jul 21 10:01:23 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Jul 21 10:01:23 2026" }, "message": "zstd: Re-enable unsafe decodeSync memory copies (#1168) (#1171)\n\n* zstd: Re-enable unsafe decodeSync memory copies (#1168)\n\ndecodeSyncSimple has hardcoded `const useSafe \u003d true` since #644 (2022),\nmaking the faster `sequenceDecs_decodeSync_{amd64,bmi2,arm64}` asm variants\ndead code that nothing calls. The #644 comment cites \"rare, random crashes\nwith fuzz testing\" from the extended (16-byte-block) copies.\n\nThose copies were only the amplifier. The root cause was an unguarded\nbitReader overread in updateLength that produced out-of-range match\noffsets/lengths; it was fixed three days later in #645, which also added\nthe Go fuzz corpus that has guarded this path ever since. With the source\nof the bad values gone and the +compressedBlockOverAlloc (16-byte) slack\npresent on every output/literal buffer today, the dynamic useSafe guard —\nidentical in shape to the still-active one in executeSimple — is sound.\n\nRestore the dynamic selection. On arm64 (Cortex-A72) DecodeAll improves\nby +8.7% geomean throughput (n\u003d6, p\u003d0.002), up to +16% on text.\n\nBecause a stray 15-byte overrun lands in an adjacent live allocation,\nneither -race nor plain fuzzing can observe it; add a CGO+clang asan fuzz\njob over FuzzDecodeAll/FuzzDecAllNoBMI2 on amd64+arm64 so any future\nmaxSyncLen/allocation regression is caught deterministically.\n\nValidated on amd64 and arm64: full test suite; asm-vs-noasm differential\nfuzz (byte-identical output); and asan over the decode corpus plus\n2.4M mutation execs with no reports.\n\nCo-Authored-By: Claude Opus 4.8 \u003cnoreply@anthropic.com\u003e\n\n* zstd: fix out-of-bounds write in unsafe decodeSync space check\n\nThe per-sequence space check in executeSingleTriple (used by the sync\ndecoder\u0027s inlined execute) requires only outPos+ll+ml \u003c\u003d cap(s.out). The\nunsafe extended copies, however, write in 16-byte blocks and overrun the\nlogical end of a run by up to compressedBlockOverAlloc-1 (15) bytes. When a\nstream\u0027s decoded length lands within the 16-byte over-allocation slack --\ne.g. a malformed frame whose declared content size is a few bytes below what\nits sequences actually produce -- the check passes but the final block copy\nwrites past cap(s.out).\n\nThe overrun is at most 15 bytes and Go\u0027s size-class rounding places it in\nslop that -race and -asan do not poison, so it corrupts an adjacent live\nallocation only intermittently, surfacing later as a crash in the garbage\ncollector rather than at the write. That is exactly the \u0027rare, random\ncrashes with fuzz testing\u0027 #644 disabled the unsafe path for in 2022; it was\nnever the bitReader overread fixed in #645, but this missing copy margin.\n\nReserve compressedBlockOverAlloc in the space check on the unsafe (non-safe)\npath only, so the overrun stays within cap(s.out); a well-formed decode ends\nat cap-16 and is unaffected, while an over-producing stream now errors with\nerror_not_enough_space instead of corrupting memory. The safe copies are\nbounds-exact and keep the tight check. amd64 asm gains one ADDQ per unsafe\nvariant; the safe variants are unchanged.\n\nTestDecodeSyncUnsafeOOB drives the unsafe asm with real captured sequences\ninto a canary-guarded buffer sized to model the under-reported case; it\nfails (canary overwritten past cap) on the unpatched asm and passes here, on\namd64 and arm64. Validated additionally with the full zstd suite and\nFuzzDecodeAll (default and -tags\u003dnounsafe) on go1.25 + go1.26, both arches.\n\nCo-Authored-By: Claude Fable 5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: assert TestDecodeSyncUnsafeOOB exercises the unsafe copy path\n\nReview feedback on the regression test: it verified only that the asm path\nwas dispatched, relying implicitly on the buffer geometry to select the\nunsafe (extended-copy) variant. If the useSafe conditions or the test\nharness allocation ever changed, the test could silently degrade into\nexercising the bounds-exact safe copies while still passing.\n\nExtract decodeSyncSimple\u0027s selection into useSafeDecodeSync() (pure\nrefactor, no behavior change) and have the test call the same helper to\nfail loudly if its buffer sizing would select the safe variant. The canary\nvalidation and asm-path assertion are unchanged; the test still fails via\nthe canary against the unpatched assembly on both architectures.\n\nCo-Authored-By: Claude Fable 5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: fold unsafe-copy margin into the space-check LEAQ\n\nKlaus\u0027s review on #1171 pointed out the compressedBlockOverAlloc margin\ncan be folded into the existing LEAQ displacement instead of a separate\nADDQ. Saves one instruction per unsafe variant on amd64/bmi2; the safe\npath (addMargin\u003d0) and arm64 codegen are unaffected in substance.\n\n---------\n\nCo-authored-by: Claude Opus 4.8 \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "c73af0c12cc767386af8388f30d5aa7428e6dfc8", "tree": "2829b920c876b9ca46b0a59e95f5aeebe155722f", "parents": [ "2602f4afea09fe72f2b58d4ed04d43a6047a0131" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Mon Jul 20 13:55:25 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Jul 20 13:55:25 2026" }, "message": "huff0: add arm64 assembly for Decompress4X/1X via avo lowering (#1172)\n\nGenerate arm64 asm from the existing avo source with the honeycombio/avo\narm64 lowering printer (-arch amd64,arm64), the same mechanism zstd\u0027s\nseqdec/fse asm already uses. The amd64 output is unchanged.\n\nThe Go-side wrappers move to decompress_asm.go, shared between amd64 and\narm64; per-arch files keep only the asm stubs and dispatch (amd64 retains\nthe BMI2 selection for Decompress1X; arm64 lowers the generic twin).\n\nThe avo pin gains exact x86 partial-register semantics (MOVB/MOVW/ADDB/\nADCB/SETcc lower to bit-field inserts preserving untouched bits) — the\nhuff0 loops pack output bytes through AH and accumulate byte counters,\nwhich the previous lowering silently corrupted — plus constant-folding of\nBMI2 control registers (BEXTR with an adjacent constant control becomes a\nsingle UBFX). zstd/seqdec_arm64.s is regenerated with the same printer;\nits amd64 output is byte-identical to before.\n\nOn a 16-core Cortex-A72 (ARMv8.0), huff0 Decompress1X improves 25-34%\n(twain 114-\u003e153 MB/s, digits 128-\u003e161 MB/s); Decompress4X is content-\ndependent (+12% twain, -4% low-entropy). zstd DecodeAll improves 1-7%\nend-to-end (urls.10K +3.8%, html +2.2%, comp-data.bin +6.6%), with\nchecksum (18% of CPU, at scalar xxhash roofline) and sequence decoding\ndominating the remainder.\n\nValidated: full huff0 + zstd test suites on arm64 (Cortex-A72) and amd64;\navo differential execution tests for every new/changed lowering on both\narchitectures.\n\nCo-authored-by: Claude Fable 5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "2602f4afea09fe72f2b58d4ed04d43a6047a0131", "tree": "6997442490922a76e72b4cb4841f6030ab130295", "parents": [ "983acec6dacb98c0d721ca53eb5a904c3c6bec67" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Jul 20 09:36:27 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Jul 20 09:36:27 2026" }, "message": "Update changelog for version 1.19.0" }, { "commit": "983acec6dacb98c0d721ca53eb5a904c3c6bec67", "tree": "4d3d837e5b826bff43ba275730017184029d83aa", "parents": [ "9960386c58450bc4ae5fd67411c394b6a7f23c4c" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Jul 20 09:32:36 2026" }, "committer": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Jul 20 09:32:36 2026" }, "message": "add defensive checks\n" }, { "commit": "9960386c58450bc4ae5fd67411c394b6a7f23c4c", "tree": "52a939bc105e033827fd6e775484d76c549e97cc", "parents": [ "e29d4d389f0e7806e1e84392d0c43453a5048a0f" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Fri Jul 17 09:10:32 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Jul 17 09:10:32 2026" }, "message": "zstd: Validate SnappyConverter literal copies (#1170)\n\nEnable disabled length check on `SnappyConverter.Convert` that could result in out-of-bounds reads buffer reads - or in case or re-use return values from the previous run.\n\nReported by @mfazrinizar" }, { "commit": "e29d4d389f0e7806e1e84392d0c43453a5048a0f", "tree": "cb50e95d7a8e08f682fc65d82661316246ca7266", "parents": [ "c0bbc84762b7d178fe7b1e3a75140793179e9e37" ], "author": { "name": "Ruihao Chen", "email": "joechenrh@gmail.com", "time": "Wed Jul 15 10:14:59 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Jul 15 10:14:59 2026" }, "message": "flate: use `Peek` instead of `ReadByte` for the `bufio.Reader` decode path (#1169)\n\n* flate: read the bufio decode path via Peek instead of per-byte ReadByte\n\nhuffmanBufioReader filled the bit buffer with fr.ReadByte() once per input\nbyte. bufio.Reader.ReadByte does not inline, so each call is a barrier that\nspills the fill loop\u0027s fb/fnb registers -- roughly 1.85x slower than the\n*bytes.Reader variant, whose ReadByte inlines to a slice index.\n\nRead from a Peek window instead: peek bufio\u0027s buffered slice and index it\ndirectly (pbuf[pos]), refilling when drained. Peek only views the buffer, so\nconsumed bytes are Discard()ed before every return, keeping the reader\nposition correct for nextBlock/moreBits and the gzip trailer. peekBufio\nrefills from at most one underlying Read (never Peek(Size()), which would\nblock a streaming/sync writer until the whole buffer fills).\n\nOnly the *bufio.Reader variant changes; the other four generated variants\nare byte-identical.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* flate: drop stray blank lines in the generated read sites\n\nThe $GETBYTE_E$ placeholder started with a newline the template already\nsupplied, so every read site picked up a blank line before its return --\n25 in total, and across all five variants rather than only the bufio one.\n\nWith that fixed the four non-bufio variants are byte-identical to the stock\ngenerated output again (verified by diffing each generated function).\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e" }, { "commit": "c0bbc84762b7d178fe7b1e3a75140793179e9e37", "tree": "158a4d04b0fd7cd85a074337032479a236ebe52b", "parents": [ "efdb6ed96c4b5d23f9bd6c3e739414789c871a9d" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Fri Jul 10 07:36:21 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Jul 10 07:36:21 2026" }, "message": "zstd: bump avo pin, regenerate arm64 asm (#1167)\n\n* zstd: bump avo pin, regenerate arm64 with BMI2 lowering\n\nBumps the pinned avo commit (honeycombio/avo) past today\u0027s fixes: a\nflag-liveness bug and a 32-bit CMPL/TESTL fold bug (neither actually reachable\nfrom this codebase - seqdec\u0027s generated output is byte-identical without\nthem, and CMPL/TESTL aren\u0027t used here at all), plus a new EQ/NE-only\nsub-32-bit CMPW/TESTB lowering that fse_decoder\u0027s buildDtable needs.\n\nChecked whether the sub-word compare/test fold (the general form of which the\nnew avo now refuses to lower) actually produced wrong output in the\nalready-shipped fse_decoder_arm64.s: it didn\u0027t. All three call sites happened\nto operate on registers already clean above the compared width (via an\nupstream sign/zero-extending load, or a preceding bounds check), verified by\ntracing register provenance by hand. Real risk, but mitigated by accident\nrather than by design - which is exactly why the stricter check is worth\nhaving going forward.\n\nThe new avo also prefers the BMI2 twin over the generic one when lowering to\narm64 (arm64 has native flag-free equivalents for the BMI2 idioms), so\nseqdec_arm64.s now lowers sequenceDecs_decode_bmi2/decodeSync_bmi2 instead of\nthe generic variants. Symbol names are unaffected (arm64Name strips the\n_bmi2/_amd64 suffix either way).\n\nValidated: go vet and go test ./zstd/... pass natively on arm64; FuzzDecodeAll\n(26.4M execs), FuzzDecoder (9.6M execs), and FuzzDecAllNoBMI2 pass with zero\ncrashers.\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e\n\n* zstd: bump avo pin, prefer generic (non-BMI2) twin on arm64\n\nBenchmarking the BMI2-preferred seqdec on real hardware (Apple M4, Cortex-A72)\nshowed it\u0027s 13-50% slower than the generic twin, not faster as assumed -- see\nthe follow-up avo commit for why. Bump past that fix; avo\u0027s arm64 lowering now\ndefaults to the generic twin (ARM64PreferBMI2 defaults false), which this repo\ndoesn\u0027t override. seqdec_arm64.s is now byte-identical to what #1160 shipped,\naside from the skip-reason comment text; fse_decoder_arm64.s is unchanged\n(the CMPW/TESTB fix has no BMI2 twin to choose between).\n\nCo-Authored-By: Claude Sonnet 5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "efdb6ed96c4b5d23f9bd6c3e739414789c871a9d", "tree": "d59b4d69d6f67d01d74990dbbb13557bfb9b9299", "parents": [ "f0953f938f77977980c3b9f8c09eec06f2e88a69" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Thu Jul 02 09:35:04 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Jul 02 09:35:04 2026" }, "message": "build(deps): bump the github-actions group with 6 updates (#1166)\n\nBumps the github-actions group with 6 updates:\n\n| Package | From | To |\n| --- | --- | --- |\n| [actions/checkout](https://github.com/actions/checkout) | `6.0.3` | `7.0.0` |\n| [github/codeql-action/init](https://github.com/github/codeql-action) | `4.36.1` | `4.36.2` |\n| [github/codeql-action/autobuild](https://github.com/github/codeql-action) | `4.36.1` | `4.36.2` |\n| [github/codeql-action/analyze](https://github.com/github/codeql-action) | `4.36.1` | `4.36.2` |\n| [actions/setup-go](https://github.com/actions/setup-go) | `6.4.0` | `6.5.0` |\n| [goreleaser/goreleaser-action](https://github.com/goreleaser/goreleaser-action) | `7.2.2` | `7.2.3` |\n\n\nUpdates `actions/checkout` from 6.0.3 to 7.0.0\n- [Release notes](https://github.com/actions/checkout/releases)\n- [Commits](https://github.com/actions/checkout/compare/v6.0.3...v7)\n\nUpdates `github/codeql-action/init` from 4.36.1 to 4.36.2\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/87557b9c84dde89fdd9b10e88954ac2f4248e463...8aad20d150bbac5944a9f9d289da16a4b0d87c1e)\n\nUpdates `github/codeql-action/autobuild` from 4.36.1 to 4.36.2\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/87557b9c84dde89fdd9b10e88954ac2f4248e463...8aad20d150bbac5944a9f9d289da16a4b0d87c1e)\n\nUpdates `github/codeql-action/analyze` from 4.36.1 to 4.36.2\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/87557b9c84dde89fdd9b10e88954ac2f4248e463...8aad20d150bbac5944a9f9d289da16a4b0d87c1e)\n\nUpdates `actions/setup-go` from 6.4.0 to 6.5.0\n- [Release notes](https://github.com/actions/setup-go/releases)\n- [Commits](https://github.com/actions/setup-go/compare/v6.4.0...v6.5.0)\n\nUpdates `goreleaser/goreleaser-action` from 7.2.2 to 7.2.3\n- [Release notes](https://github.com/goreleaser/goreleaser-action/releases)\n- [Commits](https://github.com/goreleaser/goreleaser-action/compare/5daf1e915a5f0af01ddbcd89a43b8061ff4f1a89...f06c13b6b1a9625abc9e6e439d9c05a8f2190e94)\n\n---\nupdated-dependencies:\n- dependency-name: actions/checkout\n dependency-version: 7.0.0\n dependency-type: direct:production\n update-type: version-update:semver-major\n dependency-group: github-actions\n- dependency-name: github/codeql-action/init\n dependency-version: 4.36.2\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n- dependency-name: github/codeql-action/autobuild\n dependency-version: 4.36.2\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n- dependency-name: github/codeql-action/analyze\n dependency-version: 4.36.2\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n- dependency-name: actions/setup-go\n dependency-version: 6.5.0\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: github-actions\n- dependency-name: goreleaser/goreleaser-action\n dependency-version: 7.2.3\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "f0953f938f77977980c3b9f8c09eec06f2e88a69", "tree": "97330bafb959282fc40d1fed9f13ea37e84fc034", "parents": [ "ab2e3cc7ab261c8f287639760d8532afb7125425" ], "author": { "name": "HNO₃", "email": "xiangao.or@isrc.iscas.ac.cn", "time": "Tue Jun 30 13:21:42 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Jun 30 13:21:42 2026" }, "message": "flate: fix buffer formatting in tests (#1165)\n\nSigned-off-by: HNO3Miracle \u003cxiangao.or@isrc.iscas.ac.cn\u003e" }, { "commit": "ab2e3cc7ab261c8f287639760d8532afb7125425", "tree": "5d77297cef0fbc2383e60b4883f2fb1332f33529", "parents": [ "bb2723d55de85353fd5a1817cbc382997a6a75aa" ], "author": { "name": "Liz Fong-Jones", "email": "lizf@honeycomb.io", "time": "Tue Jun 30 11:29:08 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Jun 30 11:29:08 2026" }, "message": "zstd: Add arm64 decoder asm (#1160)\n\n* zstd: add arm64 asm implementation of fseDecoder.buildDtable\n\narm64 previously fell through to the pure-Go fse_decoder_generic.go because\nthe decoder asm is avo-generated and avo has no arm64 backend. This adds a\nhand-written scalar arm64 port of buildDtable, mirroring the avo-generated\nfse_decoder_amd64.s and the pure-Go reference. The kernel is serial bitfield\nwork (no SIMD), so it needs no NEON/SVE and builds on the stock go1.x\nassembler.\n\nCorrectness is covered by a differential test against an in-tree copy of the\ngeneric algorithm over the three RFC predefined distributions, plus the full\nexisting decode suite (which runs on macos-latest/arm64 in CI).\n\ndarwin/arm64 (M4, go1.26), asm vs -tags noasm; the bench\u0027s table reset is\nidentical on both sides so the delta is the buildDtable work:\n\n litLength ~166 vs ~201 ns/op (+18%)\n offset ~118 vs ~141 ns/op (+16%)\n matchLength ~175 vs ~210 ns/op (+17%)\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* zstd: add avo-generated arm64 asm sequence decoder\n\nGenerates the full non-BMI2 zstd sequence decoder for arm64 by lowering the\nexisting amd64 avo program (_generate/gen.go) through a new arm64 backend in\navo (a post-pass.Compile lowering printer; see mmcloughlin/avo#486). No asm is\nhand-written: `go generate` now emits both seqdec_amd64.s and seqdec_arm64.s\nfrom the same IR, so they cannot drift, and CI\u0027s generate+git-diff job enforces\nit.\n\n- seqdec_arm64.s: generated (6 non-BMI2 functions: decode/_56, executeSimple/\n _safe, decodeSync/_safe).\n- seqdec_arm64.go: shim mirroring seqdec_amd64.go (context structs, noescape\n decls, dispatch) without the BMI2 paths.\n- seqdec_generic.go: build tag now excludes arm64.\n\nScalar only — no NEON/SVE (one VLD1/VST1 pair for a 16-byte MOVUPS copy);\nbuilds on go1.26. Full zstd decode suite passes on arm64. On Apple M4,\nBenchmarkDecoder_DecodeAll is ~1.5x faster than the pure-Go fallback on\nsequence-heavy input (kppkn: 469 vs 304 MB/s).\n\nRegeneration requires the avo arm64 backend (mmcloughlin/avo#486) via a local\nreplace directive in _generate/go.mod (not committed).\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* zstd: generate arm64 fse decoder via avo + pin avo fork\n\nSwitches fse_decoder_arm64.s from the hand-written bootstrap to the\navo-generated lowering, so the entire arm64 zstd decoder (fse table build\n+ sequence decoder) is produced from the existing, unmodified amd64 avo\ngenerators via the arm64 lowering printer (mmcloughlin/avo#486). Supersedes\nthe hand-written fse PR #1159.\n\nPins the printer through a replace to the honeycombio/avo fork (which keeps\nthe github.com/mmcloughlin/avo module path, so the replace resolves) until\nthe printer merges upstream and is released, at which point the replace is\ndropped for a normal require.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* zstd: asmfmt-style formatting for generated arm64 asm\n\nRegenerate the arm64 decoder with the updated avo arm64 printer, which now\naligns operands to a common column and emits a blank line before labels —\nmatching avo\u0027s amd64 (goasm) output. asmfmt reports zero non-comment\ndifferences on the arm64 files (the same standalone-comment indentation\nresidual goasm produces on amd64). Repins the avo fork to the printer commit.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* zstd: address review — dedup shims, -arch generation, asm-vs-nonasm fuzz\n\nResponds to maintainer review on the arm64 decoder PR:\n\n- Deduplicate the asm Go wrappers: shared seqdec_asm.go holds the context\n structs, error codes and the decode/decodeSync/executeSimple bodies; each\n arch keeps only its //go:noescape decls and a small dispatch helper (amd64\n selects the BMI2 / 56-bit / safe variant, arm64 the 56-bit / safe variant).\n The fse Go wrapper is identical across arches and collapses into one\n fse_decoder_asm.go (per-arch fse_decoder_{amd64,arm64}.go removed).\n\n- Generate both architectures via avo\u0027s new -arch flag: one\n `go run gen.go -out ../seqdec.s -arch amd64,arm64` writes seqdec_amd64.s and\n seqdec_arm64.s from a single compile, so neither file\u0027s header references the\n other architecture.\n\n- FuzzDecodeAll now cross-checks the assembly decoder against a non-assembly\n path (BMI2 disabled on amd64) for byte-identical output; run with -tags\u003dnoasm\n over the shared corpus to cover assembly-vs-generic on arm64.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* ci: run build matrix and zstd fuzzers on linux arm64\n\nAdds ubuntu-24.04-arm to the build matrix (vet/test/noasm/race across Go\nversions on Linux arm64) and gives the fuzz-zstd job an OS matrix so the zstd\ndecode fuzzers also run on arm64 — exercising the new arm64 decoder assembly\ndirectly in CI, as requested in review.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* zstd: defer BMI2 restore in FuzzDecodeAll asm/non-asm comparison\n\nUse the defer cpuinfo.DisableBMI2()() idiom so BMI2 is re-enabled on every exit\npath (panic/return), matching FuzzDecAllNoBMI2. Addresses review feedback.\n\nCo-Authored-By: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 4.8 (1M context) \u003cnoreply@anthropic.com\u003e\nCo-authored-by: Klaus Post \u003cklauspost@gmail.com\u003e" }, { "commit": "bb2723d55de85353fd5a1817cbc382997a6a75aa", "tree": "6055559dedbf9a52cf363411d2f9c98b1d259150", "parents": [ "539243b8823ee8f03e49969823d57c348c917536" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Tue Jun 30 09:30:04 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Jun 30 09:30:04 2026" }, "message": "Merge commit from fork\n\n`int(r)` would wrap negative for r \u003e MaxInt64, slipping past the bounds check and causing an OOB read in encode.\n\nFix \u0026 add regression test." }, { "commit": "539243b8823ee8f03e49969823d57c348c917536", "tree": "f8f06b7259ba2256650a32c675f93f7844daf2b6", "parents": [ "bd9228c884559c21e572a0eaf827ee903bca805c" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon Jun 29 14:52:46 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Jun 29 14:52:46 2026" }, "message": "s2sx: Clean symlink targets (#1163)\n\nWhen self-extractor is extracting files from TAR validate that symlinks remains inside the destination.\n\nExecutables are already expected to be trusted and since they are embedded with the archive, so there really is no breach of trust.\n\nThis change is mainly to discourage false/pointless AI reports." }, { "commit": "bd9228c884559c21e572a0eaf827ee903bca805c", "tree": "51467d6b53d938c083c2000ff03b3d00088feb33", "parents": [ "4cc176c3325ba1d87aacfdea015a3a2038fefb08" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Wed Jun 17 09:03:42 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Jun 17 09:03:42 2026" }, "message": "Apply go fix and check in CI (#1158)" }, { "commit": "4cc176c3325ba1d87aacfdea015a3a2038fefb08", "tree": "611996d9256fbcb0b50494f9d549cc3b848293ba", "parents": [ "4705fcbaf5155f466ca877527f66e88634cdf1c2" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Wed Jun 17 07:57:10 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Jun 17 07:57:10 2026" }, "message": "zstd: Add true concurrent stream encoding (#1136)\n\n* zstd: Add true concurrent encoding\n\n```go\n// WithConcurrentBlocks enables job-based parallel compression for streams.\n// When enabled and concurrent \u003e 1, input is split into large sections (jobs)\n// that are compressed simultaneously by multiple goroutines.\n// Each non-first job receives an overlap prefix from the previous job for match context.\n// Output is flushed in order, producing a valid single-frame zstd stream.\n//\n// This is incompatible with dictionary encoding.\n// Cannot be changed with ResetWithOptions.\nfunc WithConcurrentBlocks(b bool) EOption {\n```\n\nExample:\n\n| Level | 1 thread | 4 threads | 16 threads | 1T ratio | 16T ratio |\n|---------|:----------:|:------------------:|:-------------------:|:--------:|:---------:|\n| fastest | 783 MB/s | 2950 MB/s (3.8×) | 6939 MB/s (8.9×) | 12.24% | 12.26% |\n| default | 728 MB/s | 2533 MB/s (3.5×) | 5340 MB/s (7.3×) | 10.67% | 10.68% |\n| better | 434 MB/s | 1105 MB/s (2.5×) | 2206 MB/s (5.1×) | 9.14% | 9.21% |\n| best | 129 MB/s | 367 MB/s (2.8×) | 884 MB/s (6.8×) | 8.48% | 8.63% |\n\n* Tweaks\n\n* Just disable when using dictionary encoding.\n\n* Fix Close nil-writer check ignoring concurrent buffer\n\nClose() checked s.w \u003d\u003d nil before the e.o.concurrentBlocks branch. In concurrent mode, Write fills js.filling (not s.filling), so the guard len(s.filling) \u003d\u003d 0 \u0026\u0026 s.nInput \u003d\u003d 0 would incorrectly return nil even with pending data. Fix: moved the concurrentBlocks branch before the nil-writer check, and added a matching nil-writer guard inside closeJobs() that checks js.filling instead of s.filling.\n\n* Clean up reuse\n\n* Share encoder channel with single-shot." }, { "commit": "4705fcbaf5155f466ca877527f66e88634cdf1c2", "tree": "a839f3b8153f823e030820c75c2e64e4203c476b", "parents": [ "e9c2e569c112ecc1ad8ab818ed7328e7178973e4" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Wed Jun 03 08:15:39 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Jun 03 08:15:39 2026" }, "message": "build(deps): bump the github-actions group with 3 updates (#1157)\n\nBumps the github-actions group with 3 updates: [actions/checkout](https://github.com/actions/checkout), [github/codeql-action](https://github.com/github/codeql-action) and [goreleaser/goreleaser-action](https://github.com/goreleaser/goreleaser-action).\n\n\nUpdates `actions/checkout` from 6.0.2 to 6.0.3\n- [Release notes](https://github.com/actions/checkout/releases)\n- [Commits](https://github.com/actions/checkout/compare/v6.0.2...v6.0.3)\n\nUpdates `github/codeql-action` from 4.35.3 to 4.36.1\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/e46ed2cbd01164d986452f91f178727624ae40d7...87557b9c84dde89fdd9b10e88954ac2f4248e463)\n\nUpdates `goreleaser/goreleaser-action` from 7.2.1 to 7.2.2\n- [Release notes](https://github.com/goreleaser/goreleaser-action/releases)\n- [Commits](https://github.com/goreleaser/goreleaser-action/compare/1a80836c5c9d9e5755a25cb59ec6f45a3b5f41a8...5daf1e915a5f0af01ddbcd89a43b8061ff4f1a89)\n\n---\nupdated-dependencies:\n- dependency-name: actions/checkout\n dependency-version: 6.0.3\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n- dependency-name: github/codeql-action\n dependency-version: 4.36.1\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: github-actions\n- dependency-name: goreleaser/goreleaser-action\n dependency-version: 7.2.2\n dependency-type: direct:production\n update-type: version-update:semver-patch\n dependency-group: github-actions\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "e9c2e569c112ecc1ad8ab818ed7328e7178973e4", "tree": "e4e1d1d7af2a6a64ee4357c4157d4a4c87eb78ae", "parents": [ "ecdb779d8745b0b074dcee75d9f7fd2bd524aff8" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Mon May 25 08:06:28 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon May 25 08:06:28 2026" }, "message": "huff0: Allow building table from oversized histogram. (#1156)\n\nA histogram of a block larger than max block size could cause huff0 to panic. Scale table down on BuildCTable." }, { "commit": "ecdb779d8745b0b074dcee75d9f7fd2bd524aff8", "tree": "a9693e1b139b653ef7f9480f8bbb7f663863ef4f", "parents": [ "38efa458d47cc5555ce4a08796ddf2bf5ca22d87" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Thu May 21 20:36:46 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu May 21 20:36:46 2026" }, "message": "huff0: Allow building tables from histogram. (#1155)\n\n* huff0: Allow building tables from histogram.\n\nAdd interface to build tables from the histograms themselves.\n\nAdds direct method to export the current table and helpers for size reuse (also via histograms).\n\n* Check nil" }, { "commit": "38efa458d47cc5555ce4a08796ddf2bf5ca22d87", "tree": "f7a5cdc16f1ab298b3cf42ac03ea34b1269290c5", "parents": [ "5b9abedb0ceca2baed4ae217c23ea175075d86c2" ], "author": { "name": "Sai Asish Y", "email": "say.apm35@gmail.com", "time": "Thu May 21 15:11:11 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu May 21 15:11:11 2026" }, "message": "zip: add NameDecoder callback for legacy encoding rewrite (#1150)" }, { "commit": "5b9abedb0ceca2baed4ae217c23ea175075d86c2", "tree": "f42b2ffd2c602ff6cacf6eebdc0ce11ed5f739e0", "parents": [ "dba8e6f41069ddb1fd8c33e4a325bad06d14f7d7" ], "author": { "name": "Klaus Post", "email": "klauspost@gmail.com", "time": "Thu May 21 15:07:49 2026" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu May 21 15:07:49 2026" }, "message": "flate: Add inflate checkpoints (#1154)\n\nAllows adding an inflate end-of-block callback with checkpoint information. Callback is cleared on reset and cannot be re-applied to an existing Reader.\n\nFor resuming either use `WithResumeFrom` or assert the `ResetCP(io.Reader, flate.InflateCheckpoint) error` interface. The input stream should be forwarded to the compressed offset." } ], "next": "dba8e6f41069ddb1fd8c33e4a325bad06d14f7d7" }