Kill the naked int/long size: ZstdByteSize value type#98
Conversation
| /// [#toIntExact()] rather than silently truncating. | ||
| /// | ||
| /// @param value the byte size or count, non-negative | ||
| public record ZstdByteSize(long value) { |
There was a problem hiding this comment.
maybe it should be called ZstdSize?
|
|
||
| List<byte[]> samples = ...; // representative records | ||
| ZstdDictionary dict = ZstdDictionary.train(samples, 8 * 1024); | ||
| ZstdDictionary dict = ZstdDictionary.train(samples, new ZstdByteSize(8 * 1024)); |
There was a problem hiding this comment.
maybe we should also expose a better way to build 8K or 1M constants? this could be useful for tests internally but also for end users to define the buffer size for copies, etc
3de81d6 to
b4773df
Compare
|
Re: ZstdSize naming — kept Re: KB/MB constants — added |
Introduces ZstdByteSize, a record wrapping a byte size/count as a validated non-negative long, so that holding one guarantees validity instead of leaving callers to wonder whether a raw long from the library still needs checking. Provides toIntExact() for callers that must narrow to an int because the result ends up in a byte[] (throws ArithmeticException past Integer.MAX_VALUE, matching Math.toIntExact), and ofKiB(long)/ofMiB(long) factories for common buffer sizes instead of spelling out arithmetic like `new ZstdByteSize(16 * 1024)`. Replaces every naked int/long size parameter, and every long return value that represents a byte count, across the public API: Zstd.compressBound, Zstd.estimateCompressContextSize/estimateDecompressContextSize/ estimateCompressDictSize/estimateDecompressDictSize, Zstd.decompressedSize(MemorySegment), ZstdDictionary.train/trainCover/ trainFastCover/finalizeFrom, ZstdOutputStream.withPledgedSize, ZstdFrame.decompressedSize/decompressedBound, and sizeOf() on ZstdCompressContext/ZstdDecompressContext/ZstdCompressStream/ ZstdDecompressStream/ZstdCompressDictionary/ZstdDecompressDictionary. Centralizes interpretation of zstd's raw frame-content-size results as ZstdByteSize factories rather than scattered sentinel checks: - ZstdByteSize.fromFrameContentSize(long) turns the ZSTD_getFrameContentSize/ZSTD_findDecompressedSize/ZSTD_decompressBound sentinels (and any other negative reading, i.e. an unsigned value >= 2^63) into a ZstdException instead of a bogus negative size or the wrong exception type. - ZstdByteSize.fromFrameHeaderContentSize(long) backs ZstdFrameHeader.contentSize(), now Optional<ZstdByteSize> instead of OptionalLong. It treats any negative reading as absent (not just the sentinel), since ZSTD_getFrameHeader never validates the declared size's magnitude, so a hostile header can wrap unsigned >= 2^63 into a negative long the same way a raw content-size read can. The CONTENTSIZE_UNKNOWN/CONTENTSIZE_ERROR sentinel constants move from Zstd to become a private implementation detail of ZstdByteSize. Zstd.decompress(byte[]) and decompress(byte[], ZstdByteSize) both route through a shared decompressInternal(byte[], int), so a caller-supplied bound too large for a byte[] fails with the documented ZstdException instead of a bare ArithmeticException, matching the frame-header path. ZstdDictionary's train/trainCover/trainFastCover/finalizeFrom do the same via a shared toBufferLength helper, and their public parameter names are now consistent (maxDictBytes throughout, no -Bound suffix on a subset of the overloads). Breaking change; the library is pre-1.0 so no compatibility shims are added. Closes #96.
b4773df to
0e12130
Compare
|
Ran an independent review over the whole diff and applied the findings that survived verification:
Full build/tests/checkstyle/javadoc all green. Pushed as |
Summary
ZstdByteSize, a record wrapping a byte size/count as a validated non-negativelong, withtoIntExact()for callers that must narrow to anintbecause the result ends up in abyte[](throwsArithmeticExceptionpastInteger.MAX_VALUE, matchingMath.toIntExact).int/longsize parameter with it:Zstd.decompress,Zstd.compressBound,Zstd.estimateCompressDictSize/estimateDecompressDictSize,ZstdDictionary.train/trainCover/trainFastCover/finalizeFrom,ZstdOutputStream.withPledgedSize.ZstdByteSizefactories rather than scattered sentinel checks:ZstdByteSize.fromFrameContentSize(long)turns theZSTD_getFrameContentSize/ZSTD_findDecompressedSize/ZSTD_decompressBoundsentinels — and any other negative reading (an unsigned value ≥ 2^63) — into aZstdExceptioninstead of a bogus negative size or the wrong exception type. Used byZstd.decompress(byte[]),Zstd.decompressedSize(MemorySegment), andZstdFrame.decompressedSize/decompressedBound, which now returnZstdByteSizeinstead of unwrapping immediately back tolong.ZstdByteSize.fromFrameHeaderContentSize(long)backsZstdFrameHeader.contentSize(), nowOptional<ZstdByteSize>instead ofOptionalLong.CONTENTSIZE_UNKNOWN/CONTENTSIZE_ERRORsentinel constants move fromZstdto become a private implementation detail ofZstdByteSize.ZstdFrame's other size-returning methods (compressedSize,decompressionMargin,headerSize) are unaffected — they never construct aZstdByteSize, no sentinel semantics involved.[Unreleased].Closes #96.
Design notes
long-backed type, not two split by width — narrowing happens at the four array-bounded call sites viatoIntExact(); native/streaming totals passlongstraight through. See discussion on Consider a size value type for byte-size parameters #96.IllegalArgumentExceptioninstead of the documentedZstdException). Covered by a new regression test (ZstdTest.UntrustedInput.rejectsAFrameDeclaringAContentSizeWithTheSignBitSet).Test plan
./mvnw -pl zstd,integration-tests,benchmark -am clean verify— build success, all unit + integration tests pass./mvnw -pl zstd validate— checkstyle clean./mvnw -pl zstd javadoc:javadoc— zero output (failOnError/failOnWarnings)ZstdByteSizeTestcovering construction, negative rejection,toIntExact()in-range/overflow🤖 Generated with Claude Code