SBK

Storage Benchmark Kit

Exact latency recorders in PerL: architecture, memory, and performance

Abstract

PerL retains every valid integer latency as an exact frequency distribution. It does not sample observations. The storage structure behind that distribution therefore affects the benchmarker’s own CPU consumption, heap footprint, garbage production, cache locality, and the delay required to produce periodic percentiles.

This paper studies PerL’s three exact recorders:

On the measured 16-vCPU Intel Xeon Platinum 8462Y+ virtual machine, JDK 25.0.2 with ZGC and compact object headers, a 4,096-value update workload produced:

Recorder Updates/second Allocation/update
ArrayLatencyRecorder 561.9 million effectively 0 B
LongHashMapLatencyRecorder 439.0 million 0.001 B
HashMapLatencyRecorder 64.6 million 47.855 B

The array delivered 28.0% higher throughput than the primitive map for this dense bounded workload. The primitive map delivered 580.0% higher throughput than the boxed map and removed effectively all hot-path allocation. These are measurements of one controlled environment, not universal constants. The portable conclusions are structural: array indexing avoids hashing; primitive storage avoids boxing; and a reusable primitive sorting buffer avoids a large allocation on every percentile extraction.

1. Research questions

This study answers six questions:

  1. How does each recorder represent the exact latency distribution?
  2. Which work happens for every completed storage operation?
  3. Which work happens only at a reporting-window boundary?
  4. How do latency range and number of distinct observed values affect memory?
  5. When should PerL select a dense array or sparse primitive map?
  6. Which performance statements are architectural facts, and which are environment-sensitive measurements?

The authoritative source files are:

2. Position in the SBK measurement pipeline

Benchmark workers do not update the latency distribution directly. They send completed timestamps through a PerL channel. One recorder thread consumes the timestamps and owns both the periodic and whole-run latency windows.

flowchart LR
    W1["Writer or reader 1"] --> Q["PerL timestamp queues"]
    W2["Writer or reader 2"] --> Q
    WN["Writer or reader N"] --> Q
    Q --> C["Single PerL consumer"]
    C --> V{"Latency valid and<br/>inside configured range?"}
    V -->|no| D["Increment invalid or<br/>discard counter"]
    V -->|yes| R["Exact latency recorder"]
    R --> P["Periodic percentile extraction"]
    P --> L["System, CSV, Prometheus,<br/>Web, or gRPC logger"]

    classDef producer fill:#dbeafe,stroke:#1d4ed8,color:#000
    classDef consumer fill:#dcfce7,stroke:#166534,color:#000
    classDef decision fill:#fef3c7,stroke:#a16207,color:#000
    class W1,W2,WN producer
    class C,R,P consumer
    class V decision

This ownership model is essential: all three recorders are deliberately @NotThreadSafe. The queues provide cross-thread publication; the consumer alone mutates a recorder. Adding atomic counters or locks inside the recorder would add overhead without improving correctness under the intended topology.

3. Shared semantics

All three classes extend LatencyRecordWindow. They share the same validation and accounting implemented by the LatencyRecorder/LatencyWindow hierarchy:

The normal call is:

recordLatency(startTime, events, bytes, latency)
    |
    +-- record(events, bytes, latency)
    |       |
    |       +-- update totals
    |       +-- classify invalid / lower / higher
    |       +-- return true only for a valid in-range latency
    |
    +-- reportLatency(latency, events)
            |
            +-- update exact frequency bucket

events is the frequency increment. A single timing observation can represent more than one record, so the distribution stores latency -> record count, not merely latency -> number of method calls.

3.1 Exact percentile calculation

At the end of a window, each implementation emits latency/count pairs in ascending latency order. LatencyPercentiles walks their cumulative counts:

flowchart TB
    B1["100 ns x 4 records"] --> C1["Cumulative interval 0..4"]
    B2["105 ns x 7 records"] --> C2["Cumulative interval 4..11"]
    B3["110 ns x 6 records"] --> C3["Cumulative interval 11..17"]
    C1 --> X["Locate configured percentile indexes"]
    C2 --> X
    C3 --> X
    X --> O["Exact integer-bucket percentile values"]

No recorder approximates an accepted integer latency. Precision is determined by the configured time unit and the inclusive [lowLatency, highLatency] range. Values outside that range are counted explicitly but do not enter the percentile distribution.

3.2 Lifecycle invariant

The production lifecycle is:

startWindow -> record many values -> copyPercentiles -> reset -> startWindow

copyPercentiles() consumes the current distribution. The array implementation zeros each visited non-empty slot during extraction. The map implementations clear their maps after extraction. reset() resets common counters and window time.

This order matters. ArrayLatencyRecorder.reset() does not scan and clear its entire backing array; doing so would turn every reset into an O(range) memory write. Code that bypasses the production lifecycle must extract before reusing the array window.

4. ArrayLatencyRecorder

4.1 Representation

For inclusive bounds L and H, the recorder allocates:

slot count R = H - L + 1
counter bytes = 8R
index(latency) = latency - L
flowchart LR
    LAT["Observed latency = 105"] --> SUB["105 - lowLatency 100"]
    SUB --> IDX["index = 5"]
    IDX --> A["latencies[5] += count"]
    A --> RANGE["Update minIndex / maxIndex"]

The dense layout contains no keys. The array position is the key:

lowLatency = 100

array index       0    1    2    3    4    5    6   ...   10
latency value   100  101  102  103  104  105  106   ...  110
record count      4    0    0    0    0    7    0   ...    6

4.2 Hot path

For each valid measurement:

  1. subtract lowLatency;
  2. update minIndex and maxIndex;
  3. increment one long array element.

There is no hash calculation, collision probe, object allocation, pointer chase, or resize.

4.3 Extraction

The recorder scans from the smallest observed index to the largest observed index. For each non-zero slot it:

  1. reconstructs the latency as index + lowLatency;
  2. feeds the bucket into LatencyPercentiles;
  3. optionally copies it to another recorder;
  4. zeros that slot.

The minIndex/maxIndex bounds avoid scanning unused prefixes and suffixes. They do not avoid holes inside the observed span.

4.4 Best use cases

Use the array when:

Avoid it when:

4.5 Important API constraint

recordLatency() validates the configured bounds before indexing and is the normal safe API. Direct callers of reportLatency() must provide an in-range value. The current direct method checks the upper index but not a negative index; a latency below lowLatency can therefore cause a negative array access. Production input reaches it only after validation or through compatible recorder-to-recorder aggregation.

5. HashMapLatencyRecorder

5.1 Representation

HashMapLatencyRecorder supplies new HashMap<Long, Long>() to MapLatencyRecorder:

flowchart LR
    L["primitive latency"] --> BK["box Long key"]
    BK --> H["HashMap bucket / node"]
    C["primitive count"] --> BV["box Long value"]
    BV --> H
    H --> E["Long key -> Long count"]

JDK HashMap provides expected constant-time get and put with a suitable hash distribution. Iteration costs are proportional to capacity plus size, and its default load factor is 0.75. See the JDK 25 HashMap specification.

5.2 Hot path

Long value = latencies.get(latency);
if (value == null) {
    latencies.put(latency, count);
} else {
    latencies.replace(latency, value + count);
}

Generic Map<Long, Long> requires boxing. For latencies and counts outside the small wrapper cache, an update can allocate temporary key wrappers and a replacement value wrapper. The map also retains a node and key object for each distinct latency.

The measured steady update workload allocated 47.855 bytes per operation. At 64.6 million updates/second this corresponded to approximately 2.9 GB/s of allocation in JMH, producing 36 observed ZGC collections across the measured forks.

Compact object headers reduce the size of many Java objects, but do not remove the wrappers or map nodes. SBK enables the JDK 25 product feature -XX:+UseCompactObjectHeaders; see JEP 519.

5.3 Extraction

The boxed map creates a sorted stream over its key set:

key set -> boxed stream -> sort -> iterator -> map lookup per key

This produces ascending exact buckets, but sorting is O(D log D) for D distinct latencies and creates temporary stream/sorting machinery. The map is cleared after extraction.

5.4 Current role

HashMapLatencyRecorder is useful as:

It is not selected by PerlBuilder in production. For a sparse exact window, the builder selects LongHashMapLatencyRecorder.

6. LongHashMapLatencyRecorder

6.1 Representation

The recorder uses Eclipse Collections 13.0.0 LongLongHashMap. Keys and values reside in one interleaved primitive array:

keysValues:
+--------+--------+--------+--------+--------+--------+
| key 0  | value 0| key 1  | value 1| key 2  | value 2|
+--------+--------+--------+--------+--------+--------+

The implementation uses open addressing and linear probing. It grows before the table exceeds approximately 50% occupancy. Latencies 0 and 1 are handled as sentinel keys by the collection implementation.

flowchart LR
    L["primitive long latency"] --> HASH["spread and mask"]
    HASH --> SLOT{"table slot"}
    SLOT -->|same key| ADD["addToValue(count)"]
    SLOT -->|empty| PUT["write primitive key and count"]
    SLOT -->|collision| PROBE["linear probe"]
    PROBE --> SLOT

No Long wrapper, entry node, or replacement value object is required.

6.2 Hot path

long value = latencies.get(latency);
if (value == 0) {
    latencies.put(latency, count);
} else {
    latencies.addToValue(latency, count);
}

SBK supplies positive record counts. Under that invariant, zero means “missing” for the recorder’s update path. A general-purpose caller that stores zero or negative bucket counts would violate this assumption.

The measured update path reached 438.8 million operations/second and allocated 0.001 B/op, effectively eliminating the boxed map’s hot-path garbage.

6.3 Reusable percentile sorting buffer

Hash iteration is not ordered, so exact percentile calculation still needs sorted keys. The recorder keeps a long[] sortedLatencies:

flowchart TB
    M["Primitive hash table with D keys"] --> SIZE["Read D"]
    SIZE --> CAP{"sorting buffer capacity >= D?"}
    CAP -->|no| GROW["Allocate one larger long array"]
    CAP -->|yes| REUSE["Reuse existing array"]
    GROW --> COPY["Copy primitive keys"]
    REUSE --> COPY
    COPY --> SORT["Arrays.sort(buffer, 0, D)"]
    SORT --> WALK["Walk sorted keys and cumulative counts"]
    WALK --> CLEAR["Clear primitive table; retain capacity"]

The buffer grows only when a window exceeds the previous high-water mark. It is reused by later windows. This converts extraction from repeated O(D) temporary allocation to retained O(Dmax) storage.

For 65,536 keys, the measured comparison was:

Extraction mechanism Time Allocation
Allocating primitive sorted list 320.282 us 524,379 B
Reusable primitive array 173.992 us 49.701 B

The reusable array reduced measured extraction time by 45.7% and allocation by 99.99% in this environment.

6.4 Clear and retention behavior

LongLongHashMap.clear() fills its complete backing array with zeros. It does not shrink the table. This has two intentional effects:

copyPercentiles() clears the map. The subsequent reset() checks notEmpty() before clearing, avoiding a second full-array fill over an already-empty large table.

6.5 Best use cases

Use the primitive map when:

7. Builder selection and production roles

PerlBuilder.buildLatencyRecordWindow() estimates the dense-array payload:

estimated bytes = (highLatency - lowLatency) * 8

If the estimate is below maxArraySizeMB and the range is indexable by an integer, it chooses ArrayLatencyRecorder. Otherwise it chooses LongHashMapLatencyRecorder.

flowchart TD
    INPUT["lowLatency, highLatency,<br/>maxArraySizeMB"] --> RANGE["range = high - low"]
    RANGE --> FIT{"range x 8 fits array budget<br/>and range < Integer.MAX_VALUE?"}
    FIT -->|yes| ARRAY["Periodic window:<br/>ArrayLatencyRecorder"]
    FIT -->|no| PRIMITIVE["Periodic window:<br/>LongHashMapLatencyRecorder"]
    TOTAL["Whole-run window"] --> PRIMITIVE_TOTAL["Always LongHashMapLatencyRecorder"]

    classDef decision fill:#fef3c7,stroke:#a16207,color:#000
    classDef dense fill:#dcfce7,stroke:#166534,color:#000
    classDef sparse fill:#dbeafe,stroke:#1d4ed8,color:#000
    class FIT decision
    class ARRAY dense
    class PRIMITIVE,PRIMITIVE_TOTAL sparse

Default memory configuration:

maxArraySizeMB=64
maxHashMapSizeMB=192
totalMaxHashMapSizeMB=256

The periodic window may therefore use an array, while the whole-run window uses the primitive map to avoid reserving memory for the full theoretical latency range.

7.1 Selection examples

Range Inclusive slots Dense counter payload Likely choice
0..180,000 ms 180,001 1.37 MiB Array
0..5,000,000 ns 5,000,001 38.15 MiB Array
0..180,000,000 ns 180,000,001 1.34 GiB Primitive map
0..180,000,000,000 ns 180,000,000,001 1.31 TiB Primitive map

These examples describe storage feasibility, not measurement quality. Operators should choose bounds that include meaningful expected latencies without allowing rare pathological values to dominate memory.

8. Algorithmic complexity

Let:

Operation Array Boxed HashMap Primitive LongLongHashMap
Construct O(R) zeroing O(1) O(1) small initial table
Record existing latency O(1) worst case expected O(1) expected O(1)
Record new latency O(1) worst case expected O(1), occasional resize expected O(1), occasional rehash
Additional allocation per normal update none wrapper garbage, possibly nodes none
Extract ordered buckets O(S) O(C + D log D) O(C + D log D)
Extraction temporary storage O(1) O(D) plus stream machinery O(Dmax) retained reusable buffer
Clear during extraction included in O(S) O(C + D) implementation-dependent O(C) array fill
Reset after extraction O(1) normally O(C) clear O(1) when already empty
Fixed/retained distribution memory O(R) O(C + D) objects/references O(C + Dmax) primitives

Worst-case hash-table operations can degrade under excessive collisions. The expected constant-time classification assumes a suitable hash spread.

9. Memory analysis

9.1 Array

The recorder reports exactly 8R bytes for counter payload:

R = 5,096 slots in the JMH configuration
payload = 5,096 x 8 = 40,768 bytes

The Java array header and small recorder object are additional but do not scale with observations. No per-distinct-latency object exists.

9.2 Primitive map

Eclipse Collections stores key/value pairs in an interleaved long[] and maintains a maximum occupancy of roughly 50%. Around a stable power-of-two capacity, the backing table therefore requires approximately:

2 table slots per distinct key
x 2 longs per slot
x 8 bytes per long
= approximately 32 bytes per distinct latency

The reusable sorting buffer adds up to 8 bytes per high-water distinct key. Consequently, a stable large recorder is approximately 40 bytes per distinct latency plus small object/array headers, with step changes at power-of-two resizes.

For 4,096 distinct latencies:

interleaved hash table: approximately 128 KiB
reusable sorted keys:                    32 KiB
combined primitive payload:             160 KiB

9.3 Boxed map

The boxed representation retains:

It additionally allocates temporary wrappers during updates. Exact retained bytes depend on JVM object layout, compressed references, table capacity, counts, and whether wrapper-cache values are reused. The documented JMH run’s allocation result of 47.855 B/update is therefore a stronger observed statement than a universal retained-byte formula.

9.4 Configured budget versus actual heap

Both map recorders increment mapBytesCount by 16 bytes for each distinct latency, representing one primitive key and one primitive count. This is a logical payload estimate used by isFull(), not a measurement of the backing collection’s full retained heap.

For the primitive map, open-addressing capacity and the sorting buffer make actual retained primitive storage closer to roughly 40 bytes per distinct key at stable capacity. For the boxed map, nodes, references, wrappers, and temporary garbage increase the difference further.

Therefore:

maxHashMapSizeMB is a recorder payload policy,
not a strict JVM heap or RSS limit.

Capacity is retained after clear(), so heap use follows the largest observed window rather than the current empty-window size.

9.5 Approximate dense-versus-sparse memory crossover

Using only the dominant primitive payloads:

array memory         ~= 8R
primitive-map memory ~= 40D

The approximate equality is:

8R = 40D
D / R = 0.20

If more than roughly 20% of a bounded range becomes populated, the dense array can use less retained primitive storage than the map plus sorting buffer. If only a small fraction is populated, the primitive map can save substantial memory. Power-of-two table growth, array/object headers, and high-water retention shift the exact crossover, so 20% is a design estimate rather than a runtime guarantee.

Range slots R Distinct values D Density Array payload Approx. primitive-map payload Memory-oriented choice
1,000,000 1,000 0.1% 7.63 MiB 39 KiB Primitive map
1,000,000 100,000 10% 7.63 MiB 3.81 MiB Primitive map
1,000,000 250,000 25% 7.63 MiB 9.54 MiB Array
1,000,000 1,000,000 100% 7.63 MiB 38.15 MiB Array

Memory is not the only criterion. The array extraction cost follows observed span S, while map extraction sorts D keys. A sparse distribution clustered inside a narrow span can still favor the array; a few values spread across the entire range can favor the map even when the array fits.

10. Experimental method

The reproducible benchmark is LatencyMapBenchmark.java. Run:

./gradlew :perl:latencyMapPerformanceTest

The task:

The benchmark cycles across 4,096 exact values beginning at latency 1,000. Values are outside the small boxed-Long cache. It measures:

  1. individual frequency updates;
  2. complete 4,096-value record-and-extract windows;
  3. allocating versus reusable primitive key extraction for 65,536 keys.

10.1 Environment

Component Measured value
CPU Intel Xeon Platinum 8462Y+, 16 vCPUs
Topology VMware VM, 1 NUMA node
Memory 61 GiB
OS Ubuntu Linux, kernel 5.15.0-181
JVM Oracle JDK 25.0.2+10-LTS-69
GC ZGC
Object headers -XX:+UseCompactObjectHeaders
JMH 1.37

JMH is the OpenJDK harness for JVM nano-, micro-, milli-, and macrobenchmarks; see the OpenJDK JMH project.

11. Performance results

11.1 Update hot path

Recorder Mean throughput 99.9% confidence interval Allocation
Array 561.874 M ops/s 552.030..571.719 M effectively 0 B/op
Primitive map 439.011 M ops/s 435.194..442.828 M 0.001 B/op
Boxed map 64.560 M ops/s 38.833..90.287 M 47.855 B/op

Derived comparisons:

The boxed result has a wide confidence interval because its high allocation rate introduced GC-dependent bimodality. Even its upper confidence bound is well below the primitive map’s lower bound in this run.

11.2 Complete window

The complete-window benchmark records 4,096 distinct values and then calculates percentiles while clearing the distribution. Results are populated from the same raw JMH report and include both hot-path and boundary work:

Recorder Time/window Allocation/window
Array 9.531 us 0.093 B
Primitive map 32.346 us 48.317 B
Boxed map 213.952 us 429,058.098 B

This measure is more representative of PerL than update throughput alone. The update benchmark isolates the cost paid for every record; the window benchmark includes ordering, percentile traversal, and clearing. For this dense window, the array completed in 29.5% of the primitive-map time. The primitive map completed in 15.1% of the boxed-map time and avoided about 419 KiB of allocation per window.

11.3 Primitive extraction optimization

Mechanism, 65,536 keys Mean time Allocation
Allocating sorted primitive list 320.282 us 524,379.133 B/op
Reusable primitive array 173.992 us 49.701 B/op

The reusable implementation required 45.7% less time and avoided approximately 512 KiB of allocation per extraction.

12. Correctness evidence

The recorders are tested for exact equivalence rather than approximate tolerance.

ArrayLatencyRecorderTest.java compares array and primitive-map behavior for:

LatencyMapRecorderTest.java compares primitive and boxed map behavior, verifies production builder selection, exercises reusable sorting-buffer growth, and tests repeated windows.

These tests establish semantic equivalence for the supported lifecycle. JMH establishes performance observations, not correctness.

13. Decision guide

flowchart TD
    START["Need exact integer latency buckets"] --> FIT{"Inclusive range safely fits<br/>within dense-array budget?"}
    FIT -->|no| PRIMITIVE["Use LongHashMapLatencyRecorder"]
    FIT -->|yes| DENSE{"Expected observations dense or<br/>observed span narrow?"}
    DENSE -->|yes| ARRAY["Use ArrayLatencyRecorder"]
    DENSE -->|no| TEST["Benchmark array scan versus<br/>primitive map on target workload"]
    TEST -->|array wins| ARRAY
    TEST -->|map wins or memory matters| PRIMITIVE
    BASELINE["Need a correctness/reference baseline"] --> BOXED["Use HashMapLatencyRecorder<br/>only in tests/benchmarks"]

    classDef decision fill:#fef3c7,stroke:#a16207,color:#000
    classDef recommended fill:#dcfce7,stroke:#166534,color:#000
    classDef baseline fill:#fee2e2,stroke:#991b1b,color:#000
    class FIT,DENSE decision
    class ARRAY,PRIMITIVE recommended
    class BOXED baseline
Workload Recommended recorder Reason
Small bounded millisecond range Array fastest direct indexing, small fixed array
Dense microsecond range below budget Array cache-friendly exact counters
Nanosecond range spanning billions Primitive map memory proportional to observed values
Sparse values separated by large holes Primitive map avoids scanning/reserving holes
Whole-run aggregation Primitive map theoretical range can be enormous
Correctness oracle or regression baseline Boxed map simple JDK representation

14. Limitations and threats to validity

  1. Microbenchmarks do not include timestamp queue traffic, logger formatting, storage SDK calls, or operating-system I/O.
  2. Results apply to the declared CPU, JVM, GC, and flags. Different CPUs, collectors, heap sizes, or JDK builds can change throughput.
  3. The benchmark has one recorder thread because production ownership is single-threaded. It does not measure unsupported concurrent mutation.
  4. The update workload has 4,096 repeating values. Different cardinality, hash distribution, and density change cache behavior.
  5. gc.alloc.rate.norm values near zero are profiler/harness noise and should be interpreted as “no structural per-operation allocation,” not literally a fractional object.
  6. Reported array memory is counter payload; reported map budgets are logical payload estimates, not complete retained-heap measurements.
  7. The reusable sorting buffer trades lower repeated allocation for retained high-water memory.
  8. Exact percentile retention can require much more memory than an approximate histogram. That is a deliberate precision tradeoff.

15. Conclusions

The three recorders occupy distinct architectural roles:

The production policy—array when the range fits, primitive map otherwise—is sound. A future density-aware policy could improve decisions for ranges that fit in memory but contain very sparse observations. Such a change should be driven by end-to-end PerL measurements, not by data-structure theory alone.

The most durable result is not a single throughput number. It is the mapping between workload shape and representation:

dense bounded domain  -> direct primitive array
sparse large domain   -> primitive open-addressed map
boxed object graph    -> reference/testing baseline

References

  1. SBK PerL source: perl/src/main/java/io/perl.
  2. OpenJDK, Java Microbenchmark Harness.
  3. Oracle, JDK 25 HashMap API.
  4. OpenJDK, JEP 519: Compact Object Headers.
  5. Eclipse Collections, LongLongHashMap.
  6. Gil Tene, HdrHistogram for the optional approximate whole-run extension used by PerL.