4.3. ADR-003: Memory-Map Uncompressed MDD Files for FlatBuffers Access

4.3.1. Status

Accepted

Date: 2026-03-10

4.3.2. Context

The Classic Diagnostic Adapter loads ECU diagnostic databases stored as MDD files. Each MDD file is a protobuf container whose chunks hold FlatBuffers data compressed with LZMA. At startup every MDD file must be read, the protobuf parsed, the FlatBuffers payload decompressed, and the resulting data kept available for the lifetime of the process.

The main target platform is Linux (onboard automotive ECUs), where RAM is limited and the system may reclaim memory aggressively under pressure via the kernel page cache.

Three strategies were evaluated:

  1. Heap – decompress into heap-allocated Vec<u8> buffers.

  2. MmapSidecar – decompress into separate .fb sidecar files next to the MDD files, then memory-map those sidecar files.

  3. MmapMdd (in-place) – decompress the MDD files themselves once (i.e. during a software update), rewriting them with uncompressed chunk data, then memory-map the MDD files directly with zero-copy protobuf decoding.

4.3.3. Decision

We will use the MmapMdd (in-place) strategy: MDD files are decompressed once and are subsequently used read-only via mmap. The protobuf layer uses prost’s Bytes support (Bytes::from_owner(mmap)) so that chunk data fields are zero-copy slices into the memory-mapped file – no heap allocation is required for the FlatBuffers payload.

Before the atomic rename of a rewritten MDD file, the written data is verified by re-parsing the temporary file and comparing SHA-512 checksums of every chunk against the expected values.

4.3.4. Rationale

4.3.4.1. Performance Comparison

Benchmarking was conducted on Linux 6.18.2-arch2-1 (x86_64, i5-7200U CPU) with 32 GB RAM using 68 MDD files (~47 MB compressed, 242 MB uncompressed), Rust 1.92.0, --release profile, ~3 minutes idle warm-up, and swap disabled.

RSS Comparison (KB)

Strategy

Idle

Under Pressure

Disk Usage

Heap (baseline)

486,900

469,320

47 MB

MmapSidecar

307,552

171,904

~282 MB

MmapMdd (in-place)

152,780

118,988

242 MB

Note

The MmapMdd implementation uses memmap2::Advice::Random (MADV_RANDOM) immediately after mmap() to disable read-ahead for the sparse FlatBuffers vtable lookups that dominate runtime access. This avoids a libc dependency – the hint is set directly via memmap2 before ownership is transferred to Bytes::from_owner().

4.3.4.2. MmapMdd Advantages over Heap

  1. RSS under pressure: -75 % (119 MB vs 469 MB)

    All FlatBuffers data is backed by the MDD file on disk. Under memory pressure the kernel cleanly drops those pages and re-reads them on demand – no swap I/O required. On the heap strategy, anonymous pages can only be compressed or swapped, incurring significant I/O overhead with a modest -3.7 % reduction.

  2. Idle RSS: -69 % (153 MB vs 487 MB)

    The zero-copy protobuf decode (Bytes::from_owner(mmap)) avoids copying every bytes field to the heap. Chunk data fields are slices into the mmap, so there is no second copy of the decompressed data in memory.

    Setting MADV_RANDOM via memmap2 prevents the kernel from prefetching adjacent pages during random-access FlatBuffers queries, keeping idle RSS well below the heap baseline.

4.3.4.3. MmapMdd Advantages over MmapSidecar

  1. Simpler file management

    No additional .fb sidecar files to create, track, or clean up. The MDD files are the single source of truth. This eliminates an entire class of consistency bugs (stale sidecar, missing sidecar, partial write).

  2. Lower RSS under pressure (119 MB vs 172 MB)

    The in-place strategy benefits from zero-copy protobuf decoding (Bytes::from_owner) which the sidecar approach did not use. All data – protobuf metadata and FlatBuffers payloads – lives in the single mmap, giving the kernel a unified region to evict.

  3. Lower idle RSS (153 MB vs 308 MB)

    Zero-copy decoding avoids duplicating chunk data on the heap, resulting in 50 % lower idle RSS than the sidecar approach.

  4. Less extra disk space (+195 MB vs +235 MB)

    Sidecar files duplicated the FlatBuffers payload alongside the original compressed MDD. In-place rewriting replaces the compressed data, so the growth is only the difference between compressed and uncompressed sizes.

4.3.4.4. Runtime CPU Performance (perf Profiling)

In addition to the RSS benchmarks above, perf profiling was conducted under a realistic end-to-end workload on the target Linux system to compare the MmapMdd implementation against the main (heap/compressed) branch.

Test setup

  • Platform: Linux target (i5-7200U), Release build

  • Workload: CDA started, 20s idle warm-up, then perf attached for profiling, followed by filling ~54 GB of memory with garbage data (two 27 GB bytearray allocations in parallel to induce memory pressure), then a full ECU flash session via DoIP.

  • Tool: perf stat attached to the running process (after warm-up) with events: cycles, instructions, faults, cache-references, cache-misses.

perf stat Results – Main vs MmapMdd (under load)

Metric

Main (compressed)

MmapMdd (decompressed)

Delta

cycles

448,473,942

437,102,422

-2.5 %

instructions

194,934,031

194,316,925

-0.3 %

IPC (insn/cycle)

0.43

0.44

+2.3 %

page faults

340

1,206

+255 % (see note)

cache-references

25,838,457

26,031,415

+0.7 %

cache-misses

18,142,040 (70.21 %)

18,478,329 (70.98 %)

-0.77 pp

wall time

42.19 s

42.19 s

negligible

Note

The higher page-fault count in MmapMdd (1,206 vs 340) reflects the kernel mapping mmap pages on first access rather than heap pages already loaded in at startup. The absolute numbers are negligible (< 1,500 faults over ~42 s) and have no measurable impact on wall time.

Runtime profiling conclusions

Under realistic ECU-flash load with concurrent memory pressure both implementations are effectively equivalent in CPU efficiency (within 2.5 % of each other) and identical in wall time. The workload is dominated by network I/O (DoIP) rather than database access, so the expected RSS savings of MmapMdd (-75 % under pressure) are realized without any runtime CPU regression.

perf report call-graph analysis confirmed that the top hotspots (alloc::vec::in_place_collect, flatbuffers::vtable::VTable::get, cda_database::datatypes::DiagService::find_request, mimalloc internals) are present in both branches with similar weights, confirming that no new hot paths were introduced by the MmapMdd implementation.

4.3.4.5. Trade-offs

  • Disk usage increases: MDD files grow from ~47 MB to 242 MB (~5.1x). This is a one-time cost during the software update and is acceptable on the target platform where storage is less constrained than RAM.

  • MDD files are modified: The original compressed MDD files are replaced with uncompressed versions. This is acceptable because:

    • Decompression happens once during a controlled update step, not at runtime. - This will be implemented at a later point in time in the update plugin, for now the CDA does this at runtime.

    • SHA-512 verification ensures data integrity before the atomic rename.

4.3.5. Consequences

4.3.5.1. Positive

  • 75 % RSS reduction under memory pressure compared to the heap baseline (119 MB vs 469 MB), critical for embedded Linux targets with limited RAM.

  • 69 % lower idle RSS (153 MB vs 487 MB) due to zero-copy protobuf decoding and MADV_RANDOM via memmap2 to suppress wasteful read-ahead during sparse FlatBuffers lookups.

  • Zero-copy data path: mmap –> Bytes –> FlatBuffers – no intermediate heap allocations for the diagnostic payload.

  • Single file, single source of truth: no sidecar files to manage, eliminating consistency and cleanup issues.

  • Atomic, verified writes: SHA-512 checksums and temp-file + rename ensure data integrity even if the update is interrupted.

  • Read-only at runtime: after the initial update, MDD files are opened read-only, compatible with read-only filesystems or integrity-checked partitions.

  • No libc dependency: MADV_RANDOM is set via memmap2 before ownership transfer, avoiding the need for direct libc::madvise() calls.

4.3.5.2. Negative

  • 5.1x disk usage increase for the MDD database directory.

  • One-time decompression cost i.e. during software update or first startup

  • Platform dependency: relies on OS-level mmap, page cache behaviour, and madvise(2) support (POSIX systems), although the latter is guarded by a cfg flag, so the CDA still works on platforms without MADV_RANDOM support (e.g. Windows) possibly with higher idle RSS.

4.3.6. Alternatives Considered

4.3.6.1. Heap (Baseline)

Decompress FlatBuffers data into heap-allocated Vec<u8> buffers. Simplest implementation but RSS remains high (~487 MB idle, ~469 MB under pressure). Anonymous heap pages cannot be cleanly evicted by the kernel – they must be compressed or swapped, incurring I/O overhead. Unsuitable for memory-constrained targets.

4.3.6.2. Separate Flatbuffer file (Sidecar)

Decompress into separate .fb files and memory-map those. Achieves good pressure behaviour (~172 MB) but introduces additional file management complexity: sidecar files must be created, kept in sync with MDD files, and cleaned up on updates. Uses more disk space (+235 MB) because both compressed MDD and uncompressed sidecar exist side by side. The sidecar approach was prototyped and benchmarked but rejected in favour of the simpler in-place strategy.

4.3.7. References