Architecture

A map of how the codebase fits together, not a manual. Start here, then follow the file/type names into the source: the code is expected to carry its own meaning through naming and flow, so read it directly rather than looking for a commentary alongside it.

What the code deliberately does not carry is the evidence behind a decision — benchmark numbers, memory-footprint measurements, approaches that were tried and rejected. That lives in docs/, and STYLEGUIDE.md says why.

The four format families

Each format has its own workbook (CsvReader for CSV) and a writer implementing IWorkbookWriter<TSheet> (src/ExcelReader.Core/Writer/IWorkbookWriter.cs):

Format	Workbook	Writer	Sheet/row writer
XLSX	`XlsxWorkbook`	`XlsxWorkbookWriter`	`XlsxSheetWriter`/`XlsxRowWriter`
XLSB	`XlsbWorkbook`	`XlsbWorkbookWriter`	`XlsbSheetWriter`/`XlsbRowWriter`
XLS	`XlsWorkbook`	`XlsWorkbookWriter`	`XlsSheetWriter`/`XlsRowWriter`
CSV	`CsvReader`	`CsvWorkbookWriter`	`CsvSheetWriter`/`CsvRowWriter`

CSV has one extra layer: CsvWriter is the low-level RFC4180 writer (buffered rows straight to the stream, no sheets/styles/shared-strings machinery); CsvWorkbookWriter adapts it to the shared IWorkbookWriter<CsvSheetWriter> contract, exposing exactly one sheet.

All four open through Excel.Open/OpenAsync, which return an IExcelWorkbook and take an optional ExcelFileFormat. The three signed formats are detected from the file's first bytes; CSV carries no signature, so it is named rather than detected, and its dialect rides along in ExcelReaderOptions.Csv. That one options object is what the native ABI's xl_open_options maps onto, so ReadApi.Open dispatches formats rather than reimplementing them.

A workbook, a sheet and an enumerator are three separate things. The workbook (XlsxWorkbook and its siblings, IExcelWorkbook) owns the shared state: the file or stream, the sheet list, the shared strings and styles. A sheet (XlsxSheet and its siblings, IExcelSheet) is a readonly struct of workbook plus index; it holds no resources, and workbook.Sheets, FirstSheet, SheetAt and TryGetSheet hand them out without opening anything. Iterating a sheet creates an enumerator (XlsxWorkbook.Enumerator), the one place that streams rows and the only mutable part, used by one thread at a time. Lifetime is reference counted (ReaderLifetime): the workbook holds one reference and each live enumerator another, so disposing the workbook closes it to new sheets and enumerators but releases the file, stream and pooled buffers only when the last enumerator is disposed. Shared state is published lazily and race-safely, which is what lets sheets of one workbook be read in parallel.

On top of all four workbooks sits the typed-parsing layer (src/ExcelReader.Core/Parser/): ExcelParser<T>, built by ExcelParser.FromAttributes (reflection), ExcelParser.Generated (the source-generated map) or ExcelParser.Build (a runtime-configured map). Its model may be a class, a struct or a ref struct; a ref struct model binds directly to Cell.Value spans — zero allocation for the container and, for span-typed columns, for the values too. It consumes Row/Cell from any sheet uniformly.

Parallel sheet reading

SheetParallel (src/ExcelReader.Core/Reader/SheetParallel.cs) reads one sheet on several threads. It is opt-in, takes any IExcelSheet, validates its arguments and dispatches on the sheet's concrete type:

sheet	engine
`CsvSheet`	the CSV engine (`ParallelCsvProcessor`, `ParallelCsvEnumerable`), described below. Its source comes from `CsvReader.TryGetChunkSource`, which the native layer also uses.
`XlsxSheet`, `XlsbSheet`	the pipeline engine (`src/ExcelReader.Core/Parser/SheetPipeline`)
anything else, and every fallback case	one sequential runner over `IExcelRowEnumerator`

The fallback cases give the same results with no error: a degree of parallelism of 1, a sheet under 256 KB uncompressed, XLS, a table view (ExcelTable.AsSheet), and a CSV that cannot be partitioned (a stream that is not a file or a non-UTF-8 encoding). The caller owns the workbook; the engine holds a ReaderLifetime lease until the task completes or the enumeration ends.

The CSV engine is not folded into a shared partition abstraction. It is tuned, and its model, a static byte-range plan with speculative starts and re-reads, differs from the pipeline's dynamic, exact chunks. What the two share is the contract types (IRowRecord, IRowAccumulator, RowAggregation, RowAction) and the binders in Parser/ParallelCsv (MappedProjection, MappedAggregation, MappedCallback, RecordAggregation, RecordCallback), which pick the parser's CSV type map for a CsvSheet and the general one otherwise.

The pipeline engine

zip entry -> producer (1 thread) -> bounded queue -> N workers -> ordered merge
             inflate, cut at the     row-aligned      parse +      by chunk index
             last row boundary       chunks, indexed  row action

Inflate is sequential, so the engine splits the work around it instead of through it. One producer thread owns the entry stream, opened through ZipIndex.OpenEntryStream so the decompressed-byte limits count where they do on the sequential path. It inflates into pooled buffers and cuts each at the last row boundary, copying the partial tail into the next buffer; chunks default to 256 KB (SheetPipelineSource.DefaultChunkBytes). In XLSX the boundary is the last <row followed by a name boundary, and the producer stops at the sheet-data end tag, so the rest of the part is neither inflated nor buffered. XLSB has no sync marker, so the producer walks every record with Biff12RecordReader and remembers the last BrtRowHdr; it stops at BrtEndSheetData.

Chunks go through a bounded System.Threading.Channels queue, about twice the worker count, so a full queue blocks the producer and peak memory is roughly capacity times chunk size whatever the sheet's size. dop - 1 workers (at least one) each parse a chunk in place with the format's own enumerator over ReadOnlyMemory<byte>, call Seed(), then run the row action for each row. Shared strings are loaded before the pipeline starts, and a prologue reads up to HeaderRow with an ordinary enumerator, so header rows never reach a chunk. The engine's one primitive is an async stream of (index, state, failure) in chunk order, reordered through a ring of slots as ParallelCsvEnumerable does; AggregateAsync folds the states with Combine, ForEachAsync ignores them, and ParseAsync uses a List<T> per chunk and yields its items.

The XLSX row boundary is a candidate, not a proof: the same bytes can sit in CDATA, a comment or malformed markup. So every chunk enumerator reports whether it ended cleanly, on a row boundary with no row or tag cut short. A run of clean chunks equals the sequential read by induction. A chunk that does not end cleanly stops before delivering its cut-short row; the merge keeps that chunk's rows, cancels the pipeline and reads the rest with one sequential enumerator, skipping the rows already delivered, and later chunks' results are dropped. This recovery re-inflates the sheet and only happens on such sheets. It leaves ParseAsync and AggregateAsync exact, but ForEachAsync may already have called back for dropped chunks, so it is exactly once only for a well-formed XLSX sheet. XLSB frames records from the first byte as the sequential reader does, so it has no such case.

The measured gain is bounded by inflate, which stays on one thread: see "Parallel intra-sheet parsing" in reader-performance.md. Parallel inflate would replace the producer only; the queue, workers and merge stay.

The CSV engine

CsvSheet takes the older path (src/ExcelReader.Core/Parser/ParallelCsv/ParallelCsv*.cs), over the same typed-parsing machinery, for sources large enough that partitioning pays for itself. ParallelCsvFactory decides eligibility, a seekable, UTF-8, large-enough source, and falls back to the ordinary sequential ExcelParser<T> otherwise, so callers get identical output either way.

When it partitions, CsvChunkPlan slices the data range into four chunks per worker (not one — see the file's comments on why one-per-worker would buffer up to the whole output) and hands them out through a lock-free pull queue. Each worker reads its chunk through a RangedFileStream, a positional, non-seeking view over one shared SafeFileHandle (RandomAccess.Read, thread-safe, no per-worker file handle). A chunk's nominal start is only a guess — record boundaries do not respect chunk boundaries — so CsvBoundaryResolver has each worker also resolve where its own first real record begins.

ParallelCsvEnumerable<T>.GetAsyncEnumerator is where the guesses get corrected and the output gets ordered. It walks chunks in index order; a chunk is only trusted once its predecessor has confirmed where the chunk actually starts, and a chunk whose guess was wrong is reparsed from the confirmed offset before anything downstream sees it. That correction can cascade into the following chunk, which the loop's confirmed-offset threading handles without special-casing. Boundary confirmation is therefore strictly ordered — it is a correctness requirement — but this also happens to be the merge that yields rows in file order, identical to the sequential parser.

Whether a mode other than the ordered merge above would be worth adding as an opt-in was a measurement question the design left open, against a >=10% throughput bar. It is now settled for the confirmation-decoupling variant, and the ordered merge stays the only shipped mode.

The first measurement, on a thermally-limited mobile CPU, was discarded: flat-to-worse at mid-range degrees of parallelism and improving only at the extremes, which is throttling under sustained multi-threaded load rather than the algorithm's own scaling. The clean re-run (Ryzen 7 5700X, 8C/16T, CsvParallelParseBenchmark, MediumRun) shows the path scaling monotonically instead — 3.95x on the conversion-heavy corpus and 2.61x on narrow ints at dop 16, saturating past dop 8 as it becomes bandwidth-bound.

With that baseline trustworthy, the variant itself was measured: confirmation decoupled from consumption, publishing confirmed rows to a bounded channel rather than yielding them inline, so confirmation can run ahead instead of lockstepping with a slow consumer. It lost at every consumer speed whose numbers were tight enough to read — 2.9% behind the ordered merge with a trivial consumer, 23% and 28% behind as consumer cost rose past the pipeline's own — while allocating more. The slowest consumer leg produced no usable signal, because modelling a slow consumer with Thread.SpinWait makes it contend for the same cores as the workers; a blocking consumer would need a different harness. This reproduces the earlier ad hoc experiment's finding on hardware that can be trusted, so the variant is discarded rather than kept as a second mode.

Still unresolved, and untested by any of the above: whether true unordered emission would fare differently. The ordered merge is the safer default and the only mode either public overload can reach, so it stays absent a specific reason to revisit.

SheetParallel.ForEachAsync exposes the same partitioning to a per-row callback rather than an accumulator type. It accepts a ref struct row, which the enumerating SheetParallel.ParseAsync cannot — not because of the interface (IAsyncEnumerable<T>'s T is declared allows ref struct on net9.0+), but because a row holding spans into a worker's buffer cannot cross to the consumer thread at all.

That constraint also settles what an ordered ref struct enumerator would cost. Since a worker cannot convert a row it must hand onward, conversion would run on the consumer thread — and conversion is the floor here: the ref struct aggregate path, which shares this projection code, costs 833.8 ms single-threaded on the conversion-heavy corpus, against 272.5 ms for the typed path at dop 8. Such an API would be ~3.1x slower than what already ships while adding nothing over the sequential ref struct parse, so it was not built. Ordered delivery at full speed needs workers to convert into a per-chunk arena of packed fields that the consumer rehydrates; that remains unbuilt.

Because partitions start at guessed record boundaries, a partition whose guess was wrong is read again and the callback sees a whole partition's records twice — 1 MiB to 64 MiB of them, not a handful near the seam. A wrong start also shifts every field boundary, so on a source with quoted fields the discarded pass can hand the callback values that appear nowhere in the file. For an accumulator both are invisible: the discarded instance is thrown away.

A guess is only wrong when the offset falls inside a quoted field. CsvChunkWorker.GuessStart first reads the byte before the chunk start and returns the chunk start unchanged when it already sits on a record boundary, because CsvBoundaryResolver.FindRecordStart only ever scans past a terminator and would otherwise overshoot by a whole record on every chunk that happens to be aligned — which, at a 1 MiB minimum chunk size, is every chunk of a fixed-width file. That leaves quote-free sources exact, and the repeat documented as a caller obligation for the rest rather than designed out. Removing it there too needs a quote-parity prefix pass over the bytes before any parsing starts; that pass is a barrier and would cost roughly half the dop-16 runtime unless pipelined, which is more machinery than the hazard currently warrants.

Shared plumbing

Reader internals that would otherwise be duplicated four times over live in one place:

CellAccumulator — pooled per-row cell storage (raw decoded UTF-8 text + a CellDesc[] describing each cell's column/type/style/offset). Used by every format's enumerator. Also hosts the shared BIFF numeric-error-code → text lookup (#DIV/0! etc.) for XLS/XLSB.
PooledStreamRowEnumerator — abstract base centralizing the pooled-buffer lifecycle (BufferedStreamCursor + CellAccumulator, Fill/FillAsync/Ensure wrappers) that every format's enumerator subclasses. MoveNext/MoveNextAsync stay concrete per format — this only removes the buffer-lifecycle boilerplate, not the parsing itself.
BufferedStreamCursor — the refill/compact-or-grow cursor behind the XLSX/XLSB/CSV forward-only stream enumerators. Has a second constructor for the in-memory-ZIP path that wraps an already-fully-decompressed ReadOnlyMemory<byte> instead of a Stream (Eof = true immediately, no refills).
WorkbookLookups — small lookups that were once duplicated identically across readers: sheet name→index, sheet-index bounds checks, date-style flags, shared-string offsets, and (ZIP formats only) worksheet entry resolution plus prefetch/limit-counting stream composition. Takes arrays as parameters rather than requiring a shared interface, since each format's backing arrays differ in shape.
LimitChecks — the DoS/resource-limit guards (ExcelReaderOptions/CsvReaderOptions), including the single buffer-growth-cap function (NextBufferSize) every pooled buffer in the stack grows through, so one limit policy governs all of them consistently.

Encrypted OOXML workbooks

A password-protected .xlsx/.xlsb is not a ZIP: it is an OLE/CFB container holding an EncryptionInfo descriptor and an EncryptedPackage (the real ZIP, encrypted). src/ExcelReader.Core/Crypto/ turns one back into a stream the ordinary readers consume:

CfbContainer — the CFB parse, factored out of XlsCompoundFile so one parse can yield named streams. XlsCompoundFile.OpenWorkbook is now a thin wrapper over it, which is why the XLS path kept its behavior and its fuzz coverage unchanged.
EncryptionDescriptor — parses EncryptionInfo and dispatches. Version 4.4 is agile (AES-CBC, 4096-byte segments, XML descriptor). Versions 2.2/3.2/4.2 share a binary header+verifier used by both ECMA-376 standard encryption (AES-ECB/SHA-1) and RC4 CryptoAPI, so the cipher is decided by EncryptionHeader.algId, not by the version — standard is supported, RC4 is rejected. Anything else is rejected outright. This is the one parser in the codebase that uses XmlReader rather than hand-rolled scanning; the file header explains why.
AgileKeyDerivation / StandardKeyDerivation — password to key, plus the verifier check that distinguishes "wrong password" from "corrupt file". Standard's SHA-1 and 50,000 iterations are fixed by the scheme rather than stated in the file, which is why MaxPasswordSpinCount has nothing to bound there.
PackageCipher — the keyed half of a scheme: derive, decrypt one aligned window, verify integrity if the scheme has any. DecryptedPackageStream and EncryptedPackageOpener hold one of these and name no scheme, which is why adding standard encryption did not touch their segmenting or their ZIP handling.
DecryptedPackageStream — a read-only seekable Stream. Agile derives a per-segment IV from the segment index, so ZipArchive finds its central directory without the package ever being materialized. One 4 KiB segment is cached, which is enough because ZIP reads are sequential within an entry.

XlsxWorkbook and XlsbWorkbook are untouched by any of this: they receive a stream that happens to decrypt. Writing encrypted workbooks is a separate step, Excel.EncryptPackage/EncryptPackageAsync in PackageEncryptor.cs — it wraps a plaintext package (written by any of the ordinary writers) in an agile-encrypted CFB container, the inverse of DecryptedPackageStream. There is still no round-trip check between the two directions: they don't share a derivation implementation to guard against a shared bug, so the fixtures in tests/ExcelReader.Tests/data/encrypted/ (paired with plaintext produced by an independent implementation) remain the only correctness oracle for decryption specifically.

XLSX and XLSB packages are written by Writer/Zip/ZipPackageWriter, not ZipArchive. It writes the layout ZipArchive uses in create mode (patched local headers on seekable streams, data descriptors otherwise, Zip64 only when a field overflows). CompressionLevel.Fastest entries go through DeflateEncoder, a fixed-Huffman deflate with a stored-block fallback, and are checksummed by Crc32, which folds 128 or 256 bits at a time with PCLMULQDQ. Optimal and SmallestSize keep zlib through DeflateStream. When a write faults, Abandon still writes a central directory for the entries that completed, as disposing ZipArchive did; a failed entry write marks the whole archive failed, so nothing more can be added to it.

WriteRecordsParallelAsync and WriteTableParallelAsync (Writer/Parallel/) cut a record source into batches. Each batch is formatted on a pool thread by a detached copy of the sheet writer, which has its own buffer and never touches the ZIP, then compressed into a byte-aligned deflate segment that refers to nothing outside itself. The calling flow appends segments in order through ZipEntryStream.AppendSegmentAsync, which merges CRCs with Crc32.Combine. Once an entry has received a segment, everything written to it is a segment, including later serial rows, so no compressor ever carries history across one.

The `excelreader` CLI

src/ExcelReader.Cli/ is a thin dotnet tool shell (excelreader) over Core's public API — it parses no spreadsheet bytes of its own. It splits in two on purpose: Commands is a one-line-per- command adapter whose XML doc comments ConsoleAppFramework's source generator turns into argument parsing, routing and --help, while CliCommands holds the bodies as plain functions over explicit writers. The split keeps the tested surface free of the framework's static output hooks, so the CLI tests run in parallel like every other test class. ConsoleAppFramework is compile-time only (PrivateAssets), so the published tool's only runtime dependency besides ExcelReader.Core is Spectre.Console, used for sheets/schema's tables and convert's progress spinner on a real terminal.

Rendering follows Commands/CliCommands's own split, one level further: Commands picks plain vs. interactive per call (Console.IsOutputRedirected/IsErrorRedirected), so a script gets the exact same tab-separated text and stderr line the tool always wrote, unchanged. Two small always-stderr helpers back that split - ErrorConsole (a Spectre IAnsiConsole pinned to Console.Error, since Spectre's own default instance targets stdout, which convert may be using for the converted bytes themselves) and ColorizingErrorWriter (a TextWriter that renders CliCommands.Execute's one-line failure in red through ErrorConsole on a terminal, or passes it through byte-for-byte otherwise). Both live in ExcelReader.Cli, not CliCommands.cs - the interactive/plain decision is Console-shaped state, exactly what that file's tests are built to never touch.

Why workbooks are split into partial classes

XlsxWorkbook and XlsbWorkbook are large enough that one file would be unwieldy, so each is split by concern rather than by size:

XlsxWorkbook.cs / XlsbWorkbook.cs — fields, constructors, the sheet list, dispose.
XlsbWorkbookParts.cs (XLSB only) — parses workbook.bin for the sheet names, paths and visibility.
*.Loading.cs (XLSX only) — one-time workbook-level XML parsing (sheets, shared strings, date1904).
XlsxWorkbook.Memory.cs (XLSX only) — the shared-string parse over an already-decompressed in-memory part, so it never suspends even under await foreach.
*.Styles.cs (XLSX only) — builds the cellXfs-index → is-date-style table.
*.Enumerator.cs — the nested Enumerator: the actual streaming row/cell parser. By far the largest file in each workbook.

All partials of one workbook share one field set (C# partial classes are one type), so e.g. .Loading.cs's shared-string parse populates fields the nested Enumerator in .Enumerator.cs reads back. XlsWorkbook follows a reduced version of the same split (.Strings.cs for its string tables, no .Loading.cs/.Memory.cs). The sheet values live in their own small files (XlsxSheet.cs, XlsbSheet.cs, XlsSheet.cs, CsvSheet.cs).

The sync/async twin convention

Hot-path search/refill primitives (e.g. IndexOf/IndexOfAsync/IndexOfSlowAsync in XlsxWorkbook.Enumerator.cs) come in three tiers, not one generic async method:

A blocking sync loop for the sync caller.
An async method whose common case — the data is already in the buffered window — is a synchronous check returning an already-completed ValueTask, so no async state machine is allocated on the hot path.
A separate ...SlowAsync method holding the actual await-in-a-loop, split out so the rare awaiting branch doesn't bloat the fast path's IL/JIT inlining.

Parsing a row (ParseRowInWindow) has no async twin at all — it is pure span work over the buffered window, so both MoveNext and MoveNextAsync call the same synchronous parse. It finds the row's cells and its </row on one forward walk, and returns false without committing anything if the row runs past the window; only the refill loop around it (ParseRowBody / ParseRowBodySlowAsync) differs between the two callers.

A parity test suite (tests/ExcelReader.Tests/Reader/SyncAsyncParityTests.cs) asserts identical cell snapshots across sync / async-open / GetAsyncEnumerator for all four formats, guarding against the twins drifting apart.

The `Row`/`Cell` ref-struct lifetime model

Row and Cell are public readonly ref structs: zero-allocation views aliasing the reader's own pooled buffers (the accumulator's cell/value arrays, the shared-string buffer, or — for XLSX's bare <v> fast path — the live read buffer directly). Being ref struct lets them hold ReadOnlySpan<T> fields, and the compiler physically prevents a caller from doing anything that would outlive them: they can't be boxed, stored in a field, captured in a closure, or crossed over an await/yield.

Validity window: exactly one enumeration step. MoveNext/MoveNextAsync resets the accumulator and may compact or resize the buffer, so a Row/Cell from step N is invalidated the instant step N+1 starts.

Escape hatch: Cell.GetString() materializes the value as a real string (deduplicated for repeated shared-string cells via an internal cache), TryFormat copies raw text into a caller-owned buffer without allocating a string, and TryGetDouble/TryParse<T>/TryGetDateTime extract plain value types — all safe to keep past the row's lifetime since none of them are spans.

Typed-model instantiation: three tiers, different tradeoffs

ExcelParser<T>/ExcelEnumerable<T> create one row model per row via TypeMapInfo<T>.CreateInstance() (Parser/Internal/TypeMapInfo.cs), and how that instance gets created depends on how T was mapped:

[ExcelSerializable] (source-generated) or a fluent map that called .Factory(static () => new T()) — the factory is a plain compiled new T(), the cheapest possible per-row allocation.
A plain struct with no explicit parameterless constructor — no factory at all; default(T) is used directly, since it's byte-for-byte what new T() would produce.
Attribute/reflection-driven mapping of a class T (TypeMapper<T>.Build, Parser/Internal/TypeMapper.cs) — falls back to Activator.CreateInstance<T>() per row. This is deliberately not an Expression.Compile()-built factory: that was tried and reverted because the JIT/expression-tree compilation cost dominated cold start (first-use latency), which matters more for this reflection path's typical caller (one-off scripts, not long-running services) than the steady-state per-row cost of Activator dispatch.

If per-row allocation throughput matters more than cold start for your workload, prefer [ExcelSerializable] or an explicit .Factory(...) over plain attribute/reflection binding on a class model — it sidesteps Activator.CreateInstance<T>() entirely.