- A spend-management budget is usually prepared outside the commerce platform.
- Finance exports a workbook containing properties, GL codes, fiscal periods, allocated budget, encumbered spend and remaining availability.
- The file is not a simple table to paste into a database.
- For a property-procurement marketplace built with Medusa, we created a tracked background import for this fiscal workbook.
The client problem
A spend-management budget is usually prepared outside the commerce platform.
Finance exports a workbook containing properties, GL codes, fiscal periods, allocated budget, encumbered spend and remaining availability. Buyers need those numbers inside the purchasing journey so approvals and reporting reflect the organisation's current plan.
The file is not a simple table to paste into a database. Every property code must map to the correct organisation site. Every GL code must map to an account the platform knows. Each property-and-account combination must contain a coherent twelve-period year. A worker may fail after some rows are written. The same email or admin action may be retried.
For a property-procurement marketplace built with Medusa, we created a tracked background import for this fiscal workbook. It validates the document structure, resolves organisation-owned properties and GL accounts, writes mapped budget lines in bounded chunks and records exactly where the import is reading, parsing, mapping or writing.
The buyer sees budget context in commerce. Finance keeps its familiar source format. Operations get an import they can inspect and recover instead of a long-running black box.
The workbook is a business contract
Budget files often look flexible because a human can visually understand them. Software needs a stricter agreement.
The parser expects columns for asset name and code, GL group and code, description, allocated budget, encumbered spend, available budget, percentages and fiscal period. Header normalisation tolerates whitespace differences and a known alias for the encumbered column, but required business fields remain required.
Codes are preserved as text even when spreadsheet software has represented them as numbers. Currency and percentage values are converted deliberately; invalid required numbers fail with the row location.
This balance matters. The importer should absorb harmless spreadsheet formatting variation without guessing what an absent property code or malformed budget means.
Before a single budget line is written, the platform knows that the workbook contains the vocabulary required by the downstream spend model.
Require one coherent fiscal year
A partial or mixed budget year can produce plausible but dangerous results.
The parser reads fiscal labels in the explicit form Month N,YYYY. Months must fall between 1 and 12. It collects report years across the file and the importer requires exactly one year.
For every asset-code and GL-code pair, the workbook must contain exactly twelve unique fiscal periods. Duplicate property-account-period rows are rejected. A group with eleven or thirteen periods is rejected with a sample explaining what is incomplete.
These checks protect the shape finance intended. Without them, a missing month might be interpreted as zero budget, while a duplicate month might double the available amount.
The platform does not merely accept rows that parse. It verifies that each budget series forms a complete annual schedule before mapping it into commerce.
Turn email attachment into a durable attempt
Finance should not need to learn a custom technical upload ritual to update budgets.
The wider ingestion flow can receive the workbook through an authorised email attachment. Once admitted for the spend-budget feature, the platform creates an import attempt linked to the organisation, inbound upload, file ID and filename.
The record begins in importing state with a queued phase and timestamps. The Medusa workflow runs the memory-heavy workbook processing as a background step with bounded automatic retries.
This keeps the event subscriber short and returns control to the surrounding system quickly. The durable attempt, not an email callback or open browser request, owns the work.
The same importer can also operate from a known file reference. Email is an entry point; organisation, file and import identity remain the actual contract.
Make every phase observable
“Importing” is not enough information when a workbook takes time or fails.
The importer records phases such as queued, reading, parsing, mapping, writing, completed and failed. Each phase carries an update timestamp and relevant counters. Reading records file size and read duration. Parsing records source row count. Mapping records matched volume. Writing records rows completed.
Memory snapshots and phase timings are retained as operational metadata rather than promoted as public performance claims. They help an engineer distinguish a file-read problem from a parser error or a long write phase.
The admin list exposes import status, row totals, imported rows, mapped and unmapped counts, validation errors, timestamps and source-email context. It is paginated and organisation-scoped.
An operator can therefore answer “where did it stop?” without searching logs by filename or asking finance to resend the workbook blindly.
Read the file with an explicit timeout
Background work still needs boundaries.
The importer obtains a stream through the configured file provider and reads it into a buffer for the spreadsheet parser. Opening and consuming that stream are both wrapped in a configurable timeout. If reading stalls, the stream can be destroyed and the import moves to a failed state with a specific message.
This prevents a lost provider response from occupying a worker indefinitely. The default timeout is deliberately a configuration detail, not a promise that every file will complete within a particular business duration.
The provider abstraction also keeps the workflow independent of one storage URL. The durable source file ID is checked against the import record, and the backend reads through the file service.
File transport remains an implementation boundary. The import attempt remains the business boundary.
Map only inside the active organisation
An asset code or GL code is meaningful only within its owner.
The mapping phase loads all properties and GL accounts for the import's organisation in bounded pages. Properties are indexed by their operational location code; accounts are indexed by GL code.
Each parsed budget row must find both. A row with an unknown property increments the unmapped-property count. A row with an unknown GL account increments the unmapped-GL count. Rows missing either relationship are not written as active budget lines.
This prevents two dangerous shortcuts: creating phantom business entities from a finance export and resolving a matching code from another organisation.
The import can still complete with mapped lines while reporting skipped, unmapped rows. Finance and operations receive a precise reconciliation task instead of a fabricated relationship.
Preserve the financial context of every mapped line
A successfully mapped row carries more than one budget number.
The stored line connects import, organisation, property and GL account. It retains asset and account labels, description, period year and month, fiscal-period label, allocated budget, encumbered spend, available budget, used and available percentages and the raw source row.
That model supports both computation and explanation. A purchasing workflow can use the typed monetary values. An operator investigating a discrepancy can see which original workbook row produced them.
The raw row does not replace the structured fields; it complements them. The platform has a stable schema for budget logic without throwing away the evidence finance supplied.
This is how an external planning document becomes native commerce context while remaining traceable to its source.
Write in chunks and report real progress
Once mapping succeeds, budget lines are created in bounded chunks.
After each chunk, the import record updates rows_imported and the writing-phase metadata. The user does not have to wait for the final row before seeing that work is progressing.
Chunking also avoids one giant create payload and gives the workflow natural progress checkpoints. Property and GL lookups are paginated before mapping so the organisation's reference data does not depend on a hidden first-page limit.
The architecture supports scalable processing without requiring an invented claim about workbook volume or benchmark speed. Its guarantee is structural: bounded reference paging, bounded create chunks and durable progress.
When the last mapped row is written, the attempt records the report year, final counters, import timestamp and completed phase.
Never leave a failed half-budget active
A worker can fail after one or more chunks have been written.
If those partial lines remained active while the import was marked failed, buyers could see a budget containing only the first part of the workbook. That is more dangerous than having no new import.
The failure path therefore cleans up active lines associated with the attempt, resets the imported-row count to zero and stores a validation error plus failed phase metadata. The same cleanup runs before an attempt starts processing, protecting a retry from stale partial lines.
This creates an attempt-level boundary: the imported budget is either the complete set of mapped rows from the validated workbook or the attempt is failed without active partial lines.
It is a targeted consistency guarantee around this importer, not a slogan that every external action across the platform is globally atomic.
Retry only when retry is demonstrably safe
Automatic retries handle transient background failures, but operators also need a controlled recovery path.
The admin retry endpoint is intentionally narrow. It verifies the organisation and import, requires a linked inbound email upload and refuses retry when active budget lines or a non-zero imported count already exist.
For a zero-row failed or stale attempt, it marks the old import failed with an operator reason, resets the linked inbound upload to ready and re-emits the normal ingestion event. The platform creates or resumes work through the same canonical path rather than executing a special hidden importer.
This avoids double-writing a budget that may already have progressed. The recovery tool is useful because it is conservative about the state it can safely replay.
Operators get a button for the scenario the system can prove is clean, not a universal “try again” switch over financial data.
Connect imported budgets to purchase decisions
The value of the import appears after onboarding.
Budget lines connect a property, GL account and fiscal month to allocated, encumbered and available values. Those dimensions are the same ones buyers select when allocating purchases and approval workflows evaluate when deciding whether spend requires escalation.
The commerce platform can combine the imported baseline with marketplace purchasing activity and external spend context. A supervisor sees why a request affects a particular property-account budget rather than comparing only the basket total with one generic threshold.
The importer does not itself prove future financial outcomes. It supplies the governed budget facts the rest of the purchasing model needs.
Finance maintains the plan in its established workbook. Commerce uses an admitted, mapped and traceable version of that plan at decision time.
Why Medusa was the right foundation
Medusa supplied the workflow engine, file module and commerce lifecycle around which budget-aware purchasing could be built. The client needed a spend-management module and ingestion contract that standard ecommerce does not provide automatically.
We added parser, import and line models, organisation mappings, background workflow execution, operational status routes and safe retry. These components sit beside the order and approval flows that consume the result.
The architecture avoids turning the budget workbook into a disconnected reporting database. Its property and GL identities are the same domain identities used by carts, allocations and purchase history.
That is the value of an extensible commerce engine: external finance data can become a controlled input to real purchasing behaviour without replacing the core order system.
A practical fiscal-budget import checklist
Before importing budget workbooks, define:
- Which columns are required and which harmless aliases are accepted?
- How are numeric-looking codes preserved as identifiers?
- How are currency and percentage values normalised?
- Must the file contain exactly one report year?
- What constitutes one fiscal period?
- Must every property-and-GL pair contain twelve unique periods?
- How are duplicate period rows rejected?
- Which organisation owns the import and file?
- How are properties and GL accounts mapped?
- What happens to an unmapped row?
- Are reference lists paged without hidden limits?
- Are writes chunked with durable counters?
- Can operators distinguish reading, parsing, mapping and writing?
- What happens to partial lines after failure?
- Which states permit automatic or manual retry?
- Can a retry duplicate already imported lines?
- Is the raw source row retained for investigation?
- Which purchasing decision consumes the imported budget next?
These contracts make the difference between loading numbers and governing spend.
The broader lesson
A budget file becomes valuable only when the commerce platform can trust its structure, ownership and relationships.
For this marketplace, the workbook enters as a durable background attempt. The parser requires one complete fiscal year for every property-and-account pair. Mapping stays inside the organisation. Unknown codes remain visible as mismatches instead of becoming invented entities. Writes happen in chunks with progress. Failure removes partial lines. Manual replay is allowed only for attempts with no imported data.
Finance keeps a familiar planning process while buyers receive budget context where purchasing decisions occur.
The important capability is not “upload XLSX”. It is turning an external annual plan into a recoverable, explainable and organisation-scoped input to everyday commerce.