The engineering centered on the mapping contract: retain the commodity identity, propose an L1–L4 path in the existing reference taxonomy, identify the method and expose exceptions. Two approaches address the problem: retrieval-assisted model classification, and exact, fuzzy and semantic analysis. They produce different output schemas and review behavior.
A taxonomy and mapping assignment delivered as files for category review. The customer imports and uses the delivered mapping.
01 / The problem
A valid shape can still express the wrong category
The target hierarchy has four levels. A model can return four well-formed strings while mixing a parent from one branch with a leaf from another, or choosing a category that shares vocabulary but describes a different kind of purchase. Output shape, hierarchy membership and semantic fit are separate questions.
Source identity adds another constraint. Repeated descriptions can carry different codes or accounting context. The crosswalk must retain that grain so a category proposal remains attached to the item it describes. Classification confidence does not resolve identity loss.
02 / Inside the system
Two mapping paths and a file-based handoff
The retrieval path begins with a reference builder that selects L1–L4, removes wholly empty rows and duplicate paths, and serializes the hierarchy as JSON. The builder uploads that reference, creates a managed vector store and polls ingestion. The classifier takes a commodity description and code, searches the reference and requests a structured classification.
The separate analysis path loads the commodity workbook and taxonomy table, constructs normalized matching keys, and tries exact, fuzzy and local semantic matching. It exports workbook views for exact matches, probable matches, unmapped candidates, duplicates, taxonomy gaps and a draft crosswalk.
The final mapping files include commodity IDs, GL information, spend classification, suggested L1–L4, confidence, method and notes. Their schema differs from the classifier's crosswalk. The handoff is a mapping file.
- Tabular preparation
- Python, pandas and openpyxl; commodity spreadsheets, taxonomy tables and CSV/XLSX exports.
- Model classification
- Responses API with gpt-4o, managed file search and a strict categorise_entry function.
- Separate matching analysis
- Exact UNSPSC and normalized descriptions, fuzzywuzzy and SentenceTransformer all-MiniLM-L6-v2.
03 / Inside the system
A constrained classifier, followed by explicit triage
The classifier uses the gpt-4o model through the Responses API, with file search returning at most three results. A strict categorise_entry function requires l1, l2, l3, l4, match_confidence, new_l4 and reasoning, with additional properties disallowed. The prompt favors an existing semantically equivalent L4 and prohibits a fifth level.
Successful rows retain original text and code, a response ID and timestamp. All successful rows go into the main crosswalk; new-category and below-0.80 rows are also copied into specialist review exports. Failed rows go into an error queue. The review views overlap; none is an approved-output state.
The local alternative tries exact UNSPSC code, exact normalized L4 description, fuzzy matching at 85, then all-MiniLM-L6-v2 cosine similarity at 0.85. A model-load failure changes this path to fuzzy-only. Each result carries its method, so a reviewer can see which rows rely on approximate meaning.
A worked example
One noun, two different purchases
Example with invented data.
01 / InputsIdentify the item and its context
Record SYN-001 describes terminal lighting maintenance. SYN-002 describes replacement lighting components. Keep each identifier and description; the common word lighting does not make the two records duplicates.
02 / ReferenceRetrieve candidate category paths
A sample hierarchy offers Facilities → Building services → Electrical services → Lighting maintenance, and Facilities → Building supplies → Electrical components → Lighting components. The paths separate service work from parts.
03 / ProposalPrepare a structured suggestion
The classifier proposes four levels, confidence, a novelty flag and reasoning. A plausible suggestion for SYN-001 is the maintenance path. SYN-002 requires the components path.
04 / ExceptionLeave a bundled description unresolved
If a record says supply and install lighting, the input may bundle goods and services. A reviewer needs the purchasing scope before choosing a category or proposing a new leaf. A high model confidence cannot supply the missing scope.
05 / DecisionLet the category owner decide
A specialist can confirm the existing path, request better source context or consider a taxonomy change. The implementation exports proposals and exception views for the category owner.
04 / Inside the system
Preserve granularity while reusing the hierarchy
L1 expresses a broad family and L4 supplies the specific category. A mapping needs a coherent path between them. Category reuse is valuable when descriptions differ but meaning agrees; a new leaf is useful when the distinction is real and the reference cannot represent it. Adding a category merely because the wording is unfamiliar fragments the shared language.
Consider the example below: lighting maintenance and replacement lighting components share a noun but represent different purchases. The reviewer must decide whether the record describes a service, a part or a bundled activity before choosing a leaf. Accounting and spend classifications add context, but a text similarity score cannot decide their business meaning.
The analysis implementation deduplicates by a description key before matching and merges the representative result back by that key. Different codes can therefore inherit one representative's match. That makes code/description conflicts a necessary evaluation case, rather than treating every repeated description as interchangeable.
05 / Inside the system
Choices that made the proposals inspectable
The implementation choices below expose the reference, output contract and matching method. Their value depends on how the resulting proposals are reconciled and reviewed.
- Decision 01
Put the reference next to the classification request
Retrieve taxonomy context and request a strict structured function result.
The implementation brings candidate category context into the request and gives downstream review stable fields.
Trade-off. Valid fields do not guarantee reference membership, coherent ancestry or the right business meaning.
- Decision 02
Keep a conventional matching path
Try exact code and description matches before fuzzy and local semantic similarity in a separate analysis.
Method labels expose how a candidate was obtained and which cases rely on approximate meaning.
Trade-off. Thresholds require evaluation, and deduplication can collapse distinct coded records.
- Decision 03
Reuse categories and surface novelty
Prefer existing L4 categories and copy new-category and low-confidence rows into review outputs.
This keeps category expansion visible rather than silently treating every unfamiliar phrase as a new branch.
Trade-off. The main crosswalk still includes these rows; a review export is not an approval decision.
06 / Inside the system
Check identity, hierarchy and meaning separately
The classifier reports a confidence with each proposal. Classification accuracy is measured against specialist-labeled expected categories.
A quality audit of the mapping files checks coverage, duplicate identifiers and category consistency. Reconciling its findings comes before the customer imports the mapping.
The classifier sends requests through a synchronous client and sends failed rows to an error queue. The new-category cap raises a warning after the output is saved; it does not block the export.
- How does each item keep its identity?
- The classifier keeps each row's original description and code with its result. The analysis path deduplicates by description before matching, so rows with different codes can share a match; source IDs are reconciled against output IDs before import.
- Do all four levels belong together?
- The function schema constrains the shape of the four levels. Checking that they form one path in the reference hierarchy is a separate validation step.
- Does the label express the purchase?
- Use specialist-labeled cases, including goods versus services and conflicting codes. Model confidence and similarity are signals, not correctness measurements.
- Where is the reviewer's decision recorded?
- Review exports triage the proposals. Acceptance is the category owner's decision and sits outside the exports.
07 / Inside the system
Join technical preparation to category ownership
Embedding Labs organized the work around a bounded taxonomy deliverable. Implementation support prepared the reference, classification methods and export artifacts. The handoff names procurement specialists, data stewards, technical staff and management as the audiences for different review outputs.
Those functions have different questions to resolve: category specialists judge meaning, data stewards reconcile identity and hierarchy, technical staff investigate processing errors, and the customer determines adoption.
The people around the workflow.
- Procurement specialists
- Review category meaning and proposed new leaves; hold category authority for the customer.
- Data stewards
- Review item identity, duplicate handling and reference consistency.
- Technical support
- Prepare methods and outputs, investigate failed processing and explain method differences.
- Customer management
- Lead the import and use of the mapping deliverable.
Technology and integrations
Services in the workflow.
- OpenAI APIAI service integration
Model calls in the procurement classification and mapping workflow.
2025 · Classification workflow
The design principle
Preserve the item, explain the category and distinguish a prepared mapping from an accepted taxonomy decision.