Product catalog fuzzy matching works best when it treats a product as a structured entity rather than a single title string. This guide presents a field-by-field workflow for matching supplier records while protecting meaningful differences in brands, model numbers, sizes, dimensions, units, and pack counts. It also provides a scoring matrix, exception rules, monitoring checkpoints, and a review-queue checklist you can reuse during supplier onboarding and catalog refreshes.
Overview
Supplier feeds rarely describe the same product in exactly the same way. One source may write “Acme UltraClean 500ml 6pk,” another “ULTRACLEAN 0.5 L – pack of 6,” and a third may omit the pack count entirely. Simple exact matching treats these as different records. Unrestricted fuzzy matching can make the opposite mistake by merging products that share a brand and title but differ in size, capacity, color, revision, or quantity.
A reliable product matching system separates two jobs:
- Candidate generation: Find plausible records using normalized text, identifiers, tokens, or approximate string matching.
- Decision-making: Determine whether the candidates represent the same sellable item, a related variant, or different products.
This distinction is important for duplicate product detection, marketplace ingestion, procurement systems, inventory consolidation, and search relevance. A high text similarity score is evidence, not proof. Product matching should give greater weight to fields that define commercial identity than to descriptive language.
Before implementing an algorithm, define the entity you are matching. A catalog may contain a parent product, such as a running shoe model, and child variants for size and color. If inventory, pricing, or fulfillment operates at the child-variant level, those attributes must be protected during entity matching rather than discarded as title noise.
What to track
1. Build a field-specific normalization pipeline
Normalize data for comparison, but retain the original values for display, auditing, and review. A general text pipeline may lowercase text, standardize Unicode, collapse whitespace, and remove harmless punctuation. Product fields need additional, field-specific treatment:
- Brand: Maintain an approved alias table for abbreviations, punctuation, and known supplier spellings. Do not assume that similar brand names are interchangeable.
- SKU and model number: Compare alphanumeric forms with separators removed when appropriate, while preserving meaningful characters. “AB-120” and “AB120” may match; “AB120” and “AB12O” should not be silently merged without supporting evidence.
- Units: Convert compatible units into a canonical representation, such as milliliters, liters, grams, kilograms, inches, or centimeters. Keep the original unit and parsed value so conversion errors can be investigated.
- Dimensions: Parse length, width, height, diameter, and screen size separately. Do not compare a dimension string as undifferentiated text.
- Pack count: Extract values such as “6 pack,” “6pk,” and “case of 6” into a structured quantity. A missing pack count should usually be treated as unknown, not equal to one.
- Variant attributes: Extract color, size, finish, voltage, capacity, gender, compatibility, and revision where they affect the purchasable item.
- Description: Use token similarity or semantic similarity as supporting evidence. Descriptions are useful for candidate generation but may contain marketing text shared by many products.
For multilingual catalogs, apply language-aware normalization and transliteration carefully. Transliteration can improve candidate generation, but it should not replace exact checks on identifiers or variant attributes. The guidance in Multilingual Entity Matching is useful when supplier feeds use different scripts or naming conventions.
2. Use identifiers before general text similarity
Search for exact or near-exact matches on manufacturer part numbers, GTIN-like identifiers, supplier SKUs, and model numbers before comparing titles. An identifier should not be accepted blindly: validate its format, source, and relationship to the brand. Reused, truncated, or incorrectly mapped identifiers can create confident false positives.
For records without reliable identifiers, generate candidates using brand, model tokens, distinctive title tokens, and compatible measurements. Trigram similarity, Levenshtein distance, or Jaro-Winkler can help compare strings, but each behaves differently with short codes, reordered tokens, and insertions. Test thresholds against labeled examples rather than choosing one universal cutoff. See this algorithm comparison and threshold-testing template for a reusable evaluation approach.
3. Score fields by identity risk
A weighted score is more useful than a single title similarity value. The following starting matrix is intentionally conservative and should be calibrated against your catalog:
| Field | Suggested role | Matching guidance |
|---|---|---|
| Validated manufacturer identifier | Blocking or near-hard evidence | Strong evidence when brand and format also agree; investigate conflicts. |
| Brand | High weight | Exact or approved alias match; conflicting brands are usually a rejection. |
| Model or part number | High weight | Compare normalized forms, but protect ambiguous characters and revisions. |
| Size, capacity, and dimensions | High weight | Parse and compare numerically with defined tolerances; mismatches often indicate variants. |
| Pack count | High weight | Require agreement when present in both records; do not infer a missing value. |
| Color or finish | Medium to high | Use a controlled vocabulary and preserve values that affect the sellable SKU. |
| Title and description | Supporting evidence | Use token or semantic similarity after structured conflicts are checked. |
Use decision bands such as automatic match, manual review, and do not match. The boundaries should depend on the cost of an incorrect merge. A catalog system that controls pricing or inventory generally needs stricter rules than an internal search index that only ranks possible results.
4. Maintain an exception table
Explicit exceptions are safer than repeatedly tuning a general-purpose similarity threshold. Track recurring cases in a table like this:
| Observed pattern | Risk | Recommended action |
|---|---|---|
| Same title, different pack count | Inventory and price are misrepresented | Reject automatic merge; require pack agreement. |
| Same model, different voltage | Product may be incompatible | Treat voltage as a hard constraint where relevant. |
| Brand omitted by one supplier | Common titles create false positives | Require model evidence plus compatible specifications. |
| Supplier SKU reused across regions | Identifier appears more reliable than it is | Scope uniqueness by supplier, region, or catalog. |
| “New,” “refurbished,” or “used” in title | Condition is confused with product identity | Store condition separately and define whether it belongs to the match key. |
| Bundle or multipack versus single item | Different commercial units are merged | Parse bundle components and quantity before matching. |
Cadence and checkpoints
Product matching should be monitored as a data-quality process, not treated as a one-time import task. Establish a baseline for each supplier and review it on a monthly or quarterly cadence, depending on feed volume and business risk.
At every supplier onboarding
- Profile null rates, field formats, character sets, units, delimiters, and identifier patterns.
- Sample records with long titles, short codes, missing brands, bundles, and multiple units.
- Label a small gold-standard set as match, non-match, variant, or insufficient evidence. Guidance on building a labeled entity-resolution dataset is available in this gold-standard dataset guide.
- Run the candidate-generation and scoring workflow in shadow mode before enabling automatic merges.
During each catalog refresh
- Record match, non-match, review, and unresolved rates by supplier and category.
- Sample high-confidence matches, especially those based on fuzzy title similarity rather than identifiers.
- Compare newly observed brands, units, attributes, and token patterns with the existing parser and alias tables.
- Check whether unmatched records are concentrated in a single supplier, category, language, or product family.
After schema or parser changes
Re-run a fixed regression set containing known matches, near matches, variant conflicts, and historical false positives. A parser change that improves pack-count extraction may alter scores across an entire category. Keep previous decisions and feature values so changes can be explained and, if necessary, rolled back.
How to interpret changes
A rising automatic-match rate is not necessarily an improvement. It may indicate better normalization, but it may also mean that a new parser is discarding variant information. Interpret matching metrics alongside error samples and business outcomes.
Investigate these patterns:
- More unmatched records: The supplier may have introduced a new format, language, category, or identifier scheme. Expand parsing and candidate generation before lowering thresholds.
- More manual reviews: The model may be correctly exposing ambiguity, or a field extraction problem may be preventing clear decisions. Separate those cases in the queue.
- More automatic matches with more corrections: Treat this as a precision warning. Examine merges involving shared titles, missing brands, pack quantities, and model revisions.
- Scores shift without source changes: Check normalization libraries, unit conversion rules, tokenization, index settings, and deployment differences.
- One category performs poorly: Use category-specific attributes and constraints rather than weakening global rules.
Review queues should show the candidate pair, field-level values, normalized values, individual similarities, triggered rules, and reason for the recommendation. A reviewer should be able to answer “why did this match?” without reconstructing the entire pipeline. For operational safety, preserve decisions and support rollback rather than permanently merging records immediately. The principles in Safe Customer Record Deduplication also apply to product entities.
For large PostgreSQL catalogs, trigram indexes can support candidate retrieval, but indexing does not replace field-aware decision rules. The PostgreSQL fuzzy search guide covers practical considerations for ranking and query tuning.
When to revisit
Revisit this matching workflow before each supplier onboarding, scheduled catalog refresh, schema migration, or audit of incorrect merges. A quarterly review is a reasonable default when feeds are stable; use a shorter cycle when suppliers change formats frequently or when matching errors affect inventory, pricing, compliance, or customer-facing search.
Use this action checklist:
- Pull the latest match, review, rejection, and unresolved counts by supplier and category.
- Sample both high-confidence matches and the lowest-scoring automatic matches.
- Review every correction involving pack count, size, capacity, dimensions, voltage, model revision, or bundle status.
- Update brand aliases, unit parsers, controlled vocabularies, and exception rules only when the observed evidence supports the change.
- Add new positive and negative examples to the regression set, including records that should remain separate variants.
- Re-test thresholds and compare precision-oriented results with review workload before enabling new automatic merges.
- Document the release, affected suppliers, changed fields, and rollback plan.
The most dependable SKU matching algorithm is not the one with the highest similarity score. It is the one that makes identity-defining differences visible, limits automatic decisions to well-supported cases, and improves through a repeatable review cycle.