Metadata Normalization: Naming, Formats, Locales, and Controlled Values
How to put catalog strings, dates, locale tags, and controlled values into one agreed form before they are checked or classified.
- 01 Fix character form
- 02 Fix dates and numbers
- 03 Fix locale tags
- 04 Map controlled values
Normalization is an agreed form, not a cleanup pass
Two values can refer to the same thing and still fail a comparison because the characters, the date order, or the wording differ. Normalization is the step that puts those values into the form your rules expect, before a quality check or a taxonomy decision treats the difference as meaningful.
It is not a license to rewrite meaning. Changing “navy” to “blue” is a classification decision. Trimming spaces and choosing one unicode form for the same accented character is normalization. If you cannot say which of those you are doing, stop and split the work.
Do this before the quality checklist and before you create a new concept. A checklist that runs on unnormalized strings will fail the same title twice and pass it once, depending on who typed it. A taxonomy that creates a concept per spelling will grow without a change in meaning.
This diagram scrolls sideways on a narrow screen.
Put text into one character form
Unicode Standard Annex 15, revision 58, dated 12 August 2026, defines normalization forms so that equivalent strings can share one binary representation. Normalization Form C is canonical decomposition followed by canonical composition. The annex states that text made only of ASCII characters is left unchanged by every normalization form, and that the W3C Character Model recommends Normalization Form C for content because canonically equivalent sequences should not stay distinct in web formats.
For catalog strings, the practical rule this publication uses is: store human-readable text in Normalization Form C unless a specific field’s specification says otherwise. Do that before you compare titles, build identifiers from text, or decide that two labels are different concepts.
Compatibility forms, NFKC and NFKD, are a different decision. The annex describes compatibility equivalence as weaker: characters can represent the same abstract character and still differ in appearance or behavior. Folding those differences can be wrong for a catalog that needs the visual distinction. Do not apply NFKC to every field because it “cleans harder.” Name the fields where compatibility folding is intended, such as a search key, and leave the stored title in NFC.
Case folding and whitespace trimming are separate rules. Unicode normalization does not decide whether “Title” and “title” are the same value. Write that rule yourself, and do not apply it to identifiers that are case-sensitive by definition.
Store dates and numbers in a locale-neutral form
W3C Best Practice 13, in the Data on the Web Best Practices recommendation of 31 January 2017, says to use locale-neutral data structures and values, or, where that is not possible, to provide metadata about the locale used. The recommendation points out that a numeric date can be read in more than one order depending on who wrote it. That warning fits catalog files directly. A date that arrives as numbers separated by slashes is not yet a date. It is a string with at least two interpretations.
RFC 3339, July 2002, is a standards-track profile of ISO 8601 for timestamps on the internet. It defines a full-date as a four-digit year, a hyphen, a two-digit month, a hyphen, and a two-digit day. This page recommends that interchange form for catalog dates that are calendar days. It does not claim to reproduce the whole of ISO 8601, and RFC 3339 itself says it is a subset for internet protocols, not the entire ISO grammar.
Times, if you store them, need an offset or an agreed UTC rule. A clock time with no offset is a local time only if you have also stored the place. If you do not have the place, do not invent an offset. Store the date alone, or reject the value.
Numbers need a similar decision: decimal separator, whether a thousands separator is allowed, and the unit. Keep the unit in a separate field or a controlled code. A string that ends in a currency sign is not a normalized price. Best Practice 13’s point is that the stored value should not depend on the reader’s country. Display can localize. Storage should not.
Write language as a tag, not as a country name
RFC 5646, September 2009, is BCP 47. It defines a language tag as a sequence of subtags separated by hyphens. The document’s own discussion of en-US and en-CA says they mean, roughly, English with features generally thought characteristic of the United States and Canada, and that they do not imply a sharp dialect boundary between arbitrary places in those countries.
Use that caution in the catalog. A language tag distinguishes content for indexing and display. It is not a full description of an audience, and a region subtag is not a second language. Do not create a separate record for each country when the only difference is a tag you have not actually translated.
DCMI Metadata Terms lists RFC 5646 among its syntax encoding schemes, alongside older language-tag schemes. You do not need to use Dublin Core properties to adopt the tag syntax. You do need one syntax. Accepting both a language name in English and a BCP 47 tag in the same field guarantees duplicates.
Store the tag in a consistent case pattern. RFC 5646’s canonical forms exist; if you have not implemented them, at least document the pattern you accept, such as a lowercase language subtag and an uppercase region subtag, and reject the other spellings at intake rather than storing all of them.
Map controlled values after the string is stable
A controlled value is a code or a concept identifier, not the label a person happened to type. Mapping is the step that takes a source string, already normalized, and either finds the concept or refuses the value. Controlled values and synonyms explains why the concept should outlive the label. Normalization’s job ends when the string is in the form the map expects.
Do not map before character normalization. The same preferred label can miss the map if one file used a decomposed accent. Do not create a new concept inside the mapping step because a string missed. A miss is a review item. Creating the concept immediately is how supplier spellings become the taxonomy.
If several source strings are allowed to map to one concept, list them as alternate labels or as an import map with an owner. An unlisted synonym that “usually” maps is an unwritten rule. It will be applied differently by the next person.
Records still at intake should be normalized before readiness, not after publication. The onboarding checklist can call this page its formatting step. It should not contain a second, shorter list of date formats.
This table scrolls sideways on a narrow screen.
| Kind of value | Store as | Do not store as |
|---|---|---|
| Readable text | Unicode NFC, plus your own trim and case rules | Whatever the file used, or NFKC on every field |
| Calendar day | RFC 3339 full-date: year, month, day, hyphens | A slash-separated numeric date |
| Clock time | A timestamp with an offset, or reject it | A clock time with a guessed offset |
| Language | One BCP 47 tag pattern | A country name in the same field as a tag |
| Controlled value | A concept identifier after an owned map | The raw supplier wording as if it were the code |
What the standards do not decide
Unicode does not choose your title case. RFC 3339 does not choose your fiscal calendar. BCP 47 does not choose which languages you publish. DCMI does not choose your field list. The W3C data best practices do not design a product catalog. This page cites them for the narrow facts above, and the order of operations is an editorial method.
Display can still be localized. A reader may see a date in a regional order. That presentation is not the stored value, and it should not be written back into the catalog as if it were.
When a difference is real
When are two titles the same string?
After both are in the agreed character form and your trim rules, and only if your case rule also says so. Unicode NFC alone does not fold case.
When needed: Before a duplicate check or a new concept is proposed.
When is a region tag worth storing?
When you truly have a regional variant of the content. RFC 5646 treats en-US and en-CA as English with regional features, not as proof of a hard dialect border. Do not mint tags for countries you have not translated.
When needed: While deciding whether a second record is a language variant or a copy.
When should a supplier date be rejected?
When it is numeric and the order is not explicit, and you cannot tell the month from the day. Ask for the year-month-day form rather than guessing from the partner's country.
When needed: At intake, before the value is stored.
Sources
- Unicode Standard Annex 15, Unicode Normalization Forms, revision 58 — Unicode Consortium
- RFC 3339, Date and Time on the Internet: Timestamps — IETF
- RFC 5646, Tags for Identifying Languages — IETF
- Data on the Web Best Practices — W3C
- DCMI Metadata Terms — Dublin Core Metadata Initiative