Skip to main content

Normalize source values

Sources often spell the same value differently: NetBox reports a device status as active, an Infrahub schema may call it in-service. A vocabulary gives a field one list of canonical values, and each contribution maps its own spellings to it. Census normalizes every value before it resolves the field, so filters and sorting apply to canonical values.

[inventories.devices.fields.status]
type = "string"
vocabulary = ["active", "planned", "offline"]
default = "offline"                       # optional

[[inventories.devices.contributions]]
id = "infrahub"
# ...
[inventories.devices.contributions.attributes]
status = "status.value"

[inventories.devices.contributions.values.status]
"in-service" = "active"
"provisioning" = "planned"

[[inventories.devices.contributions]]
id = "netbox"
# ...
[inventories.devices.contributions.defaults]
status = "planned"                        # overrides the field default for NetBox

How Census normalizes one value​

For a field with a vocabulary, Census checks each supplied value in this order and records the outcome:

  1. The value is one of the canonical values: as_reported.
  2. The value matches a canonical value apart from letter case and surrounding whitespace, for example Active: mapped to active.
  3. The contribution's values table has the value as a key, compared apart from letter case and surrounding whitespace: mapped to the target.
  4. The contribution has a default for the field in defaults, or the field has a default: defaulted to that value. The contribution's default comes first.
  5. Otherwise: unmapped. The value is unknown and the field is null for this contribution.

Then Census checks the type. A result whose JSON type is not the field's type is wrong_type, and the value is unknown.

A field without a vocabulary keeps the source value as it is (as_reported), and only the type check applies.

Defaults apply only to supplied values​

A default replaces a value that the source supplied and that has no mapping. It never fills a field that the source did not supply: a device with no status in NetBox has no status from NetBox, and Census considers the next contribution.

Rules Census checks at startup​

  • Every vocabulary value has the field's type.
  • default is one of the vocabulary values, and requires a vocabulary.
  • values and defaults name declared fields that have a vocabulary.
  • Every target in values and every value in defaults is in the vocabulary.

Find values that have no mapping​

Each unmapped and wrong_type value appears in the diagnostics with its contribution, HFID, field, original value and source path:

curl -s 'localhost:8080/v1/inventories/devices/diagnostics?kind=unmapped'

Full detail keeps the original value next to the outcome, so a consumer can also read what the source reported. Unknown values do not make the inventory incomplete.

To fix an unmapped value, add it to the contribution's values table and restart Census. Census recomputes the inventory from the stored observations, without fetching from the sources again.