Apply a configuration change
Census applies configuration changes when it restarts. At startup it compares each inventory with the definition it stored at the previous start and does the least work that keeps the data correct under the new definition. Consumers never receive values computed under the old definition as if they were valid under the new one.
What each change does
| Change | Effect at restart | What consumers read until it is done |
|---|---|---|
A contribution's connection, the connection's type or endpoint, object_types, hfid, an attributes path or a location path | Census deletes that contribution's records and refreshes it at once. | The contribution reports snapshot: none and the inventory is incomplete with the cause no_snapshot, until the refresh publishes. |
A field (added, removed, or its type, vocabulary, default or priority), the inventory's priority, freshness, the location levels, locations or priority, or a contribution's values, defaults or location_values | Census normalizes every stored observation again from its original value and recomputes every entity, without fetching from the sources. | The inventory's entity, lookup, location and diagnostics routes answer 503 inventory-rebuilding. The inventory list shows the status rebuilding. |
| A contribution removed | Census deletes its records and recomputes the inventory's entities. | Values from the removed contribution disappear. |
| An inventory removed | Census deletes all its data. | The inventory returns 404. |
| An inventory added | Census refreshes its contributions. | The inventory is incomplete until each contribution has published once. |
description, entity_kind, refresh_interval, credentials, webhook_secret, external_base_url, external_base_url_fallback, [server], [memgraph], [branching] | No data change. | The new values apply. Links to source objects use the new address for every record. |
A change can belong to more than one row. Adding a field and mapping it in a contribution
changes both the field list and the contribution's attributes paths: Census rebuilds the
inventory and also deletes and refreshes that contribution's records, because it never
fetched the new path.
Other inventories are not affected by a change to one inventory. Each inventory is rebuilt on its own.
Rebuild time
A rebuild reads only the cache. At 10,000 entities from two contributions, recomputing after a
priority change took 2.57 seconds in the
recorded scale test.
If a rebuild fails, the inventory stays rebuilding until the next start, and the error is
in the log.
Invalid configuration
Census checks the whole file before it connects to anything. A file that breaks a rule stops startup with one line naming the file, the setting and the rule, and Census exits with status 2. The previous data in the cache is not changed. The configuration reference lists the rules.
A file that still uses a section of the format before named inventories ([schema],
[budgets], [normalization], [matching] or [[sources]]) is refused, with the name of
the section that replaces it.
Upgrade Census
The cache carries a layout version. When a new Census build uses a different layout, it empties the cache at startup and refills it from the sources. Until each contribution has published again, inventories are incomplete.