Metadata examples

Shared data setup

The following examples use pandas objects with metadata. In hetida designer, these are supplied as component inputs. To try the examples in a Python session, create them using the shared data setup below.

The setup creates three independent pandas objects with complete metadata following the current conventions. Use these objects for the API reference examples as well:

  • series: a SERIES with one value per timestamp, measured in metres.

  • dataframe: a SINGLETSFRAME describing one metric with the value columns value (temperature) and state (measurement state).

  • multitsframe: a MULTITSFRAME with pressure and temperature metrics, whose identifiers are stored in the metric column.

md is the imported hdhelpers.metadata module. deepcopy keeps changes to the example metadata independent of the supplied templates. The same setup is run for each documentation test group. To start an example again with unchanged data, rerun the setup.

from copy import deepcopy

import pandas as pd

from hdhelpers import metadata as md
from hdhelpers.metadata.conventions import (
    MULTITSFRAME_METADATA_EXAMPLE,
    SERIES_METADATA_EXAMPLE,
    SINGLETSFRAME_METADATA_EXAMPLE,
)

timestamps = pd.to_datetime(["2026-01-01T00:00:00Z", "2026-01-01T00:15:00Z"])

# SERIES: one metric, with timestamps in the index; values are in metres.
series = pd.Series([1.0, 1.2], index=timestamps, name="level")
series.attrs = deepcopy(SERIES_METADATA_EXAMPLE)

# SINGLETSFRAME: one metric with several value columns; "value" is in degrees Celsius.
dataframe = pd.DataFrame(
    {"timestamp": timestamps, "value": [20.0, 21.0], "state": ["valid", "valid"]}
)
dataframe.attrs = deepcopy(SINGLETSFRAME_METADATA_EXAMPLE)

# MULTITSFRAME: several metrics, identified by the "metric" column.
multitsframe = pd.DataFrame(
    {
        "timestamp": timestamps,
        "metric": ["plant_a.pressure", "plant_a.temperature"],
        "value": [1.0, 20.0],
    }
)
multitsframe.attrs = deepcopy(MULTITSFRAME_METADATA_EXAMPLE)

See First steps for examples within components.

Metadata lookup basics

A metric is one measured signal. A value column (called a value dimension in the schema) holds values or related information, such as measurement state. Metadata is stored in attrs as convention fields or as a model under hd_metadata. Shared metadata in value_dimensions_shared supplies defaults for matching columns across metrics.

The metadata models are immutable and allow additional fields at every level. Metadata timestamps are UTC ISO strings with an explicit Z or +00:00 offset. Adapters exchange convention fields; hdhelpers.metadata.attrs_to_json() converts an attached model back to those fields for export.

Series and SINGLETSFRAME getters select dataset_metadata.single_metric, or the only metric described if no identifier is set. They do not select by Series.name. Without a unique selection, metric-specific information is unavailable. Absent fields and fields set to None count as missing. Inherited properties are supplied by the source system for the metric or its parent assets. They are used as fallbacks when include_inherited=True. Explicit paths such as "inherited.site" access them directly, even when this fallback is disabled. Missing or invalid metadata gives empty mappings for mapping getters; missing-key lookups still use their documented defaults.

Automatic metadata type selection

When parsing raw attrs, read_metadata, attach_metadata and check_metadata_conventions infer the type as follows:

  • A Series selects SERIES.

  • A DataFrame with a metric column selects MULTITSFRAME.

  • Other DataFrames select SINGLETSFRAME.

An explicit data_type overrides this selection. read_metadata also checks the type of an attached model when requested; attach_metadata keeps an already attached model unchanged.

Units, measurements and return shapes

SERIES getters return a single value. SINGLETSFRAME getters return information by value column; MULTITSFRAME getters return it by metric and value column. The measurement describes the physical quantity, whereas the unit describes its scale.

>>> md.get_series_unit(series), md.get_series_measurement(series)
('m', 'level')
>>> dict(md.get_singlets_units(dataframe))
{'value': '°C', 'state': None}
>>> md.get_singlets_measurements(dataframe)["value"]
'temperature'
>>> md.get_units(multitsframe)["plant_a.pressure"]["value"]
'bar'
>>> md.get_measurements(multitsframe)["plant_a.pressure"]["value"]
'pressure'
>>> md.get_units(multitsframe)["unknown metric"]["value"] is None
True
>>> start, end = md.get_queried_interval(series)
>>> start.isoformat(), end.isoformat()
('2026-01-01T00:00:00+00:00', '2026-01-02T00:00:00+00:00')

Names and display-name fallbacks

These getters use the same return shapes. A short display name falls back to the display name, which falls back to the name. The templates omit display names, so the examples show that fallback. See First steps for the lookup order and the optional include_inherited=True fallback to inherited properties.

>>> md.get_series_name(series)
'Water level north'
>>> md.get_series_display_name(series), md.get_series_short_display_name(series)
('Water level north', 'Water level north')
>>> md.get_singlets_names(dataframe)["state"]
'measurement state'
>>> md.get_singlets_display_names(dataframe)["state"]
'measurement state'
>>> md.get_singlets_short_display_names(dataframe)["state"]
'measurement state'
>>> md.get_names(multitsframe)["plant_a.pressure"]["value"]
'Plant A pressure'
>>> md.get_display_names(multitsframe)["plant_a.pressure"]["value"]
'Plant A pressure'
>>> md.get_short_display_names(multitsframe)["plant_a.pressure"]["value"]
'Plant A pressure'

Metric fields and value-dimension fields

get_singlets_info returns information for each value column, whereas get_singlets_metric_info returns one value for the metric. The Series variant looks up its single value dimension; get_metric_info returns values by metric. The field argument may also be a dotted path into nested metadata.

>>> md.get_series_info(series, "decimals")
2
>>> md.get_singlets_info(dataframe, "name")["state"]
'measurement state'
>>> md.get_singlets_metric_info(dataframe, "name")
'ABC temperature'
>>> md.get_metric_info(multitsframe, "name")["plant_a.pressure"]
'Plant A pressure'

Recording information

Single-metric getters return a Recording object; get_recordings returns these objects by metric. Access its fields to distinguish samples, periodic recording and recording on change. Frequency, offset and time zone fall back to the dataset metadata together, as described by hdhelpers.metadata.get_recordings().

>>> recording = md.get_series_recording(series)
>>> recording.recording_mode, recording.data_frequency
('periodic', '15min')
>>> md.get_singlets_recording(dataframe).data_frequency is None
True
>>> md.get_recordings(multitsframe)["plant_a.temperature"].recording_mode
'on_change'

Default time zones

A metric’s time zone falls back to the dataset’s default, otherwise to None. Use an explicit fallback such as or "UTC" when choosing a time zone for a computation. See First steps for a daily aggregation example and the distinction between computation and display time zones.

>>> md.get_series_default_timezone(series)
'Europe/Berlin'
>>> md.get_singlets_default_timezone(dataframe) is None
True
>>> md.get_singlets_default_timezone(dataframe) or "UTC"
'UTC'
>>> md.get_default_timezones(multitsframe)["plant_a.pressure"]
'Europe/Berlin'

Default aggregation methods

The getters return a method name, or method names by metric. An explicit method wins; AVG is normalized to MEAN. Otherwise, value_data_type in metadata selects LAST for boolean/string and MEAN for other types.

>>> md.get_series_default_aggregation_method(series)
'MEAN'
>>> md.get_singlets_default_aggregation_method(dataframe)
'MEAN'
>>> md.get_default_aggregation_methods(multitsframe)["plant_a.temperature"]
'MAX'

For the metadata lifecycle, start with hdhelpers.metadata.check_metadata_conventions(), then hdhelpers.metadata.read_metadata() or hdhelpers.metadata.attach_metadata(). Use hdhelpers.metadata.set_metadata() to attach changed metadata to a result and hdhelpers.metadata.attrs_to_json() before serialization. Each function has an example using the shared data setup.

See Metadata model reference for the model classes and their reading, update and conversion methods.