Define the observation grain
One observation is identified by station, timestamp and source system. A deterministic key preserves this distinction rather than silently combining sources.
Data engineering
Case study / 03
A consistent grain across different sources.
Normalize heterogeneous weather feeds into traceable analytical observations with layered SQL and explicit quality rules.
Technologies & concepts
Technical demonstration / selected source
Private repository
01 / The context
Weather feeds use different station identifiers and field structures. Before observations can be compared, the analytical model needs a stable grain, source provenance and clear quality expectations.
02 / Engineering decisions
One observation is identified by station, timestamp and source system. A deterministic key preserves this distinction rather than silently combining sources.
Separate raw, staging, intermediate and mart layers normalize two source contracts and connect observations to station metadata.
Uniqueness, non-null, relationship, allowed-value, range and temporal checks express the conditions needed for analysis. Duplicate observations are detected rather than silently removed.
03 / System view
Illustrative architecture from the project presentation. It describes the selected design; it does not imply a deployed service or a measured operational result.
View the full-size diagramRaw observations
Staging contracts
Unified records
Station dimension
Observation facts
04 / Selected artifacts
dbt/models/staging/dbt/models/intermediate/dbt/models/marts/dbt/macros/generic_tests/dbt/tests/docs/model-design.mddocs/data-quality.md05 / Scope & limitations
Demonstration and source showcase using synthetic station metadata and generated observations. No live ingestion, deployment, execution results, throughput figures or test pass rates are claimed. Airbyte is an ingestion boundary; connections and operational scheduling require separate configuration. The six-station check is fixture-specific, and time-based checks depend on the snapshot.
The GitHub repository is private. Its link works for authorized collaborators; other visitors can contact me to discuss the project.
Next case study / 04
The next conversation