Ingestion
Synthetic telemetry, CDC envelopes, batch manifests, and local events model coexistence patterns.
Platform Overview
This lab is intentionally concise. It is built to support architecture conversations about hybrid modernization, industrial telemetry ingestion, governed lakehouse design, and scalable data foundations.
Lakehouse
Bronze raw data, Silver interval metrics, and Gold daily summaries represent lakehouse layers.
Governance
Contracts, catalog metadata, security boundaries, quality reports, and lineage notes keep the platform explainable.
Physical Installation
The mini project starts from a 3 kW photovoltaic installation with a Huawei inverter monitored through FusionSolar, then turns local production signals into governed analytical data products.
Data Flow
Connect
Legacy, batch, and telemetry sources.
Legacy, batch, and telemetry sources.
Bronze
Store raw interval observations.
Store raw interval observations.
Silver
Standardize metrics and flags.
Standardize metrics and flags.
Gold
Publish daily summaries.
Publish daily summaries.
Serve
Expose curated API endpoints.
Expose curated API endpoints.
Lakehouse Platform
Technical view of the ingestion and processing path from local telemetry sources to governed Bronze, Silver, and Gold datasets.
API Surface
/api/telemetry/latest exposes curated interval telemetry. /api/analytics/daily-production exposes Gold production summaries. /api/governance/data-products exposes platform metadata examples.
Generated Production Chart
The chart reads the Gold daily summary endpoint at runtime.
Roadmap
Next iterations focus on real telemetry, MinIO-backed lakehouse storage, Databricks workflows, and forecasting-ready feature datasets.
- Real sources: Huawei/FusionSolar exports, weather data, and operational event feeds.
- MinIO lakehouse: Bronze, Silver, and Gold datasets on S3-compatible storage.
- Databricks lab: notebooks and workflows over governed lakehouse datasets.
- Forecasting features: weather-enriched datasets for production forecasting experiments.