Time-Series Native Operations
EnerOS treats time-series data as a kernel-native data type, supporting millions of points per second writes and millisecond-level queries without an external time-series database. This is the direct embodiment of the “Time-Series Native” philosophy: grid voltage, current, and power are inherently time-series data, and the operating system should understand them natively rather than supporting them through external plugins.
The power system is a “time-intensive” domain: PMU data is sampled at 100–120 Hz, fault recording reaches above 10 kHz, and state estimation runs every 100 ms. Traditional architectures rely on external InfluxDB / TimescaleDB, introducing network round-trips and serialization overhead. EnerOS embeds the time-series engine into the kernel, sharing the same memory space as topology and load flow, with a write path as short as 1μs.
Architecture Overview
┌─────────────────────────────────────────────────────┐
│ Agent / Application / Digital Twin │
└──────────────────┬──────────────────────────────────┘
│ write / query / aggregate
┌──────────────────┴──────────────────────────────────┐
│ eneros-timeseries (kernel time-series engine) │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Writer │ │ Reader │ │ Compactor│ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ CQ Engine│ │ Retention│ │ Downsamp │ │
│ └──────────┘ └──────────┘ └──────────┘ │
└──────────────────┬──────────────────────────────────┘
│ Columnar Storage + Gorilla Encoding
┌──────────────────┴──────────────────────────────────┐
│ LSM-Tree (per-tenant) │
│ MemTable → SSTable → Compaction │
└─────────────────────────────────────────────────────┘
Data Model
Time-series data consists of three elements: measurement point (tag), timestamp, and field value.
use eneros_timeseries::{Series, Sample, Timestamp, Quality};
let mut series = Series::new("bus_1.voltage");
series.push(Sample::new(ts1, 1.02));
series.push(Sample::new(ts2, 1.025));
// Sample with quality flag
series.push(Sample::with_quality(ts3, 1.03, Quality::Good));
series.push(Sample::with_quality(ts4, 0.98, Quality::Suspect));
Sample Field Reference
| Field | Type | Meaning | Note |
|---|---|---|---|
| timestamp | Timestamp | Nanosecond-precision timestamp | UTC |
| value | f64 | Measured value | Scalar |
| quality | Quality | Data quality flag | Good / Suspect / Bad |
| flags | u8 | User-defined flags | 0-255 |
Quality Enum
| Variant | Meaning | Value |
|---|---|---|
| Good | Data is valid | 0 |
| Suspect | Data is suspect (e.g., sensor alarm) | 1 |
| Bad | Data is invalid | 2 |
| Missing | Data is missing (filled) | 3 |
| Calculated | Computed (virtual sensor) | 4 |
Writing
Supports single-point, batch, and streaming writes:
use eneros_timeseries::TimeSeriesStore;
let store = TimeSeriesStore::open("/var/eneros/tsdb")?;
// Single-point write
store.write("bus_1.voltage", ts, 1.02).await?;
// Write with quality flag
store.write_with_quality("bus_1.voltage", ts, 1.02, Quality::Good).await?;
// Batch write (recommended, 5x higher throughput)
let batch = vec![
Sample::new(ts1, 1.02),
Sample::new(ts2, 1.025),
Sample::new(ts3, 1.03),
];
store.write_batch("bus_1.voltage", batch).await?;
// Multi-tag batch write
let multi_batch = vec![
("bus_1.voltage", Sample::new(ts, 1.02)),
("bus_1.frequency", Sample::new(ts, 50.01)),
("bus_1.load", Sample::new(ts, 50.0)),
];
store.write_multi(multi_batch).await?;
// Streaming write (for real-time collection)
let mut stream = store.stream("bus_1.voltage");
stream.send(Sample::now(1.02)).await?;
stream.flush().await?;
Write API Overview
| Method | Input | Applicable Scenario | Throughput |
|---|---|---|---|
| write | tag, ts, value | Single-point write | 2M samples/s |
| write_with_quality | tag, ts, value, q | With quality flag | 2M samples/s |
| write_batch | tag, Vec | Batch write | 5M samples/s |
| write_multi | Vec<(tag, Sample)> | Multi-tag batch | 5M samples/s |
| stream | tag | Streaming write | 5M samples/s |
Querying
use eneros_timeseries::{Query, Range, Aggregation, Filter};
let result = store.query(
Query::select("bus_1.voltage")
.range(Range::last(Duration::hours(1)))
.downsample(Duration::seconds(1), Aggregation::Avg)
.filter(Filter::quality(Quality::Good))
).await?;
for sample in result {
println!("{}: {:.4}", sample.timestamp, sample.value);
}
Multi-Tag Query
let multi = store.query(
Query::select_multi(&["bus_1.voltage", "bus_1.frequency", "bus_1.load"])
.range(Range::between(t_start, t_end))
.align(AlignStrategy::Interpolate)
).await?;
for (tag, samples) in multi {
println!("Tag {}: {} samples", tag, samples.len());
}
Range Types
| Variant | Meaning | Example |
|---|---|---|
| Last(d) | The most recent d duration | Last 1 hour |
| Between(t1, t2) | Closed interval | [t1, t2] |
| Since(t) | From t until now | Since last restart |
| Relative(offset) | Relative to now | Offset -1h |
| All | All history | Full replay |
Filter Types
| Variant | Meaning |
|---|---|
| quality(q) | Filter by quality flag |
| value_range(min, max) | Filter by value range |
| anomaly_only | Only anomalies |
| time_mask(mask) | Filter by time mask (e.g., working hours) |
Aggregation and Downsampling
Built-in common aggregation functions, supporting both tumbling and sliding windows:
// Overall aggregation
let avg = store.aggregate(
"bus_1.voltage",
Range::last(Duration::hours(24)),
Aggregation::Avg,
).await?;
// Tumbling window aggregation
let windowed = store.window_aggregate(
"bus_1.load",
Range::last(Duration::hours(24)),
Duration::hours(1),
Aggregation::Percentile(95),
).await?;
for window in &windowed {
println!("{} ~ {}: P95 = {:.2}", window.start, window.end, window.value);
}
// Sliding window aggregation
let sliding = store.sliding_window(
"bus_1.load",
Range::last(Duration::hours(1)),
Duration::minutes(15), // Window size
Duration::minutes(5), // Step
Aggregation::StdDev,
).await?;
| Aggregation Function | Description | Applicable Scenario |
|---|---|---|
| Avg | Arithmetic mean | Voltage, power average |
| Sum | Summation | Energy meter accumulation |
| Min / Max | Extrema | Extreme voltage detection |
| Percentile(p) | Quantile | P95/P99 load |
| Count | Counting | Number of faults |
| First / Last | First/last value | State change |
| StdDev | Standard deviation | Fluctuation analysis |
| Rate | Rate of change | Rate of change of frequency |
| Integral | Integral | Energy (Wh) |
| Median | Median | Noise-resistant statistics |
Continuous Queries
Register automatically executed continuous queries to avoid repeated computation:
use eneros_timeseries::ContinuousQuery;
// SQL-style CQ
store.register_cq(
"voltage_5min_avg",
"SELECT avg(value) INTO bus_1.voltage_5m FROM bus_1.voltage GROUP BY time(5m)",
).await?;
// Typed CQ
let cq = ContinuousQuery::new("load_p95_hourly")
.source("bus_1.load")
.into("bus_1.load_p95_1h")
.aggregate(Aggregation::Percentile(95))
.window(Duration::hours(1))
.build();
store.register_cq_typed(cq).await?;
// List all CQs
let cqs = store.list_cqs().await?;
for cq in &cqs {
println!("{}: {} (last_run: {:?})", cq.name, cq.sql, cq.last_run);
}
// Drop a CQ
store.drop_cq("voltage_5min_avg").await?;
Data Retention Policy
Supports configuring retention period and precision degradation per measurement point:
use eneros_timeseries::RetentionPolicy;
store.set_retention("bus_1.voltage",
RetentionPolicy::new()
.raw_for(Duration::days(7)) // Raw data retained for 7 days
.downsampled(Duration::seconds(1), Duration::days(90)) // 1s granularity retained for 90 days
.downsampled(Duration::minutes(1), Duration::years(5)) // 1min granularity retained for 5 years
.downsampled(Duration::hours(1), Duration::years(20)) // 1h granularity retained for 20 years
).await?;
// Batch configuration
let policy = RetentionPolicy::default()
.raw_for(Duration::days(30));
store.set_retention_multi(&["bus_1.*", "bus_2.*"], policy).await?;
RetentionPolicy Tiers
| Tier | Granularity | Retention | Use |
|---|---|---|---|
| Raw | Raw | 7 days | Fault recording, real-time control |
| High | 1s | 90 days | Short-term analysis |
| Medium | 1min | 5 years | Monthly reports |
| Low | 1h | 20 years | Long-term trends, planning |
Anomaly Detection
Built-in time-series anomaly detection algorithms that trigger event notifications to Agents:
use eneros_timeseries::{AnomalyDetector, AnomalyKind};
let detector = AnomalyDetector::new("bus_1.voltage")
.method(AnomalyKind::Spike { threshold: 3.0 }) // 3σ spike
.method(AnomalyKind::LevelShift { window: 60 }) // 60s mean shift
.method(AnomalyKind::Flatline { window: 30 }) // 30s flat
.on_anomaly(|event| {
log::warn!("Anomaly: {:?} at {}", event.kind, event.timestamp);
});
store.register_detector(detector).await?;
Storage Engine
The underlying layer uses columnar storage + Gorilla encoding + Delta-of-Delta timestamp compression. A single node can hold 10 years of grid historical data.
Compression Algorithms
| Data Type | Algorithm | Compression Ratio |
|---|---|---|
| Timestamp | Delta-of-Delta | 12x |
| Floating point | Gorilla XOR | 8-15x |
| String | Dictionary | 5-10x |
| Integer | Simple8b | 6x |
| Boolean | Bit-packing | 32x |
LSM-Tree Structure
MemTable (memory)
│ flush
▼
SSTable L0 (disk, 10MB)
│ compaction
▼
SSTable L1 (disk, 100MB)
│ compaction
▼
SSTable L2+ (disk, 1GB+)
Performance Metrics
| Operation | Throughput | Latency |
|---|---|---|
| Single-point write | 2M samples/s | < 1μs |
| Batch write (10k batch) | 5M samples/s | < 50μs/batch |
| Streaming write | 5M samples/s | < 1μs |
| Range query (1 day, 1s granularity) | - | < 5ms |
| Range query (1 year, 1min granularity) | - | < 50ms |
| Aggregation query (24h Avg) | - | < 10ms |
| Multi-tag query (100 tags) | - | < 20ms |
| Continuous query execution | - | < 100ms |
| Compression ratio | 8-15x | - |
| Single-node storage limit | 10 years of grid data | - |
Test environment: 4 cores / 8GB / Ubuntu 22.04, NVMe SSD.
Configuration Parameters
Time-series-related configuration in eneros.toml:
[timeseries]
# Storage path
data_dir = "/var/eneros/tsdb"
# MemTable size (MB)
memtable_mb = 64
# SSTable compression algorithm: snappy / lz4 / zstd
compression = "lz4"
# Number of compaction threads
compaction_threads = 2
# Default retention period (days)
default_retention_days = 365
# Whether to enable continuous queries
enable_cq = true
# CQ execution interval (seconds)
cq_interval_secs = 10
# Whether to enable anomaly detection
enable_anomaly = true
# Write cache size (MB)
write_buffer_mb = 128
| Parameter | Type | Default | Description |
|---|---|---|---|
| data_dir | String | /var/eneros/tsdb | Storage path |
| memtable_mb | u32 | 64 | MemTable size |
| compression | enum | lz4 | SSTable compression algorithm |
| compaction_threads | u32 | 2 | Number of compaction threads |
| default_retention_days | u32 | 365 | Default retention period |
| enable_cq | bool | true | Enable continuous queries |
| cq_interval_secs | u32 | 10 | CQ execution interval |
| enable_anomaly | bool | true | Enable anomaly detection |
| write_buffer_mb | u32 | 128 | Write cache size |
Relationship with Other Capabilities
| Related Capability | Interaction |
|---|---|
| Digital Twin Engine | Time-series data drives twin replay |
| IoT Ubiquitous Access | IoT device time-series collection |
| Grid Analysis Advanced | State estimation depends on time-series data |
| Multi-Agent Collaboration | Agent memory is based on time-series data |
| Multi-tenant and Isolation | Time-series data is isolated per tenant LSM |
| Safety Guard | Command audit logs are ingested |
| Real-Time Dual Execution Domain | PMU data is written in streaming fashion |
Related Documentation
- Digital Twin Engine — Time-series data drives twin replay
- IoT Ubiquitous Access — IoT device time-series collection
- Grid Analysis Advanced — State estimation depends on time-series data
- Safety Guard — WORM audit logs
- EnerOS Introduction — Time-Series Native design philosophy