Multi-tenant and Isolation
EnerOS’s multi-tenant model corresponds to the boundaries of power operation entities: each tenant is an independent grid operation responsibility entity (e.g., provincial dispatch, regional dispatch, virtual power plant operator), with its data, Agents, equipment, and quotas completely isolated. The tenant boundary is the grid operation responsibility boundary.
The power system is inherently a multi-entity collaboration scenario: national dispatch, sub-network dispatch, provincial dispatch, regional dispatch, and county dispatch form a hierarchy; generation groups, grid companies, electricity sales companies, and users form multi-lateral relationships. Each entity’s data, control permissions, and responsibility scope must be strictly isolated. EnerOS’s multi-tenant model maps these real-world boundaries to the operating system level.
Architecture Overview
┌─────────────────────────────────────────────────────┐
│ Tenant A (Provincial) │ Tenant B (Regional) │ Tenant C (VPP) │
│ - Topology A │ - Topology B │ - Topology C │
│ - Agent A │ - Agent B │ - Agent C │
│ - Time-series A │ - Time-series B │ - Time-series C │
│ - Quota A │ - Quota B │ - Quota C │
└──────────────────┬──────────────────────────────────┘
│ Namespace isolation
┌──────────────────┴──────────────────────────────────┐
│ eneros-tenant (Multi-tenant kernel module) │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Namespace│ │ Quota │ │ ACL │ │
│ │ Namespace│ │ Quota │ │ Access │ │
│ │ │ │ Mgmt │ │ Control │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ KMS │ │ Audit │ │ Cross │ │
│ │ Key Mgmt │ │ Audit │ │ Cross- │ │
│ │ │ │ │ │ tenant │ │
│ └──────────┘ └──────────┘ └──────────┘ │
└──────────────────┬──────────────────────────────────┘
│ cgroup + network namespace
┌──────────────────┴──────────────────────────────────┐
│ Resource Layer: CPU / Memory / Network / Storage │
└─────────────────────────────────────────────────────┘
Tenant Model
use eneros_tenant::{Tenant, TenantId, TenantConfig, TenantTier};
let tenant = Tenant::new(TenantId::from("province_a"))
.name("Province A Power Dispatch Center")
.tier(TenantTier::Enterprise)
.config(TenantConfig {
max_agents: 100,
max_devices: 100_000,
max_ts_throughput: 1_000_000, // samples/s
storage_quota_tb: 10,
api_rate: 1000, // req/s
realtime_cpu_cores: 2,
});
Tenant Field Reference
| Field | Type | Description |
|---|
| id | TenantId | Unique tenant identifier (e.g., “province_a”) |
| name | String | Human-readable name |
| tier | TenantTier | Tier (Community / Pro / Enterprise / National) |
| config | TenantConfig | Resource quota |
| created_at | DateTime | Creation time |
| status | TenantStatus | Active / Suspended / Terminated |
| kms_key_id | String | KMS key ID |
TenantTier Levels
| Tier | Max Agents | Max Devices | Applicable Scenario |
|---|
| Community | 10 | 1,000 | Lab, teaching |
| Pro | 50 | 10,000 | Distribution network, microgrid |
| Enterprise | 100 | 100,000 | Provincial dispatch, regional dispatch |
| National | 1000 | 1,000,000 | National grid |
Namespace Isolation
All kernel resources (topology, equipment, Agents, time-series data) carry a tenant namespace prefix, ensuring physical isolation:
use eneros_tenant::TenantContext;
// Resource namespace is automatically injected
let bus = network.add_bus(Bus::new(1, ...));
// Actual storage key: "tenant:province_a:bus:1"
// Cross-tenant access is rejected
let other = network.get_bus(TenantId::from("province_b"), 1);
// → PermissionDenied
// Namespace covers all resources
// tenant:<id>:topology:<...>
// tenant:<id>:equipment:<...>
// tenant:<id>:timeseries:<tag>
// tenant:<id>:agent:<id>
// tenant:<id>:audit:<...>
Namespace Hierarchy
tenant:province_a:
├── topology:network:default
├── equipment:line:1
├── timeseries:bus_1.voltage
├── agent:dispatch_1
├── twin:instance:main
├── audit:2026-07
└── kms:key:primary
Tenant Context
Each request/Agent execution carries a tenant context, which the kernel automatically validates:
use eneros_tenant::TenantContext;
async fn handler(ctx: &TenantContext, req: Request) -> Response {
// ctx.tenant_id() is automatically extracted from the mTLS certificate
let tenant = ctx.tenant();
// All operations are automatically scoped to the tenant namespace
let network = tenant.network().await?;
let agents = tenant.agents().await?;
let timeseries = tenant.timeseries().await?;
// Topology operations are automatically isolated
let bus = network.get_bus(1).await?;
// Equivalent to network.get_bus_in_namespace(ctx.tenant_id(), 1)
Response::ok()
}
// API middleware automatically injects the tenant context
async fn tenant_middleware(req: Request, next: Next) -> Response {
let cert = req.peer_certificate()?;
let tenant_id = extract_tenant_from_cert(cert)?;
let ctx = TenantContext::new(tenant_id);
next.run(ctx, req).await
}
TenantContext Methods
| Method | Returns | Description |
|---|
| tenant_id() | &TenantId | Get tenant ID |
| tenant() | Tenant | Get tenant object |
| has_permission(perm) | bool | Permission check |
| quota() | Quota | Get quota |
| usage() | Usage | Get current usage |
| kms() -> Kms | Kms | Key management service |
Quota Management
Enforces quotas per tenant dimension to prevent a single tenant from exhausting cluster resources:
use eneros_tenant::Quota;
let quota = Quota::for_tenant(&tenant_id)
.agents(100)
.devices(100_000)
.ts_throughput(1_000_000) // samples/s
.storage(10_tb)
.api_rate(1000) // req/s
.realtime_cpu_cores(2)
.memory_gb(16)
.network_mbps(100);
// The kernel automatically validates the quota
tenant.spawn_agent(my_agent).await?;
// Exceeding the quota → QuotaExceeded
// Real-time usage monitoring
let usage = tenant.usage().await?;
println!("Agents: {}/{}", usage.agents.used, usage.agents.quota);
println!("Storage: {:.1}/{:.1} TB", usage.storage.used_tb, usage.storage.quota_tb);
println!("Time-series throughput: {:.0}/{:.0} samples/s",
usage.ts_throughput.current, usage.ts_throughput.quota);
Quota Resource Matrix
| Resource | Billing Dimension | Default Quota | Over-limit Handling |
|---|
| Agent count | Instance count | 100 | Reject creation |
| Device count | Count | 100k | Reject creation |
| Time-series write | samples/s | 1M | Rate limit |
| Storage | TB | 10 | Reject write |
| API call | req/s | 1000 | 429 rate limit |
| Real-time CPU | Cores | 2 | Reject creation |
| Memory | GB | 16 | OOM protection |
| Network | Mbps | 100 | Rate limit |
| Twin forks | Count | 8 | Reject fork |
| Cross-tenant calls | Calls/day | 10000 | Reject call |
Quota Alerts
use eneros_tenant::QuotaAlert;
// Configure quota alerts
tenant.set_alert(QuotaAlert {
resource: Resource::Storage,
threshold: 0.8, // 80%
action: AlertAction::Notify { emails: vec!["ops@a.com".into()] },
});
tenant.set_alert(QuotaAlert {
resource: Resource::TsThroughput,
threshold: 0.95, // 95%
action: AlertAction::AutoScale { factor: 1.5 },
});
Resource Sharing and Whitelist
Some scenarios require cross-tenant collaboration (e.g., provincial-regional coordination), authorized through an explicit whitelist:
// Province A authorizes Province B to read part of the data
tenant_a.grant(&tenant_b, Permission::read(&[
"bus:1:voltage",
"bus:2:voltage",
"branch:5:flow",
])).duration(Duration::from_hours(24)).await?;
// Province B can access within the whitelist
let v = tenant_b.cross_read(&tenant_a, "bus:1:voltage").await?;
// Revoke authorization
tenant_a.revoke(&tenant_b, Permission::read(&["bus:1:voltage"])).await?;
// Query authorizations
let grants = tenant_a.list_grants().await?;
for g in &grants {
println!("{} → {} : {:?} (remaining {:?})",
g.grantor, g.grantee, g.permission, g.expires_in);
Permission Types
| Type | Meaning | Typical Scenario |
|---|
| read(tags) | Read specified data | Provincial-regional data sharing |
| write(tags) | Write specified data | Collaborative control |
| control(devices) | Control specified devices | Delegated control |
| query(topology) | Query topology | Coordinated planning |
| invoke(tools) | Invoke tools | Shared services |
Isolation Strength
| Dimension | Isolation Method | Strength |
|---|
| Data | Namespace prefix + ACL | Strong |
| Compute | cgroup v2 + CPU/memory limits | Strong |
| Network | per-tenant network namespace | Strong |
| Agent | Independent process + IPC communication | Strong |
| Time-series | per-tenant LSM tree | Strong |
| Keys | per-tenant KMS key | Strong |
| Audit | per-tenant WORM log | Strong |
| Real-time domain | per-tenant CPU isolation | Strong |
cgroup v2 Configuration Example
# /sys/fs/cgroup/eneros/tenant_province_a/
# cpu.max = 200000 100000 (2 cores)
# memory.max = 17179869184 (16 GB)
# io.max = "8:16 rbps=104857600 wbps=104857600" (100 MB/s)
Multi-tenant Operations
use eneros_tenant::Admin;
let admin = Admin::connect(&cluster);
// List all tenants
let tenants = admin.list_tenants().await?;
for t in &tenants {
println!("{}: {:?} - {:?}", t.id, t.tier, t.status);
}
// Create a new tenant
let new_tenant = admin.create_tenant(TenantConfig {
id: TenantId::from("vpp_x"),
name: "X Virtual Power Plant".into(),
tier: TenantTier::Pro,
..Default::default()
}).await?;
// Adjust quota
admin.update_quota(&tenant_id, Quota::new().storage(20_tb)).await?;
// Resource usage statistics
let usage = admin.usage(&tenant_id).await?;
println!("Storage: {:.1}/{:.1} TB", usage.storage.used_tb, usage.storage.quota_tb);
println!("Agents: {}/{}", usage.agents.used, usage.agents.quota);
// Suspend/resume tenant
admin.suspend(&tenant_id).await?; // Pause all Agents, but keep data
admin.resume(&tenant_id).await?; // Resume running
// Terminate tenant
admin.terminate(&tenant_id).await?; // Delete all data, release resources
Tenant Lifecycle
Created → Active ⇄ Suspended → Terminated
↓
Degraded (quota exceeded)
Cross-Tenant Collaboration Example: Provincial-Regional Coordination
use eneros_tenant::CrossTenantChannel;
// Establish a provincial-regional coordination channel
let channel = CrossTenantChannel::new(
&tenant_province, // Provincial dispatch
&tenant_local, // Regional dispatch
CoordinationProtocol::TieLineControl,
).await?;
// Provincial dispatch sends coordination command
channel.send(CoordCmd::AdjustTieLine {
branch: 100,
target_mw: 200.0,
}).await?;
// Regional dispatch receives and executes
while let Ok(cmd) = channel.recv().await {
match cmd {
CoordCmd::AdjustTieLine { branch, target_mw } => {
let decision = DispatchDecision::new()
.set_generator(local_gen, target_mw);
ctx.execute(decision).await?;
}
}
}
| Operation | Latency | Note |
|---|
| Tenant context injection | < 5μs | mTLS certificate extraction |
| Namespace prefix check | < 1μs | - |
| Quota validation | < 10μs | Atomic counter |
| Cross-tenant whitelist validation | < 20μs | Cache hit |
| Tenant resource isolation overhead | < 1% | cgroup |
| Cross-tenant message latency | < 100μs | Same cluster |
| KMS key access | < 1ms | - |
| Max tenants per node | 1000 | - |
| Max tenants in cluster | 100,000 | - |
Test environment: 4 cores / 8GB / Ubuntu 22.04.
Security Compliance
Multi-tenant isolation meets the following compliance requirements:
| Standard | Requirement | EnerOS Implementation |
|---|
| NERC CIP-005 | Electronic security boundary | per-tenant network namespace |
| IEC 62443-4-1 | Secure development lifecycle | Built-in security module |
| GB/T 22239 | MLPS 2.0 Level 3 | Mandatory access control |
| ISO 27001 | Information security management | WORM audit + KMS |
| GDPR | Data subject isolation | per-tenant data encryption |
Configuration Parameters
Multi-tenant-related configuration in eneros.toml:
[tenant]
# Whether to enable multi-tenancy
enabled = true
# Default tenant tier
default_tier = "pro"
# Maximum number of tenants per node
max_tenants_per_node = 1000
# Whether to enable cgroup isolation
cgroup_isolation = true
# Whether to enable network namespace
network_isolation = true
# Default storage quota (TB)
default_storage_quota_tb = 10
# Quota check interval (seconds)
quota_check_interval_secs = 10
# Default cross-tenant call quota (calls/day)
cross_tenant_quota_daily = 10000
# Whether to enable KMS
kms_enabled = true
| Parameter | Type | Default | Description |
|---|
| enabled | bool | true | Enable multi-tenancy |
| default_tier | enum | pro | Default tenant tier |
| max_tenants_per_node | u32 | 1000 | Max tenants per node |
| cgroup_isolation | bool | true | cgroup isolation |
| network_isolation | bool | true | Network namespace isolation |
| default_storage_quota_tb | u32 | 10 | Default storage quota |
| quota_check_interval_secs | u32 | 10 | Quota check interval |
| cross_tenant_quota_daily | u32 | 10000 | Cross-tenant call quota |
| kms_enabled | bool | true | KMS enabled |
Relationship with Other Capabilities
| Related Capability | Interaction |
|---|
| Zero Trust and Security Enhancement | Tenant identity authentication |
| Operations Automation | Tenant operations |
| Multi-Agent Collaboration | In-tenant Agent orchestration |
| Grid Topology First-Class Citizen | Topology is isolated per tenant namespace |
| Time-Series Native Operations | Time-series data is isolated per tenant LSM |
| Safety Guard | Commands are audited per tenant namespace |
| Digital Twin Engine | Twin forks are subject to tenant quota |
| Real-Time Dual Execution Domain | Real-time CPUs are isolated per tenant |