Testing Standards

EnerOS has 7300+ test cases covering all core functionality. All PRs must include corresponding tests and pass CI, otherwise they will not be merged. This page details test layering, writing standards, coverage requirements, CI process, and performance benchmarking.

Test Layering

EnerOS adopts a “test pyramid” layering strategy, converging from bottom-layer unit tests to top-layer end-to-end tests:

        ┌──────────┐
        │ E2E Tests │  ← Real deployment, covers key user scenarios
        └────┬─────┘
       ┌─────┴─────┐
       │ Integration │  ← Cross-crate collaboration
       └─────┬─────┘
      ┌──────┴──────┐
      │ Property Tests │  ← Invariant verification
      └──────┬──────┘
     ┌───────┴───────┐
     │  Unit Tests   │  ← Single function/module
     └───────────────┘

Unit Tests

Unit tests are located in #[cfg(test)] mod tests within src/, named starting with test_:

// crates/eneros-powerflow/src/solver.rs
pub fn solve_newton_raphson(
    topology: &Topology,
    max_iterations: usize,
    tolerance: f64,
) -> Result<PowerflowResult> {
    // Implementation
    Ok(PowerflowResult { converged: true, /* ... */ })
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_solve_newton_raphson_converges_for_simple_case() {
        let topo = Topology::single_bus();
        let result = solve_newton_raphson(&topo, 20, 1e-8).unwrap();
        assert!(result.converged);
    }

    #[test]
    fn test_solve_newton_raphson_returns_error_on_non_convergence() {
        let topo = Topology::invalid_case();
        let result = solve_newton_raphson(&topo, 1, 1e-8);
        assert!(result.is_err());
        match result {
            Err(PowerflowError::NonConvergence { iterations, max_iterations }) => {
                assert_eq!(iterations, 1);
                assert_eq!(max_iterations, 1);
            }
            _ => panic!("Should return NonConvergence error"),
        }
    }
}

Integration Tests

Integration tests are located in the tests/ directory, with filenames ending in _test.rs:

// crates/eneros-powerflow/tests/integration_test.rs
use eneros_powerflow::{NewtonRaphsonSolver, PowerflowResult};
use eneros_topology::Topology;

#[test]
fn test_powerflow_with_ieee_14bus_topology() {
    let topo = Topology::from_ieee_14bus();
    let solver = NewtonRaphsonSolver::new(50, 1e-8);
    let result: PowerflowResult = solver.solve(&topo).unwrap();

    assert!(result.converged, "IEEE 14-bus should converge");
    assert_eq!(result.buses.len(), 14);

    // Compare with IEEE public reference values, tolerance 1e-6
    let bus_1_voltage = result.buses[0].voltage_magnitude;
    assert!((bus_1_voltage - 1.06).abs() < 1e-6, "Bus 1 voltage should be 1.06 p.u., actual {}", bus_1_voltage);
}

#[test]
fn test_powerflow_handles_isolated_bus() {
    let topo = Topology::with_isolated_bus();
    let solver = NewtonRaphsonSolver::new(50, 1e-8);
    let result = solver.solve(&topo);
    assert!(result.is_ok());
}

Property Tests

Property tests use proptest to automatically generate large amounts of random input and verify invariants:

// crates/eneros-topology/src/graph.rs
#[cfg(test)]
mod proptests {
    use proptest::prelude::*;
    use super::*;

    proptest! {
        #[test]
        fn test_adding_bus_does_not_break_connectivity(
            n in 1usize..100,
            edges in proptest::collection::vec((1usize..100, 1usize..100), 0..200)
        ) {
            let mut graph = Graph::new();
            for i in 0..n {
                graph.add_bus(i as u64);
            }
            for (a, b) in edges {
                if a < n && b < n && a != b {
                    graph.add_branch(a as u64, b as u64);
                }
            }
            // Invariant: node count equals the number added
            prop_assert_eq!(graph.bus_count(), n);
            // Invariant: every edge is bidirectional
            for (a, b) in &edges {
                if *a < n && *b < n && *a != *b {
                    prop_assert!(graph.has_branch(*a as u64, *b as u64));
                    prop_assert!(graph.has_branch(*b as u64, *a as u64));
                }
            }
        }
    }
}

Performance Benchmarks

Performance benchmarks use criterion, located in the benches/ directory:

// benches/benches/powerflow_bench.rs
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use eneros_powerflow::NewtonRaphsonSolver;
use eneros_topology::Topology;

fn bench_ieee_14bus(c: &mut Criterion) {
    let topo = Topology::from_ieee_14bus();
    let solver = NewtonRaphsonSolver::new(50, 1e-8);

    c.bench_function("powerflow/ieee_14bus", |b| {
        b.iter(|| {
            let result = solver.solve(black_box(&topo)).unwrap();
            black_box(result);
        })
    });
}

fn bench_ieee_118bus(c: &mut Criterion) {
    let topo = Topology::from_ieee_118bus();
    let solver = NewtonRaphsonSolver::new(50, 1e-8);

    c.bench_function("powerflow/ieee_118bus", |b| {
        b.iter(|| {
            let result = solver.solve(black_box(&topo)).unwrap();
            black_box(result);
        })
    });
}

criterion_group!(benches, bench_ieee_14bus, bench_ieee_118bus);
criterion_main!(benches);

End-to-End Tests

E2E tests are located in tests/e2e/, using Docker Compose to start the complete environment:

// tests/e2e/tests/e2e_tests.rs
use std::process::Command;

#[test]
fn test_e2e_full_startup_and_dispatch() {
    // Start the complete environment (assumed already started via docker-compose up)
    let output = Command::new("enerosctl")
        .args(&["--server", "localhost:8080", "dispatch", "solve"])
        .output()
        .expect("enerosctl execution failed");

    assert!(output.status.success(), "enerosctl dispatch solve should succeed");
    let stdout = String::from_utf8_lossy(&output.stdout);
    assert!(stdout.contains("converged"), "Output should contain converged");
}

Running Tests

Basic Commands

# All tests (default)
cargo test

# Recommend using nextest (faster, friendlier output)
cargo nextest run

# Single crate
cargo test -p eneros-powerflow

# Single crate (nextest)
cargo nextest run -p eneros-powerflow

# Show println! output
cargo test -- --nocapture

# Run only specific tests
cargo test powerflow::newton_raphson

# Run only specific tests (nextest)
cargo nextest run -E 'test(powerflow::newton_raphson)'

# Performance benchmarks
cargo bench -p eneros-powerflow

# Performance benchmarks (specific case)
cargo bench -p eneros-powerflow -- ieee_14bus

Advanced Usage

# Run only unit tests
cargo nextest run --lib

# Run only integration tests
cargo nextest run --test '*'

# Skip tests requiring network
cargo nextest run -E 'not test(network)'

# Parallelism control
cargo nextest run --test-threads=4

# Stop immediately on failure
cargo nextest run --fail-fast

# Repeat runs (verify flaky tests)
cargo nextest run --retries 3

# Run only tests affected by modified code
cargo nextest run -p eneros-powerflow --no-fail-fast

# Show execution time
cargo test -- --report-time

# Output JUnit XML (CI integration)
cargo nextest run --junit-path target/test-results.xml

Test Filtering

cargo-nextest supports powerful filter expressions:

# Match by name
cargo nextest run -E 'test(ieee_14bus)'

# Filter by package
cargo nextest run -E 'package(eneros-powerflow)'

# Combined conditions
cargo nextest run -E 'package(eneros-powerflow) & test(solve)'

# Exclude certain tests
cargo nextest run -E 'not test(network) & not test(slow)'

# Filter by compile features
cargo nextest run --features worm -E 'test(audit)'

Configuration File

Create .config/nextest.toml in the repository root:

[default-profile]
# Default retry count
retries = 0
# Test timeout (seconds)
slow-timeout = { period = "60s", terminate-after = 1 }
# Stop immediately on failure
fail-fast = false

[ci-profile]
# Retry 2 times in CI environment to avoid flaky
retries = 2
# Parallelism
test-threads = 8
# JUnit output
junit.path = "target/test-results.xml"

# Use CI configuration file
cargo nextest run --profile ci

Writing Test Standards

Naming Conventions

Test Structure

Follow the Arrange-Act-Assert (AAA) pattern:

#[test]
fn test_solve_converges_for_simple_two_bus_system() {
    // Arrange: prepare test data
    let mut topo = Topology::new();
    topo.add_bus(1, BusType::Slack, 1.06, 0.0);
    topo.add_bus(2, BusType::PQ, 1.0, 0.0);
    topo.add_branch(1, 2, 0.01, 0.05);
    topo.add_load(2, 0.5, 0.2);
    let solver = NewtonRaphsonSolver::new(20, 1e-8);

    // Act: execute the function under test
    let result = solver.solve(&topo).expect("Load flow should succeed");

    // Assert: verify results
    assert!(result.converged, "Two-bus system should converge");
    assert_eq!(result.buses.len(), 2);
    assert!((result.buses[1].voltage_magnitude - 0.95).abs() < 0.1,
        "Bus 2 voltage should be within a reasonable range");
}

Test Writing Key Points

• New features must include at least one failure case and one success case
• Physics-related calculations should compare with IEEE standard reference values, tolerance 1e-6
• Async tests use #[tokio::test]
• Avoid depending on external networks and time; use injected clocks and in-memory channels
• Tests should not depend on each other; each test can run independently
• Test execution time should be within 100ms; tests exceeding 1s must be marked with #[ignore]

#[tokio::test]
async fn test_ed_respects_unit_limits() {
    // Arrange
    let agent = EconomicDispatchAgent::new(Duration::from_secs(60));
    agent.add_unit(Unit::new(1, 0.0, 50.0, 10.0));  // Pmin=0, Pmax=50
    agent.add_unit(Unit::new(2, 0.0, 50.0, 12.0));
    agent.set_total_demand(80.0);

    // Act
    let result = agent.solve_ed(100.0).await.unwrap();

    // Assert
    assert_eq!(result.len(), 2);
    for sp in &result {
        assert!(sp.power >= 0.0 && sp.power <= 50.0,
            "Unit output {} exceeds [0, 50] range", sp.power);
    }
    let total: f64 = result.iter().map(|sp| sp.power).sum();
    assert!((total - 80.0).abs() < 1e-6, "Total output should equal load demand");
}

#[tokio::test]
async fn test_ed_returns_error_when_demand_exceeds_capacity() {
    let agent = EconomicDispatchAgent::new(Duration::from_secs(60));
    agent.add_unit(Unit::new(1, 0.0, 50.0, 10.0));
    agent.set_total_demand(100.0);  // Exceeds total capacity 50

    let result = agent.solve_ed(100.0).await;
    assert!(matches!(result, Err(EconomicDispatchError::InsufficientCapacity)));
}

#[test]
#[ignore = "Requires external database, run with --ignored"]
fn test_timeseries_persistence_with_real_database() {
    // Only run in CI or manually locally
}

Using Test Utilities

The eneros-test-utils crate provides common test utilities:

use eneros_test_utils::{assert_close, TestTopology, MockClock};

#[test]
fn test_powerflow_with_mock_clock() {
    let clock = MockClock::fixed("2024-01-01T00:00:00Z");
    let topo = TestTopology::ieee_14bus();
    let solver = NewtonRaphsonSolver::new(50, 1e-8);

    let result = solver.solve_with_clock(&topo, &clock).unwrap();

    assert_close(result.buses[0].voltage_magnitude, 1.06, 1e-6);
}

Test Organization

Coverage

Coverage Requirements

CI uses cargo-llvm-cov to generate coverage reports:

Generate Coverage

# Install
cargo install cargo-llvm-cov

# Generate summary
cargo llvm-cov --summary-only

# Generate HTML report
cargo llvm-cov --html

# Generate LCOV format (CI integration)
cargo llvm-cov --lcov --output-path target/lcov.info

# Specific crate only
cargo llvm-cov -p eneros-powerflow --html

# Show only uncovered lines
cargo llvm-cov --show-instantiations --summary-only | grep "MISS"

Coverage Configuration

Add to .cargo/config.toml in the repository root:

[llvm-cov]
# Interface output
output-dir = "target/llvm-cov"
# Exclude standard library and external dependencies
ignore-filename-regex = "registry/|/.cargo/|target/debug/"

Coverage in CI

The CI pipeline .github/workflows/coverage.yml reports coverage on every push to main:

name: Coverage
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
        with:
          components: llvm-tools-preview
      - run: cargo install cargo-llvm-cov
      - run: cargo llvm-cov --lcov --output-path lcov.info
      - uses: codecov/codecov-action@v4
        with:
          files: ./lcov.info
          fail_ci_if_error: false

Performance Benchmarking

Benchmark Suite

EnerOS maintains a complete performance benchmark suite, located in benches/benches/:

Running Benchmarks

# Run all benchmarks
cargo bench

# Run benchmarks for a specific crate
cargo bench -p eneros-powerflow

# Run a specific case
cargo bench -p eneros-powerflow -- ieee_14bus

# Save baseline (for comparison)
cargo bench -p eneros-powerflow -- --save-baseline before-change

# Compare after modifying code
cargo bench -p eneros-powerflow -- --baseline before-change

# Generate HTML report (requires cargo-criterion)
cargo install cargo-criterion
cargo criterion --message-format=json > target/criterion.json

Performance Regression Detection

The CI pipeline .github/workflows/benchmark.yml runs benchmarks on every push to main and compares with the baseline:

name: Benchmark
on:
  push:
    branches: [main]

jobs:
  benchmark:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
      - run: cargo bench --workspace | tee target/bench-output.txt
      - uses: benchmark-action/github-action-benchmark@v1
        with:
          tool: 'cargo'
          output-file-path: target/bench-output.txt
          github-token: ${{ secrets.GITHUB_TOKEN }}
          auto-push: true
          alert-threshold: '120%'
          comment-on-alert: true
          fail-on-alert: true

Performance regression thresholds:

Complete Benchmark Example

// benches/benches/powerflow_bench.rs
use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
use eneros_powerflow::{NewtonRaphsonSolver, FastDecoupledSolver};
use eneros_topology::Topology;

fn bench_powerflow_solvers(c: &mut Criterion) {
    let cases = [
        ("ieee_14bus", Topology::from_ieee_14bus()),
        ("ieee_30bus", Topology::from_ieee_30bus()),
        ("ieee_118bus", Topology::from_ieee_118bus()),
    ];

    let mut group = c.benchmark_group("powerflow");
    group.sample_size(50);
    group.warm_up_time(std::time::Duration::from_secs(2));
    group.measurement_time(std::time::Duration::from_secs(10));

    for (name, topo) in &cases {
        group.throughput(Throughput::Elements(topo.bus_count() as u64));

        let nr = NewtonRaphsonSolver::new(50, 1e-8);
        group.bench_with_input(BenchmarkId::new("newton_raphson", name), topo, |b, t| {
            b.iter(|| nr.solve(black_box(t)).unwrap())
        });

        let fd = FastDecoupledSolver::new(50, 1e-8);
        group.bench_with_input(BenchmarkId::new("fast_decoupled", name), topo, |b, t| {
            b.iter(|| fd.solve(black_box(t)).unwrap())
        });
    }
    group.finish();
}

criterion_group!(benches, bench_powerflow_solvers);
criterion_main!(benches);

CI Process

Pipeline Overview

CI configurations are located in .github/workflows/, containing the following pipelines:

CI Main Flow Details

.github/workflows/ci.yml:

name: CI
on:
  push:
    branches: [main, 'release/*']
  pull_request:
    branches: [main]

env:
  CARGO_TERM_COLOR: always
  RUST_BACKTRACE: 1

jobs:
  fmt:
    name: Rustfmt
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
        with:
          components: rustfmt
      - run: cargo fmt --all -- --check

  clippy:
    name: Clippy
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
        with:
          components: clippy
      - uses: Swatinem/rust-cache@v2
      - run: cargo clippy --all-targets -- -D warnings

  test:
    name: Test
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        rust: [stable, beta]
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@master
        with:
          toolchain: ${{ matrix.rust }}
      - uses: Swatinem/rust-cache@v2
      - run: cargo install cargo-nextest
      - run: cargo nextest run --workspace
      - run: cargo test --doc --workspace

  deny:
    name: Cargo Deny
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: EmbarkStudios/cargo-deny-action@v1
        with:
          arguments: --workspace

Simulating CI Locally

# Simulate full CI process
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo nextest run --workspace
cargo test --doc --workspace
cargo deny check --workspace
cargo doc --no-deps --all-features

Provided script scripts/ci-local.sh (Linux/macOS):

#!/bin/bash
set -e

echo "==> Rustfmt check..."
cargo fmt --all -- --check

echo "==> Clippy check..."
cargo clippy --all-targets -- -D warnings

echo "==> Running tests..."
cargo nextest run --workspace

echo "==> Running doc tests..."
cargo test --doc --workspace

echo "==> Cargo Deny check..."
cargo deny check --workspace

echo "==> Building documentation..."
cargo doc --no-deps --all-features

echo "✅ All checks passed"

Test Checklist

Please check each item before submitting a PR:

• New feature has at least one success case and one failure case
• Test naming follows test_<function>_<scenario> convention
• Tests follow Arrange-Act-Assert structure
• Physics calculations compare with IEEE reference values, tolerance 1e-6
• Async tests use #[tokio::test]
• Tests do not depend on external network and time
• Single test execution time < 100ms, timeout tests marked with #[ignore]
• cargo nextest run --workspace all pass
• cargo test --doc --workspace all pass
• New code coverage ≥ 80%
• Performance-sensitive code has benchmarks added

Related Documentation

• Development Environment Setup
• Pull Request Process
• Installation and Build - Running Tests
• Project Structure