Toward Explainable NILM: Real-Time Event-Based NILM Framework for High-Frequency Data

Basic Information

• Paper ID: 2501.16841
• Title: Toward Explainable NILM: Real-Time Event-Based NILM Framework for High-Frequency Data
• Authors: Grigorii Gerasimov, Ilia Kamyshev, Sahar Moghimian Hoosh, Elena Gryazina, Henni Ouerdane
• Institutions: Skolkovo Institute of Science and Technology, Moscow, Russia; Monisensa Development LLC., Moscow, Russia
• Classification: eess.SY (Systems and Control), cs.SY (Systems and Control)
• Publication Date: January 28, 2025
• Paper Link: https://arxiv.org/abs/2501.16841

Abstract

Non-intrusive load monitoring (NILM) is an advanced and cost-effective technique for appliance-level energy consumption monitoring. However, its adoption is limited by lack of transparency and explainability. To address this challenge, this paper proposes an explainable, real-time, event-based NILM framework specifically designed for high-frequency datasets. The framework ensures transparency at each stage by integrating a z-score-based event detector, device signature estimation, Fourier-based feature extraction, XGBoost classifier, and post-hoc SHAP analysis. SHAP analysis further quantifies the contribution of individual features (such as cosine values of specific harmonic phases) to device classification. The framework is trained and evaluated on the PLAID dataset, achieving 90% classification accuracy while maintaining low computational requirements and latency below 1 second.

Research Background and Motivation

Problem Definition

With continuous growth in energy consumption, load monitoring has become critical. In the European Union, buildings account for approximately 41% of electricity consumption. Traditional intrusive load monitoring (ILM) requires installing sensors on each device, which, while accurate, is expensive, difficult to install, and unsuitable for large-scale applications. NILM serves as an alternative approach, monitoring total energy consumption at a single point (typically the main meter) and using decomposition algorithms to estimate energy consumption of individual devices.

Limitations of Existing Methods

1. Lack of Explainability: Existing NILM methods, particularly deep learning models, exhibit "black-box" characteristics with limited transparency in decision-making processes
2. High Computational Complexity: Non-event-driven methods require more computational resources and large annotated datasets
3. Insufficient Real-Time Performance: Existing methods struggle to meet real-time application requirements on edge devices
4. Limited High-Frequency Data Processing: Existing explainable NILM research primarily focuses on low-frequency data (1 Hz)

Research Motivation

This work aims to construct an end-to-end explainable NILM framework specifically for high-frequency data (kHz range), enabling real-time decomposition with minimum latency of 320/380 milliseconds (for 60/50 Hz power systems), to build trust in NILM systems and promote their adoption in practical applications.

Core Contributions

1. First Explainable End-to-End Real-Time NILM Framework for High-Frequency Data: Proposes a complete explainable framework specifically designed for high-frequency datasets
2. Real-Time Edge Device Applicability: The method is applicable to real-time applications on edge devices with latency of 19/f₀ + τ seconds (f₀ is fundamental frequency, τ is execution time)
3. Optimized Feature Selection: Selects a small set of minimally correlated and interpretable device signature features, achieving 90% classification accuracy on real datasets
4. Open-Source Implementation: Publicly releases framework source code on GitHub, promoting research reproducibility and application

Methodology Details

Task Definition

Input: High-frequency voltage and current signals (30 kHz sampling rate) Output: Device on/off event detection and classification results with decision explanations Constraints: Real-time processing requirements, low computational complexity, high explainability

Model Architecture

1. Real-Time Voltage-Current Processing

Uses Frequency-Invariant Transform with Periodic Signal (FIT-PS) to process voltage v and current i signals:

ṽ, ĩ = FITPS(v, i)

This algorithm detects zero-crossing points in the voltage signal and resamples voltage and current vectors to a common length T=500.

2. Z-Score Event Detector

Computes active power:

p(k) = (1/T) × Σ(ṽₜ(k) × ĩₜ(k))

Detects events when z-score exceeds a predetermined threshold Z=30:

z(k) = |p(k) - μw| / σw, z(k) > Z

where μw and σw are the mean and standard deviation within a sliding window (w=10), respectively.

3. Device Signature Estimation

Computes activation/deactivation current:

Δi(k) = iₐ(k) - iᵦ

where iₐ(k) is the current vector after the event and iᵦ is the current from one cycle before the event. Selects 1 cycle before and 18 cycles after the event for computation, then extracts the median cycle to obtain device signature estimation:

iₑₛₜ = median{Δi(k)}

4. Fourier Feature Extraction

Processes the estimated current signature using Discrete Fast Fourier Transform (DFFT), extracting 8 features:

{a₁, cosφ₁, cosφ₂, cosφ₃, cosφ₄, cosφ₅, cosφ₇, cosφ₉}

where a₁ is the fundamental amplitude and cosφᵢ is the cosine of the phase angle of the i-th harmonic.

5. XGBoost Classifier

Uses gradient boosting model for device classification with hyperparameters:

• 150 estimators
• Maximum depth 8
• Learning rate 0.046
• Regularization coefficient α=10

6. SHAP Post-Hoc Analysis

Uses SHapley Additive exPlanations to quantify each feature's contribution to model predictions, providing decision explanations.

Technical Innovations

1. Frequency-Invariant Transform: Addresses power grid frequency fluctuations and simplifies subsequent mathematical operations
2. Optimized Signature Estimation: Configuration of 1 preceding cycle and 18 succeeding cycles balances accuracy and computational efficiency
3. Explainable Feature Design: Uses cosine transformation for phase features, enhancing interpretability
4. Low-Correlation Feature Set: Carefully selected 8 features with weak correlations improve explainability

Experimental Setup

Dataset

PLAID Dataset: Contains 575 high-frequency voltage-current measurements at 30 kHz sampling rate, covering 13 device types and at least 11 different brands.

• Training Set: 164,510 single-cycle current signatures
• Test Set: 2,347 activation/deactivation current signatures

Evaluation Metrics

• Accuracy
• Macro Precision
• Macro Recall
• Macro F1-Score

Baseline Methods

• Decision Tree
• Logistic Regression

Implementation Details

• Random search for XGBoost hyperparameter optimization
• Event detection threshold Z=30
• Sliding window size w=10
• Samples per cycle T=500

Experimental Results

Main Results

The XGBoost model significantly outperforms other methods across all metrics, achieving 90% classification accuracy.

Ablation Study

Impact of Post-Event Cycles: Experiments demonstrate that accuracy reaches maximum with 18 post-event cycles, with additional cycles providing no significant performance improvement.

Case Analysis

Confusion Matrix Analysis: The model performs well on most devices but confuses electric irons and light bulbs, as both are resistive loads with similar current consumption magnitudes. Air conditioners may be confused with refrigerators or fans depending on operating mode.

SHAP Explanation Example: For air conditioner classification, low cosine values of 1st, 3rd, 5th, and 9th harmonic phases indicate the signature is unlikely to correspond to an air conditioner, while low cosφ₇ values indicate correspondence to an air conditioner.

Key Findings

1. Feature Correlation: The selected 8 features exhibit weak correlation, enhancing explainability
2. Computational Efficiency: Overall complexity is O(TNₐ log TNₐ), suitable for real-time applications
3. Latency Performance: Latency is 19/f₀ + τ, approximately 380 ms for 50 Hz systems and 320 ms for 60 Hz systems

Related Work

NILM Development History

The event-driven NILM concept was first proposed by Hart, identifying devices by detecting significant changes in active and reactive power.

Current State of Explainable NILM Research

1. Murray et al.: Proposed device masking activation method, limited to single devices
2. Convolutional Neural Network Interpretation: Uses occlusion sensitivity and gradient-based activation mapping, but heatmap visualization poses challenges for non-expert users
3. Regression-Based Deep Learning: Applies advanced interpretation techniques but requires standardized benchmarks and scalable solutions

Advantages of This Work

Compared to existing work, this paper provides the first complete explainable framework for high-frequency data with higher computational efficiency and real-time performance.

Conclusions and Discussion

Main Conclusions

1. Proposes the first explainable real-time NILM framework for high-frequency data
2. Achieves 90% device classification accuracy while maintaining low computational requirements
3. Provides clear decision explanations through SHAP analysis
4. Validates the framework's potential for real-time applications on edge devices

Limitations

1. Dataset Limitations: Validated only on PLAID dataset; generalization capability requires further verification
2. Device Type Restrictions: Limited discrimination capability for devices with similar characteristics (e.g., resistive loads)
3. Environmental Adaptability: Robustness in complex noisy environments or with simultaneous multi-device state changes requires further verification

Future Directions

1. Cross-Dataset Generalization: Validate framework generalization on other datasets
2. Real-World Deployment Testing: Test with high-precision real-time energy sensors in actual apartment environments
3. Algorithm Optimization: Further optimize algorithms to handle more complex load variation scenarios

In-Depth Evaluation

Strengths

1. Strong Innovation: First complete explainable framework for high-frequency NILM
2. High Practical Value: Low latency and computational complexity suitable for edge device deployment
3. Solid Theoretical Foundation: Clear mathematical basis at each stage enhances transparency
4. Open-Source Contribution: Provides open-source code promoting research reproducibility and development
5. Excellent Feature Engineering: Carefully designed 8 features ensure both performance and explainability

Weaknesses

1. Limited Evaluation: Evaluated on single dataset only, lacking cross-domain validation
2. Insufficient Comparison: Lacks detailed comparison with other advanced NILM methods
3. Insufficient Theoretical Analysis: Lacks analysis of theoretical performance bounds
4. Limited Robustness Verification: Insufficient testing of robustness to noise and anomalies

Impact

1. Academic Value: Opens new directions for explainable NILM research, particularly high-frequency data processing
2. Practical Value: Framework's real-time performance and low complexity offer good industrial application prospects
3. Reproducibility: Open-source code and detailed implementation details ensure good reproducibility

Applicable Scenarios

1. Smart Home Systems: Real-time energy consumption monitoring and device state awareness
2. Industrial IoT: Device operation status monitoring and energy efficiency optimization
3. Edge Computing Applications: Real-time load decomposition in resource-constrained environments
4. Energy Management Systems: Fine-grained energy consumption management at building or campus level

References

The paper cites 20 important references covering NILM fundamentals, machine learning methods, explainable AI techniques, and related datasets, providing solid theoretical foundation for the research.

Overall Assessment: This is a high-quality technical paper making important contributions to the explainable NILM field. The framework design is sound, experimental validation is comprehensive, and it possesses good practical and academic value. Future work should strengthen cross-dataset validation and comparative analysis with advanced methods.