电力市场大数据分析

Thisbookaimstosolvesomekeyproblemsinthedecisionandoptimizationprocedureforpowermarketorganizersandparticipantsindata-drivenapproaches.Itbeginswithanoverviewofthepowermarketdataandanalyzesontheircharacteristicsandimportanceformarketclearance.Thenitdiscussestheessentialproblemofbusloadforecastingfromtheperspectiveofmarketorganizers.Therelatedworksincludingloaduncertaintymodeling,busloadbaddatacorrection,andmonthlyloadforecasting.Thefollowingworkstrytomodelthecomplexmarketbiddingbehaviors.Specificworksincludepatternextraction,aggregatedsupplycurvesforecasting,marketsimulation,andrewardfunctionidentificationinbidding.

Contents
1 Introduction to Power Market Data 1
1.1 Overview of Electricity Markets 1
1.2 Organization and Data Disclosure of Electricity Market 4
1.2.1 Transaction Data 5
1.2.2 Price Data 7
1.2.3 Supply and Demand Data 7
1.2.4 System Operation Data 8
1.2.5 Forecast Data 8
1.2.6 Confidential Data 9
1.3 Conclusions 9
References 9
PartⅠ Load Modeling and Forecasting
2 Load Forecasting with Smart Meter Data 13
2.1 Introduction 13
2.2 Framework 14
2.3 Ensemble Learning for Probabilistic Forecasting 16
2.3.1 Quantile Regression Averaging 17
2.3.2 Factor Quantile Regression Averaging 18
2.3.3 LASSO Quantile Regression Averaging 18
2.3.4 Quantile Gradient Boosting Regression Tree 19
2.3.5 Rolling Window-Based Forecasting 20
2.4 Case Study 20
2.4.1 Experimental Setups 2
2.4.2 Evaluation Criteria 21
2.4.3 Experimental Results 22
2.5 Conclusions 24
References 24
3 Load Data Cleaning and Forecasting 27
3.1 Introduction 27
3.2 Characteristics of Load Profiles 29
3.2.1 Low-Rank Property of Load Profiles 29
3.2.2 Bad Data in Load Profiles 30
3.3 Methodology 31
3.3.1 Framework 31
3.3.2 Singular Value Thresholding (SVT) 32
3.3.3 Quantile RF Regression 34
3.3.4 Load Forecasting 35
3.4 Evaluation Criteria 35
3.4.1 Data Cleaning-Based Criteria 35
3.4.2 Load Forecasting-Based Criteria 35
3.5 Case Study 36
3.5.1 Result of Data Cleaning 36
3.5.2 Day Ahead Point Forecast 37
3.5.3 Day Ahead Probabilistic Forecast 38
3.6 Conclusions 40
References 40
4 Monthly Electricity Consumption Forecasting 43
4.1 Introduction 43
4.2 Framework 46
4.2.1 Data Collection and Treatment 46
4.2.2 SVECM Forecasting 47
4.2.3 Self-adaptive Screening 48
4.2.4 Novelty and Characteristics of SAS-SVECM 48
4.3 Data Collection and Treatment 48
4.3.1 Data Collection and Tests 49
4.3.2 Seasonal Adjustments Based on X-12-ARIMA 49
4.4 SVECM Forecasting 49
4.4.1 VECM Forecasting 49
4.4.2 Time Series Extrapolation Forecasting 52
4.5 Self-adaptive Screening 53
4.5.1 Influential EEF Identification 53
4.5.2 Influential EEF Grouping 53
4.5.3 Forecasting Performance Evaluation Considering Different EEF Groups 55
4.6 Case Study 56
4.6.1 Basic Data and Tests 56
4.6.2 Electricity Consumption Forecasting Performance Without SAS 58
4.6.3 EC Forecasting Performance with SAS 61
4.6.4 SAS-SVECM Forecasting Comparisons with Other Forecasting Methods 65
4.7 Conclusions 67
References 67
5 Probabilistic Load Forecasting 71
5.1 Introduction 71
5.2 Data and Model 73
5.2.1 Load Dataset Exploration 73
5.2.2 Linear Regression Model Considering Recency-Effects 73
5.3 Pre-Lasso Based Feature Selection 76
5.4 Sparse Penalized Quantile Regression (Quantile-Lasso) 77
5.4.1 Problem Formulation 77
5.4.2 ADMM Algorithm 78
5.5 Implementation 80
5.6 Case Study 81
5.6.1 Experiment Setups 81
5.6.2 Results 82
5.7 Concluding Remarks 86
References 86
Part Ⅱ Electricity Price Modeling and Forecasting
6 Subspace Characteristics of LMP Data 91
6.1 Introduction 91
6.2 Model and Distribution of LMP 93
6.3 Methodology 
6.3.1 Problem Formulation 96
6.3.2 Basic Framework 97
6.3.3 Principal Component Analysis 98
6.3.4 Recursive Basis Search (Bottom-Up) 98
6.3.5 Hyperplane Detection (Top-down) 100
6.3.6 Short Summary 103
6.4 Case Study 103
6.4.1 Case 1: IEEE 30-Bus System 104
6.4.2 Case 2: IEEE 118-Bus System 106
6.4.3 Case 3: Illinois 200-Bus System 106
6.4.4 Case 4: Southwest Power Pool (SPP) 107
6.4.5 Time Consumption 108
6.5 Discussion and Conclusion 110
6.5.1 Discussion on Potential Applications 110
6.5.2 Conclusion 110
References 111
7 Day-Ahead Electricity Price Forecasting 113
7.1 Introduction 113
7.2 Problem Formulation 116
7.2.1 Decomposition of LMP 116
7.2.2 Short-Term Forecast for Each Component 117
7.2.3 Summation and Stacking of Individual Forecasts 118
7.3 Methodology 119
7.3.1 Framework 119
7.3.2 Feature Engineering 121
7.3.3 Regression Model Selection and Parameter Tuning 122
7.3.4 Model Stacking with Robust Regression 123
7.3.5 Metrics 124
7.4 Case Study 124
7.4.1 Model Selection Results 125
7.4.2 Componential Results 126
7.4.3 Stacking Results (Overall Improvements) 128
7.4.4 Error Distribution Analysis 129
7.5 Conclusion 132
References 132
8 Economic Impact of Price Forecasting Error 135
8.1 Introduction 135
8.2 General Bidding Models 137
8.2.1 Deterministic Bidding Model 138
8.2.2 Stochastic Bidding Model 139
8.3 Methodology and Framework 141
8.3.1 Forecasting Error Modeling 141
8.3.2 Multiparametric Linear Programming (MPLP)Theory 141
8.3.3 Error Impact Formulation 142
8.3.4 Overall Framework 144
8.4 Case Study 145
8.4.1 Measurement of STPF Error Level 145
8.4.2 Case 1: LSE with Demand Response Programs 147
8.4.3 Case 2: LSE with ESS 148
8.4.4 Case 3: Stochastic LSE Bidding Model 151
8.4.5 Time Consumption 153
8.5 Conclusions and Future Work 153
References 153
9 LMP Forecasting and FTR Speculation 155
9.1 Introduction 155
9.2 Stochastic Optimization Model 158
9.2.1 Model of FTR Portfolio Construction Problem 158
9.2.2 Scenario-Based Stochastic Optimization Model 159
Contents
9.3 Data-Dnven Framework 160
9.4 Methodology 161
9.4.1 Clustering 161
9.4.2 Mid-Term Probabilistic Forecasting 164
9.4.3 Copulas for Dependence Modeling 165
9.4.4 Training and Evaluation Timeline 166
9.4.5 Scenario Generation 167
9.5 Case Study 167
9.5.1 Data Description 167
9.5.2 Comparison Methods 168
9.5.3 Statistical Validation of Quantile Regression 169
9.5.4 Scenario Quality Evaluation 169
9.5.5 Impact of Node Reduction with Clustering 171
9.5.6 Revenue and Risk Estimation 171
9.5.7 Sensitivity Analysis on the Number of Clusters 175
9.6 Conclusion 177
References 177
Part Ⅲ Market Bidding Behavior Analysis
10 Pattern Extraction for Bidding Behaviors 183
10.1 Introduction 183
10.2 Assumptions and Proposed Framework 186
10.2.1 Model Assumptions 186
10.2.2 Bidding Data Format 187
10.2.3 Data-Driven Analysis Framework 188
10.3 Data Standardization Processing 188
10.3.1 Filtering Available Capacities 188
10.3.2 Sampling Bidding Curves 189
10.3.3 Unifying Data Length 189
10.3.4 Clipping Extreme Prices 191
10.4 Adaptive Clustering of Bidding Behaviors 191
10.4.1 Distance Measurement 192
10.4.2 K-Medoids Clustering 192
10.4.3 Adaptive Clustering Procedure 192
10.4.4 Clustering Algorithm 193
10.5 AEM Data Description 194
10.5.1 Description of Market Participants 194
10.5.2 Description of Bidding Data 195
10.6 Bidding Pattern Analysis 195
10.6.1 Parameter Setting 196
10.6.2 Bidding Patterns of DUs by Fuel Type 197
10.6.3 Comparison of Similar DUs 201
10.6.4 Discussion 203
10.7 Feature Analysis on Bids 203
10.7.1 Discrete Aggregation Feature 204
10.7.2 Probability Distribution Feature 205
10.7.3 Time Distribution Feature 206
10.8 Conclusions 206
References 208
11 Aggregated Supply Curves Forecasting 211
11.1 Introduction 211
11.2 Market and Framework 214
11.2.1 Market Descriptions 214
11.2.2 Forecasting Framework 215
11.3 Data Integration and Feature Extraction 216
11.3.1 Data Integration 216
11.3.2 Feature Extraction 219
11.4 ASC Forecasting 221
11.4.1 LSTM Model 221
11.4.2 Influencing Factors 222
11.4.3 Training and Forecasting 223
11.4.4 Evaluation Criteria 223
11.5 Case Study 224
11.5.1 Dataset Description 224
11.5.2 Feature Extraction 224
11.5.3 ASC Forecasting 227
11.5.4 Calculation Information 234
11.5.5 Methods Comparison 234
11.6 Conclusion 235
References 236
12 Learning Individual Offering Strategy 239
12.1 Introduction 239
12.2 Data-Driven Market Simulation Framework 242
12.2.1 Market Assumptions 242
12.2.2 Offering Data Clustering and Indexing 243
12.3 Individual Offering Strategy Learning 245
12.3.1 MFNN Model Structure 246
12.3.2 MFNN Model Inputs 247
12.3.3 MFNN Model Training 248
12.3.4 DNN-Based Model Structure 249
12.4 Market Clearing Simulation 249
12.5 Case Study 251
12.5.1 Basic Data 251
12.5.2 Individual Offering Behavior Forecasting 253
12.5.3 Market Simulation 254
12.5.4 Comparison with Current Price Forecasting Methods 259
12.5.5 Calculation Efficiency 260
12.6 Conclusions 260
References 261
13 Reward Function Identification of GENCOs 265
13.1 Introduction 265
13.2 Assumptions and Framework 267
13.2.1 Market Assumptions 267
13.2.2 Data-Driven Framework 267
13.3 Bidding Decision Process Formulation 269
13.3.1 Markov Decision Process in Wholesale Markets 269
13.3.2 Reinforcement Learning Process 270
13.3.3 Bidding Data Integration 270
13.4 Reward Function Identification 271
13.4.1 Deep Inverse Reinforcement Learning Algorithm 271
13.4.2 Discretization Methods for States and Actions 273
13.5 Bidding Behavior Simulation 273
13.5.1 DQN-Based Bidding Simulation Model 273
13.5.2 Value Function and Q-Network 274
13.6 Case Study 275
13.6.1 Dataset Description 275
13.6.2 Parameter Setting 276
13.6.3 Reward Function Identification 276
13.6.4 Bidding Behavior Simulation 281
13.7 Conclusions 282
References 283