CN117578495A - Distributed photovoltaic multi-agent cluster voltage control method and system - Google Patents

Abstract

The invention discloses a distributed photovoltaic multi-main-body cluster voltage regulation and control method and system, which construct a multi-agent collaborative optimization structure of a deep reinforcement learning framework based on a DSO and a system framework of a multi-main-body photovoltaic user; calculating the voltage support and the photovoltaic utilization rate based on the multi-agent cooperative optimization structure; comprehensively representing the voltage regulation and digestion performances of the photovoltaic by using two indexes of voltage support and photovoltaic utilization rate, and constructing a photovoltaic grading model; based on the photovoltaic grading model, establishing an upper-layer and lower-layer collaborative optimization model between DSO and a photovoltaic user; and establishing a reactive voltage control optimization model based on cooperation of a distribution network aggregator and a distributed photovoltaic user based on the upper and lower layer cooperation optimization model, and solving to obtain an optimal voltage regulation strategy with maximized upper and lower layer cooperation benefit. The invention can enable the energy utilization strategy of the photovoltaic user to dynamically track the change of the network topology, and is beneficial to improving the solving speed and improving the convergence of solving.

Description

Distributed photovoltaic multi-agent cluster voltage control method and system

Technical field

The invention belongs to the technical field of voltage regulation control of distribution networks under distributed resource access, and particularly relates to a distributed photovoltaic multi-subject cluster voltage regulation method and system.

Background technique

As a distributed resource, the proportion of distributed photovoltaic energy connected to the power grid on the user side of the distribution network is increasing. With the access of large-scale distributed energy resources, the voltage distribution of the entire network changes, which may lead to voltage over-limit faults in serious cases. Therefore, voltage regulation operations need to be performed on the distribution network side. At present, in addition to participating in regulating the capacity of the load side as a new energy source, distributed photovoltaics can also be used as a new voltage regulating entity to participate in the voltage regulation of the distribution network through its grid-connected converter.

In order to effectively use distributed photovoltaics to regulate the voltage of the distribution network and solve the problem of distribution network voltage exceeding the limit, most of the current control strategies are based on direct control, that is, the distribution network operator directly controls the voltage according to the operating status and conditions of the power grid. Distributed photovoltaics issue instructions to participate in the voltage regulation process. In view of the shortcomings in the method of direct control to adjust the distribution network voltage and the solution process, a new distribution network voltage regulation strategy is urgently needed.

Contents of the invention

The purpose of the present invention is to provide a distributed photovoltaic multi-subject cluster voltage control method and system to solve the problems existing in the existing technology. The present invention establishes a distributed photovoltaic user hierarchical cluster model that comprehensively considers voltage support and photovoltaic utilization. The research object is decomposed into upper-layer distribution network operators and lower-layer distributed photovoltaic users. The upper and lower layers implement hierarchical cluster voltage regulation control through deep reinforcement learning theory. Finally, DQN, DDPG and firefly algorithms are used for optimization and solution. It can enable photovoltaic users' energy consumption strategies to dynamically track changes in network topology, and help improve the solution speed and convergence of the solution.

In order to achieve the above objects, the present invention adopts the following technical solutions:

The distributed photovoltaic multi-agent cluster voltage control method includes the following steps:

Based on the system framework of DSO and multi-agent photovoltaic users, a multi-agent collaborative optimization structure of the deep reinforcement learning framework is constructed;

Based on the multi-agent collaborative optimization structure, the voltage support degree and photovoltaic utilization rate are calculated;

The two indicators of voltage support and photovoltaic utilization rate are used to comprehensively characterize the voltage regulation and absorption performance of photovoltaic, and build a photovoltaic classification model;

Based on the photovoltaic classification model, establish an upper-lower collaborative optimization model between DSO and photovoltaic users;

Based on the upper-lower layer collaborative optimization model between DSO and photovoltaic users, a reactive power and voltage control optimization model based on the collaboration between distribution network aggregators and distributed photovoltaic users is established;

The reactive power and voltage control optimization model based on the collaboration between distribution network aggregators and distributed photovoltaic users is solved to obtain the optimal voltage regulation strategy that maximizes the collaborative benefits of the upper and lower layers.

Further, the voltage support calculation formula is as follows:

Among them, γ _i,t represents the voltage support degree of node i at time t, Φ is the set of nodes with voltage exceeding the limit; n _ol is the number of nodes that exceed the limit, S _Q,i,j,t is the reactive power between nodes at time t - The voltage sensitivity coefficient represents the change in the voltage of node i when node j changes per unit amount of reactive power;

Among them, ΔU _i,t and ΔQ _i,t are the voltage amplitude and reactive power change of node i at time t respectively;

The calculation formula of the photovoltaic utilization rate is as follows:

P _a (t)=P _PV (t)-P _S.PV (t)

Among them, R _PV is the photovoltaic utilization rate, P _PV (t) is the theoretical power generation of photovoltaic at time t; P _a (t) is the actual photovoltaic power utilization at time t, and P _S.PV (t) is the waste energy at time t. amount of light.

Further, the two indicators of voltage support and photovoltaic utilization are used to comprehensively characterize the voltage regulation and accommodation performance of photovoltaics, and a photovoltaic classification model is constructed, specifically as follows:

On the DSO side, voltage support and photovoltaic utilization are used as two-dimensional parameters and the K-means clustering method is used to perform similarity clustering, and photovoltaic users with similar voltage regulation performance are divided into the same level, which is expressed as follows:

Among them, * indicates that due to the different orders of magnitude between voltage support and photovoltaic utilization, the two types of parameters are first normalized to values between 0 and 1; k _i is the weight of different parameters, with k ₁ + k ₂ = 1 , and in The classification results are stored in; in addition, the similarity index d _ij,t used to measure is:

The lower the similarity index, the easier it is for the corresponding distributed photovoltaic users to be divided into the same level, and the photovoltaic voltage support and photovoltaic utilization rate within the same level are similar. DSO can enable photovoltaics in the same level to use the same voltage regulation compensation price to ensure Properly allocate reactive power regulation for voltage regulation.

Further, the upper-lower collaborative optimization model between the DSO and photovoltaic users is specifically: the upper-layer DSO dominates the voltage regulation of the distribution network, as the party that provides compensation for electricity prices; the lower-layer photovoltaic users use electricity prices as incentives to participate in the auxiliary service market of voltage regulation. ;

As a distribution network operator, DSO must maintain voltage stability during the voltage regulation process and ensure that the photovoltaic utilization rate is as high as possible, that is, the total voltage regulation cost is as low as possible, as follows:

Among them, w _q,k,t is the voltage regulation compensation price of photovoltaic users in the k-th level at time t; K is the total number of levels of the user grading strategy; m _k is the number of photovoltaic users in the k-th level; U _i,t is the voltage amplitude of node i at time t; U _i,ref is the voltage amplitude reference value of node i; R _i,PV is the photovoltaic utilization rate of distributed photovoltaic connected to node i; α is the cost coefficient to maintain voltage stability , β is the light abandonment cost coefficient;

The constraints are:

P _i,t =P _PV,i,t -P _load,i,t

U _min,t ≤U _t ≤U _max,t

P _L,min,t ≤P _L,t ≤P _L,max,t

Among them, P _i,t and Q _i,t are the active and reactive power injected at node i at time t respectively; θ _ij,t is the electrical phase angle difference between nodes i and j at time t; G _ij and B _ij are respectively The conductance and susceptance of branch ij; j∈i represents all nodes adjacent to node i; P _ij,t is the active power flowing from node i to node j at time t; P _load,i,t is User load at node i at time t; U _t is the voltage amplitude of each node after voltage regulation at time t, U _max,t and U _min,t are respectively the upper and lower limits of the voltage allowed at time t; P _L,t is t The branch power flow of the network at time, P _L,max,t and P _L,min,t are the upper and lower limits of the branch power flow at time t respectively.

Furthermore, the constraints also include:

w _q,min,i,t ≤w _q,i,t ≤w _q,max,i,t

Among them, w _q,max,i,t and w _q,min,i,t are the upper and lower limits of the voltage regulation compensation price at time t respectively; Q _max and Q _min are the reactive power output of the grid-connected photovoltaic inverter respectively. Lower limit, S _inv is the rated power of the grid-connected photovoltaic inverter, P _PV is the photovoltaic active output in this state; Q′ _max and Q′ _min are respectively the upper and lower limits of the reactive power output of the grid-connected photovoltaic inverter in this state. , P′ _PV is the range where the photovoltaic active power can be reduced in this state, and P _PVmax is the maximum active power output value of the photovoltaic in this state.

Further, the reactive power and voltage control optimization model based on the collaboration between distribution network aggregators and distributed photovoltaic users is specifically:

The income of photovoltaic users mainly considers the income from participating in voltage regulation and the cost required in the adjustment process, as follows:

I _u,i,t =I _u,gro,i,t -C _i,t

I _u,gro,i,t =w _q,i,t ΔQ _i,t

Among them, I _u,i,t is the total voltage regulation income of the user at node i at time t; I _u,gro,i,t is the compensation income obtained by the user due to participating in voltage regulation; C _i,t is the voltage regulation process required costs;

The required cost C _i,t in the voltage regulation process is divided into the following two categories according to the size of the reactive power adjustment amount:

1) Only reactive power adjustment, that is, the reactive power adjustment amount does not exceed the upper limit of reactive power adjustment, and the photovoltaic active power output is normal, then only the cost of converter capacity occupation is considered:

Among them, c _p is the on-grid electricity price of photovoltaic; ξ is the utilization rate of the capacity occupied by the reactive power adjustment of the photovoltaic inverter. The higher the utilization rate under the same capacity, the higher the cost of reactive power adjustment;

2) Joint adjustment of active and reactive power, that is, if the reactive power adjustment amount exceeds the upper limit of reactive power adjustment, the photovoltaic active power has the function of reducing to release more reactive power capacity. At this time, the cost also includes reducing part of the active power sales income:

The corresponding photovoltaic active power reduction and reactive power absorption are constrained by the self-regulation capacity:

0≤ΔP _i,t ≤P _PV,i,t -P _load,i,t

Further, the solution is based on a reactive power and voltage control optimization model in which distribution network aggregators and distributed photovoltaic users collaborate, specifically as follows:

DQN and DDPG deep reinforcement learning algorithms are used to realize voltage control. First, for DQN, the voltage fluctuation index is used as the reward function, the specific actions of the voltage regulating device are used as the action space, and a new state space is obtained and stored in the data value experience pool D _DQN ; In addition, the DDPG algorithm is the same. It establishes an action set based on the continuous action space and considers the active and reactive power of the photovoltaic. It uses the node voltage limit and the power adjustment amount as the reward function to iteratively obtain a new state space and store the empirical data to the experience Pool D _DDPG , at this time, the outer layer obtains the voltage regulation strategy and results of DQN and DDPG in one cycle, and inputs them into the inner layer as parameters; the inner layer uses the comprehensive benefits of DSO and distributed photovoltaic user clusters as the objective function, and uses the firefly algorithm to calculate the given The voltage regulation strategy is used to optimize the voltage regulation compensation electricity price, and finally the optimal voltage regulation strategy that maximizes the collaborative benefits of the upper and lower layers is obtained.

Distributed photovoltaic multi-agent cluster voltage control system, including:

Multi-agent collaborative optimization structure building module: Based on the system framework of DSO and multi-agent photovoltaic users, a multi-agent collaborative optimization structure of the deep reinforcement learning framework is constructed;

Calculation module: Based on the multi-agent collaborative optimization structure, calculate the voltage support and photovoltaic utilization rate;

Photovoltaic classification model building module: Use the two indicators of voltage support and photovoltaic utilization to comprehensively characterize the voltage regulation and accommodation performance of photovoltaics and build a photovoltaic classification model;

The upper-lower layer collaborative optimization model establishment module between DSO and photovoltaic users: Based on the photovoltaic classification model, establish an upper-lower layer collaborative optimization model between DSO and photovoltaic users;

Based on the upper-lower layer collaborative optimization model between DSO and photovoltaic users, a reactive power and voltage control optimization model based on the collaboration between distribution network aggregators and distributed photovoltaic users is established;

The reactive power and voltage control optimization model based on the collaboration between distribution network aggregators and distributed photovoltaic users is solved to obtain the optimal voltage regulation strategy that maximizes the collaborative benefits of the upper and lower layers.

A computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the distributed photovoltaic multi-body cluster voltage is implemented. The steps of the control method.

A computer-readable storage medium stores a computer program, which is characterized in that when the computer program is executed by a processor, the steps of the distributed photovoltaic multi-subject cluster voltage control method are implemented.

Compared with the existing technology, the present invention has the following beneficial technical effects:

The method provided by the present invention defines voltage regulation performance evaluation including two indicators: voltage support and photovoltaic utilization rate. It takes into account that reactive power regulation does not affect the user's power generation income, and the adjustment cost is far less than the income loss caused by active power reduction. It is a distributed The photovoltaic user-side hierarchical cluster division and reactive and active power regulation and switching modeling provide an indicator basis; the distributed photovoltaic users are clustered and classified, dynamically following the network topology, reflecting the coupling relationship between photovoltaic active power and reactive power, and simplifying the distribution network operator's From the voltage regulation compensation division of side photovoltaic users; the DQN and DDPG algorithms of deep reinforcement learning are used in the upper and lower layer collaborative voltage regulation strategy algorithm of distribution network aggregators and distributed photovoltaic users, effectively improving the convergence and solution speed of the solution; Taking into account the master-slave game framework of photovoltaic user classification under the premise of voltage regulation compensation, an optimization scheme is proposed for the voltage regulation target of active and reactive power coupling on the distribution network side.

Description of the drawings

The description and drawings are used to provide a further understanding of the present invention and constitute a part of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.

Figure 1 is a schematic flow chart of a collaborative voltage regulation optimization solution algorithm implemented by the present invention based on deep reinforcement learning between upper-level aggregators and lower-level distributed photovoltaic users.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only These are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the scope of protection of the present invention.

It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the invention described herein are capable of being practiced in sequences other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus that encompasses a series of steps or units and need not be limited to those explicitly listed. Those steps or elements may instead include other steps or elements not expressly listed or inherent to the process, method, product or apparatus.

Embodiment 1

The present invention proposes a distributed photovoltaic multi-subject cluster reactive power and voltage control method based on a deep reinforcement learning framework and algorithm. The specific steps of the method are as follows:

Step 1: Deep reinforcement learning framework and photovoltaic control mode based on distributed photovoltaic voltage regulation

In order to solve the voltage regulation problem of the distribution network involving distributed photovoltaics, the present invention relies on the system framework of DSO and multi-subject photovoltaic users, in which the DSO is equipped with an energy management system (Energy Management System, EMS) for collecting the grid structure. Parameters and grid operation data determine the subsequent power flow calculation, photovoltaic classification and voltage regulation compensation price. DSO is responsible for orderly guidance of photovoltaic users during the voltage regulation process to ensure the overall safe and stable operation of the distribution network. Distributed photovoltaic users adopt the operating mode of "self-use for self-use and grid-connected surplus power". The distributed photovoltaic system mainly includes solar cells, load and user energy management system (UEMS), which is used to receive energy from the power grid. Send information and obtain its own reactive power adjustment strategy through optimization calculation.

Distributed photovoltaic uses the remaining capacity of the grid-connected converter to absorb reactive power and then changes the node injection power. When the voltage exceeds the limit seriously, that is, when the reactive power adjustment capacity of the distribution network cannot meet the voltage regulation demand, distributed photovoltaic can reduce the The active output releases part of the inverter capacity for voltage regulation. The current two photovoltaic voltage regulation methods are reactive power regulation and active and reactive combined regulation.

In the optimization framework of aggregators and distributed photovoltaic user clusters based on deep reinforcement learning, distribution network operators and distributed photovoltaic users are their respective agents, that is, a multi-agent collaborative optimization structure of the deep reinforcement learning framework is constructed. The overall environment required for cliff jumping is Environment. At the same time, according to the basic logic of Q-learning reinforcement learning, the specific voltage regulation actions of the distribution network aggregator and distributed photovoltaic users are their respective action strategies (Action), and the distributed photovoltaic users make adjustments based on the relationship between the DSO and its voltage regulation compensation electricity price. The voltage regulation strategy (State) is fed back to the distribution network operator, and finally the objective function that meets the requirements of each subject is optimized and its optimization result (Reward) is obtained.

Step 2: Voltage support and photovoltaic utilization

1.Voltage support

Voltage support indicates the degree of impact of changes in distributed photovoltaic reactive power on the distribution network voltage. Sensitivity is used to represent the linear change relationship between node voltage amplitude and node power injection:

Among them, formula (1) is the active power-voltage sensitivity coefficient, and formula (2) is the reactive power-voltage sensitivity coefficient. That is, the degree to which the output change of a certain node in the distribution network improves the node voltage. In the formula, ΔU _i,t , ΔP _i,t , ΔQ _i,t are the voltage amplitude, injected active power and reactive power changes of node i at time t respectively; S _P,i,j,t and S _{Q ,i,j,t} are respectively the active-voltage sensitivity and reactive-voltage sensitivity coefficients between nodes at time t, which respectively represent the change in the voltage of node i when node j changes unit amount of active and reactive power, which can be calculated by the power flow. Find the corrected equation.

It can be concluded that power changes between different nodes have different effects on voltage, so each node has different support for voltage and its ability to participate in voltage regulation is also different. The voltage support degree γ _{i,t of node i at time t} can be calculated using the reactive power-voltage sensitivity coefficient:

Among them, Φ is the set of nodes whose voltage exceeds the limit; _nol is the number of nodes that exceed the limit. The greater the node's voltage support index, the stronger the photovoltaic node's ability to improve voltage problems in the power grid. That is, the smaller the reactive power required to change the voltage of the entire grid to the same effect, and the higher the voltage regulation efficiency. .

2. Photovoltaic utilization rate

Considering that the fundamental purpose of setting up distributed photovoltaics is to effectively use solar energy to support the distribution network, it is considered to reflect the consumption of new energy photovoltaics through the photovoltaic utilization rate index. The photovoltaic utilization rate R _PV is defined here as the actual photovoltaic power generation. Proportion of theoretical power generation:

P _a (t)=P _PV (t)-P _S.PV (t) (5)

In the formula: P _PV (t) is the theoretical power generation of photovoltaic at time t; P _a (t) is the actual amount of photovoltaic power used at time t, and P _S.PV (t) is the amount of abandoned light at time t.

Step 3: Build a photovoltaic classification model

Use the two indicators of voltage support and photovoltaic utilization defined in step 2 to comprehensively characterize the voltage regulation and accommodation performance of photovoltaics. The DSO side uses these two indicators as two-dimensional parameters and uses the K-means clustering method to perform similarity clustering, and divides photovoltaic users with similar voltage regulation performance into the same level:

Among them, "*" means that because the voltage support degree and photovoltaic utilization rate are of different orders of magnitude, the two types of parameters are first normalized to values between 0 and 1; k _i is the weight of different parameters, with k ₁ + k ₂ =1, and in Store the grading results in . In addition, the similarity index d _ij,t used to measure is:

Obviously, the lower the similarity index, the easier it is for the corresponding distributed photovoltaic users to be divided into the same level, and the photovoltaic voltage support and photovoltaic utilization rate within the same level are similar. DSO can enable photovoltaics in the same level to use the same voltage regulation compensation The electricity price is reasonably allocated for reactive power regulation of voltage regulation.

Step 4: DSO Benefit Model

Based on the above photovoltaic classification, in order to analyze the interactive coordination process between DSO and photovoltaic users, an upper-lower layer collaborative optimization strategy between DSO and photovoltaic users was established: the upper-layer DSO dominates the voltage regulation of the distribution network and serves as the party that provides compensation for electricity prices; the lower-layer Photovoltaic users use electricity prices as incentives to participate in the ancillary service market of voltage regulation.

As a distribution network operator, DSO must maintain voltage stability during the voltage regulation process and ensure that the photovoltaic utilization rate is as high as possible, that is, the total voltage regulation cost is as low as possible:

In the formula, w _q,k,t is the voltage regulation compensation price of photovoltaic users in the k-th level at time t; K is the total number of levels of the user grading strategy; m _k is the number of photovoltaic users in the k-th level; U _{i, t} is the voltage amplitude of node i at time t; U _i,ref is the voltage amplitude reference value of node i; R _i,PV is the photovoltaic utilization rate of distributed photovoltaic connected to node i; α is the cost of maintaining voltage stability Coefficient, β is the light abandonment cost coefficient.

Restrictions:

P _i,t =P _PV,i,t -P _load,i,t (11)

U _min,t ≤U _t ≤U _max,t (12)

P _L,min,t ≤P _L,t ≤P _L,max,t (13)

Among them, P _i,t and Q _i,t are the active and reactive power injected at node i at time t respectively; θ _ij,t is the electrical phase angle difference between nodes i and j at time t; G _ij and B _ij are respectively The conductance and susceptance of branch ij; j∈i represents all nodes adjacent to node i; P _ij,t is the active power flowing from node i to node j at time t; P _load,i,t is User load at node i at time t; U _t is the voltage amplitude of each node after voltage regulation at time t, U _max,t and U _min,t are respectively the upper and lower limits of the voltage allowed at time t; P _L,t is t The branch power flow of the network at time, P _L,max,t and P _L,min,t are the upper and lower limits of the branch power flow at time t respectively.

At the same time, voltage regulation compensation is subject to market regulations:

w _q,min,i,t ≤w _q,i,t ≤w _q,max,i,t (14)

In the formula, w _q,max,i,t and w _q,min,i,t are the upper and lower limits of the voltage regulation compensation price at time t respectively.

In addition, considering the constraints of distributed photovoltaic output, since the active output of photovoltaic is determined by the lighting conditions, the active power of photovoltaic can only be reduced but not increased. When undervoltage occurs in the distribution network, only the reactive power of the unit participates in the voltage regulation process. The maximum value of photovoltaic reactive power that can participate in voltage regulation when

In the formula, Q _max and Q _min are respectively the upper and lower limits of the reactive power output of the grid-connected photovoltaic inverter in this state, S _inv is the rated power of the grid-connected photovoltaic inverter, and P _PV is the photovoltaic active output in this state. If overvoltage occurs in the distribution network, consider simultaneous voltage regulation control of reactive power and active power. At this time, the maximum photovoltaic reactive power that can participate in voltage regulation is

In the formula, Q′ _max and Q′ _min are respectively the upper and lower limits of the reactive power output of the grid-connected photovoltaic inverter in this state, P′ _PV is the photovoltaic active power reduction range in this state, and P _PVmax is the maximum photovoltaic power output in this state. Active output value.

Step 5: Photovoltaic user multi-agent benefit model

The income of photovoltaic users mainly considers the income from participating in voltage regulation and the costs required in the adjustment process:

I _u,i,t =I _u,gro,i,t -C _i,t (17)

I _u,gro,i,t ＝w _q,i,t ΔQ _i,t (18)

Among them, I _u,i,t is the total voltage regulation income of the user at node i at time t; I _u,gro,i,t is the compensation income obtained by the user due to participating in voltage regulation; C _i,t is the voltage regulation process Required costs.

For C _i,t cost, it is divided into the following two categories according to the size of the reactive power adjustment amount:

1) Only reactive power adjustment, that is, the reactive power adjustment amount does not exceed the upper limit of reactive power adjustment, and the photovoltaic active power output is normal, then only the cost of converter capacity occupation is considered:

Among them, c _p is the on-grid electricity price of photovoltaic; ξ is the utilization rate of the capacity occupied by the reactive power regulation of the photovoltaic converter. The higher the utilization rate under the same capacity, the higher the cost of reactive power regulation.

2) Joint adjustment of active and reactive power, that is, when the reactive power adjustment amount exceeds the upper limit of reactive power adjustment, the photovoltaic active power has the function of being reduced to a certain extent to release more reactive power capacity. At this time, the cost also needs to include the reduction of part of the active power sales income. :

The corresponding photovoltaic active power reduction and reactive power absorption are constrained by the self-regulation capacity:

0≤ΔP _i,t ≤P _PV,i,t -P _load,i,t (21)

Step 6: Collaborative voltage control optimization algorithm between aggregator and distributed photovoltaic based on deep reinforcement learning

Based on Q-learning, a deep reinforcement learning algorithm is introduced, and the firefly algorithm is integrated to solve the reactive power and voltage control optimization model based on the collaboration between distribution network aggregators and distributed photovoltaic users.

The DQN and DDPG algorithms are used to jointly optimize the voltage control target. The Deep QNetwork (DQN) combines the idea of deep neural networks based on Q-learning to estimate the continuous Q function. Through the experience playback mechanism, the agent The experience data obtained at each time point during the interaction with the environment is stored in the experience pool (Experience Buffer), and then randomly sampled from the experience pool to reduce the correlation of training data and improve the solution speed and convergence ability.

In the same way, the Deep Deterministic Policy Gradient (DDPG) algorithm can effectively solve the problem of obtaining the optimal response based on the benefit objective function and constraints of the underlying distributed photovoltaic users. DDPG can explore the optimal action in the continuous action domain and obtain the global optimal solution. It adopts the Actor-Critic architecture. The former takes the state as input and the action as the output. The latter takes the two-dimensional data of state and action as input, and the estimated The Q' value is the output, and optimization is achieved by minimizing the deviation value of Q' as the goal.

The present invention uses DQN and DDPG deep reinforcement learning algorithms to realize voltage control. First, for DQN, the voltage fluctuation index is used as the reward function, and the specific actions of the voltage regulating device are used as the action space and a new state space is saved to the data value experience. Pool D _DQN ; In addition, the DDPG algorithm is the same. It establishes an action set based on the continuous action space and considers the active and reactive power of the photovoltaic. It uses the node voltage limit and the power adjustment amount as the reward function to iteratively obtain a new state space and store the empirical data. to the experience pool D _DDPG . At this time, the outer layer obtains the voltage regulation strategies and results of DQN and DDPG in one cycle, and inputs them into the inner layer as parameters; the inner layer uses the comprehensive benefits of DSO and distributed photovoltaic user clusters as the objective function, and uses the firefly algorithm to calculate the given A certain voltage regulation strategy is used to optimize the voltage regulation compensation electricity price, and finally the optimal voltage regulation strategy that maximizes the collaborative benefits of the upper and lower layers is obtained.

The specific solution steps will be explained in detail below in conjunction with Figure 1:

Step 1: DQN algorithm setting and initialization

Taking the voltage, active power and reactive power of each node as controlled objects, set the state space to be the set of the above parameters of all PQ nodes, that is:

S _DSO,i ={v ₁ ,…,v _k ,…,v _n ;p ₁ ,…,p _k ,…,p _n ;q ₁ ,…,q _k ,…,q _n } (24)

In the formula, v _k , p _k and q _k are the voltage, active power and reactive power values measured at the k-th node respectively, and 1≤k≤n is the total number of PQ nodes.

The action space of the DQN algorithm is set to the reactive power compensation amount used by node k to compensate the voltage, and is transmitted to the underlying distributed photovoltaic users and their clusters through the corresponding voltage regulation compensation price. At the same time, minimizing the node voltage overrun limit is taken as the control goal, that is, the reward function is set to be the sum of the quadratic form of each node voltage overrun limit and capacitor reactive power compensation, that is:

r _DSO,i =-[Δv ₁ ,…,Δv _i ,…,Δv _n ]Q[Δv ₁ ,…,Δv _i ,…,Δv _n ] ^T -RC _q,k (25)

In the formula, Δv _i is the overshoot limit of the voltage of node i; C _q,k is the reactive voltage regulation compensation price of node k; Q and R are the weight matrix and weight coefficient.

Complete the settings of the above action space, reward function and state space, and start iteration.

Step 2: DDPG algorithm setting and initialization

Set the current voltage, active and reactive power of each node as the state space:

S _PV,i ={v ₁ ,…,v _k ,…,v _n ;p ₁ ,…,p _k ,…,p _n ;q ₁ ,…,q _k ,…,q _n } (26)

And the DDPG algorithm is used in the continuous action space, and the active and reactive output adjustment of the lower layer in response to the upper layer voltage regulation compensation price is set as the action set A _PV,i = {A _P ,A _Q }={ΔP _i ,ΔQ _i }, At the same time, the reward function is set as the sum of the quadratic form of the voltage crossing limit of each node and the power adjustment amount, that is:

r _PV,i =-[Δv ₁ ,…,Δv _i ,…,Δv _n ]Q[Δv ₁ ,…,Δv _i ,…,Δv _n ] ^T

-[p _PV,1 ,…,p _PV,i ,…,p _PV,n ]R[p _PV,1 ,…,p _PV,i ,…,p _PV,n ] ^T

-[q _PV,1 ,…,q _PV,i ,…,q _PV,n ]J[q _PV,1 ,…,q _{PV,i ,} …,q _PV,n ] ^T (27)

In the formula, Δv _i is the voltage overshoot limit at node i; p _PV,i is the distributed photovoltaic active output of node i; q _PV,i is the distributed photovoltaic reactive power output of node i, Q, R and J are the corresponding weight matrix and weight coefficient.

The voltage regulation action of distributed photovoltaic users is executed from the given action space, and the instant reward r _PV is obtained according to Equation (27). At the same time, the state space is updated and the above experience data (S _PV ,a _PV ,r _PV ,S' _PV ) to the experience pool D _DDPG . Then the mini-batch group experience data is randomly sampled from the above-mentioned experience pool D _DDPG , and the Actor and Critic networks are updated according to the policy gradient; and the Actor and Critic target networks are updated according to soft update. After obtaining the voltage regulation strategy that meets the voltage regulation requirements, it is input into the embedded firefly algorithm, and its rapid optimization and convergence characteristics are used to further optimize the voltage regulation strategy based on the economic benefits of DSO and distributed photovoltaic user clusters.

Step 3: Firefly algorithm optimization solution

The firefly optimization algorithm is used to search for the collaborative benefit optimization strategy based on voltage regulation control, with the highest comprehensive voltage regulation benefit of the distribution network aggregator and distributed photovoltaic user cluster as the objective function, and at the same time using their respective hardware constraints as constraints:

E＝μE _D,t +(1-μ)I _u,i,t (28)

In the formula, μ is the comprehensive benefit weight of DSO and distributed photovoltaic users. Generally, μ = 0.5, that is, the benefits of the two entities have the same status.

Constraints: Formula (9)-(16); Formula (18)-(23).

1) Initialize the firefly population number, problem dimension and number of iterations, randomly generate several individual firefly positions, and define the fitness function of "brightness value";

2) Fireflies change their positions according to their respective degrees of attraction, which are related to both individual brightness and distance. Among them, random perturbation is performed during the process of updating the position of the fireflies, thereby improving the algorithm's ability to jump out of the local optimal solution;

3) Calculate the brightness, and recalculate the fitness or brightness of all fireflies based on their new positions as the fireflies approach;

4) Determine the end condition, that is, when the number of iterations reaches the upper limit or the fitness deviation is within the acceptable threshold, launch the iteration and output the optimal solution. This optimal solution is the result of maximizing the economic benefits of the upper and lower layers that meets the pressure regulation requirements. If not satisfied, continue iteration;

Step 4: Optimal Q value iteration and solution

The initial state of the DQN agent determined by Step 1 is s _DSO , and it continues to iterate the maximum Q value based on the economic benefit maximization strategy fed back by the lower layer to meet the voltage regulation needs;

Based on the ε greedy algorithm and selecting specific voltage regulation actions from the action space of DQN, the instant reward r _DSO is obtained according to Equation (25), and the DQN state space is updated at the same time. Store experience data (S _DSO , a _DSO , r _DSO , S' _DSO ) to the experience pool D _DQN , and randomly sample experience data from the experience pool and update the Q network and target Q network. Through iteration, the optimal pressure regulation strategy that maximizes the economic benefits of the upper and lower layers is finally obtained.

Embodiment 2

The invention provides a distributed photovoltaic multi-subject cluster voltage control system, including:

Multi-agent collaborative optimization structure building module: Based on the system framework of DSO and multi-agent photovoltaic users, a multi-agent collaborative optimization structure of the deep reinforcement learning framework is constructed;

Calculation module: Based on the multi-agent collaborative optimization structure, calculate the voltage support and photovoltaic utilization rate;

Photovoltaic classification model building module: Use the two indicators of voltage support and photovoltaic utilization to comprehensively characterize the voltage regulation and accommodation performance of photovoltaics and build a photovoltaic classification model;

The upper-lower layer collaborative optimization model establishment module between DSO and photovoltaic users: Based on the photovoltaic classification model, establish an upper-lower layer collaborative optimization model between DSO and photovoltaic users;

Based on the upper-lower layer collaborative optimization model between DSO and photovoltaic users, a reactive power and voltage control optimization model based on the collaboration between distribution network aggregators and distributed photovoltaic users is established;

The reactive power and voltage control optimization model based on the collaboration between distribution network aggregators and distributed photovoltaic users is solved to obtain the optimal voltage regulation strategy that maximizes the collaborative benefits of the upper and lower layers.

Embodiment 3

The present invention provides a computer device, including a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the distributed photovoltaic multi-purpose system is implemented. Steps of the subject cluster voltage regulation method.

Embodiment 4

A computer-readable storage medium stores a computer program, which is characterized in that when the computer program is executed by a processor, the steps of the distributed photovoltaic multi-subject cluster voltage control method are implemented.

Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a use A device for realizing the functions specified in one process or multiple processes of the flowchart and/or one block or multiple blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions The device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and do not limit the scope of protection. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: Those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation modes of the invention after reading the present invention, but these changes, modifications or equivalent substitutions are within the protection scope of the pending claims of the invention.

Claims (10)

1. The voltage regulation and control method for the distributed photovoltaic multi-main-body cluster is characterized by comprising the following steps of:

based on a DSO and a system frame of a multi-body photovoltaic user, constructing a multi-agent collaborative optimization structure of a deep reinforcement learning frame;

calculating the voltage support and the photovoltaic utilization rate based on the multi-agent cooperative optimization structure;

comprehensively representing the voltage regulation and digestion performances of the photovoltaic by using two indexes of voltage support and photovoltaic utilization rate, and constructing a photovoltaic grading model;

based on the photovoltaic grading model, establishing an upper-layer and lower-layer collaborative optimization model between DSO and a photovoltaic user;

based on an upper layer and lower layer collaborative optimization model between DSO and photovoltaic users, establishing a reactive voltage control optimization model based on cooperation of a distribution network aggregator and distributed photovoltaic users;

and solving a reactive voltage control optimization model based on cooperation of a distribution network aggregator and a distributed photovoltaic user to obtain an optimal voltage regulation strategy with maximized upper and lower layer cooperation benefits.

2. The distributed photovoltaic multi-body cluster voltage regulation method of claim 1, wherein the voltage support calculation formula is as follows:

wherein, gamma _i,t The voltage support degree of a node i at the moment t is represented, and phi is a node set with voltage out-of-limit; n is n _ol S is the number of out-of-limit nodes _Q,i,j,t Representing the variation of the voltage of the node i under the variation unit quantity reactive power of the node j for the reactive power-voltage sensitivity coefficient among the nodes at the moment t;

wherein DeltaU _i,t 、ΔQ _i,t The voltage amplitude and the reactive power variation of the node i at the moment t are respectively;

the calculation formula of the photovoltaic utilization rate is as follows:

P _a (t)＝P _PV (t)-P _S.PV (t)

wherein R is _PV For photovoltaic utilization, P _PV (t) photovoltaic at time tIs a theoretical power generation amount; p (P) _a (t) is the actual electricity quantity utilized by the photovoltaic at the moment t, P _S.PV And (t) is the amount of waste at time t.

3. The method for regulating and controlling the voltage of the distributed photovoltaic multi-main-body cluster according to claim 2, wherein the voltage regulation and the absorption performance of the photovoltaic are comprehensively characterized by using two indexes of voltage support degree and photovoltaic utilization rate, and a photovoltaic grading model is constructed, specifically:

the DSO side performs similarity clustering by using a K-means clustering method by taking the voltage support degree and the photovoltaic utilization rate as two-dimensional parameters, and divides photovoltaic users with similar voltage regulation performance into the same level, wherein the steps are expressed as follows:

wherein, the two types of parameters are normalized to be a numerical value between 0 and 1 because the magnitude order of the voltage support degree and the photovoltaic utilization rate is different; k (k) _i For the weight of different parameters, there is k ₁ +k ₂ =1, and inStoring the grading result; in addition, for measuring similarity index d _ij,t The method comprises the following steps:

the lower the similarity index is, the easier the corresponding distributed photovoltaic users are divided into the same level, and the photovoltaic voltage support degree and the photovoltaic utilization rate in the same level are similar, and the DSO can enable the photovoltaic in the same level to adopt the same voltage regulation compensation electricity price so as to reasonably distribute reactive power regulation for voltage regulation.

4. The distributed photovoltaic multi-body cluster voltage regulation method according to claim 1, wherein the upper and lower layer collaborative optimization model between DSO and photovoltaic users is specifically: the upper DSO controls the voltage regulation of the power distribution network as a party for providing the compensation electricity price; the lower layer photovoltaic users take electricity price as an incentive to participate in the auxiliary service market of voltage regulation;

the DSO is used as a power distribution network operator, and is used for maintaining voltage stability in the participation of voltage regulation, ensuring that the photovoltaic utilization rate is as high as possible, namely the total voltage regulation cost is as low as possible, and the method is expressed as follows:

wherein w is _q,k,t The voltage regulation compensation electricity price of the photovoltaic user in the kth stage at the moment t is obtained; k is the total number of stages of the user grading strategy; m is m _k The number of photovoltaic users in the kth stage; u (U) _i,t The voltage amplitude of the node i at the moment t; u (U) _i,ref A voltage amplitude reference value of the node i; r is R _i,PV The photovoltaic utilization rate of the distributed photovoltaic is accessed to the node i; alpha is a cost coefficient for maintaining voltage stability, and beta is a light rejection cost coefficient;

the constraint conditions are as follows:

P _i,t ＝P _PV,i,t -P _load,i,t

U _min,t ≤U _t ≤U _max,t

P _L,min,t ≤P _L,t ≤P _L,max,t

wherein P is _i,t And Q _i,t Active power and reactive power are respectively injected into the node i at the moment t; θ _ij,t The electric phase angle difference between the nodes i and j at the moment t; g _ij And B _ij Conductance and susceptance of branches ij respectivelyThe method comprises the steps of carrying out a first treatment on the surface of the j e i represents all nodes adjacent to node i; p (P) _ij,t Active power flowing from node i to node j at time t; p (P) _load,i,t The user load at the node i at the moment t; u (U) _t The voltage amplitude of each node after voltage regulation at the moment t is U _max,t And U _min,t The upper limit and the lower limit of the voltage permission at the moment t are respectively set; p (P) _L,t For the network branch tide at the moment t, P _L,max,t And P _L,min,t The upper limit and the lower limit of the branch power flow at the moment t are respectively set.

5. The method of claim 4, wherein the constraint further comprises:

w _q,min,i,t ≤w _q,i,t ≤w _q,max,i,t

wherein w is _q,max,i,t And w _q,min,i,t The upper limit and the lower limit of the voltage regulating compensation electricity price at the moment t are respectively set; q (Q) _max And Q _min The reactive output upper and lower limits of the grid-connected photovoltaic inverter are respectively S _inv For rated power of grid-connected photovoltaic inverter, P _PV Photovoltaic active power in this state; q'. _max And Q' _min Reactive power upper and lower limits, P 'of grid-connected photovoltaic inverter under the state respectively' _PV The photovoltaic active power in this state can be reduced in range, P _{PV max} The maximum active output value of the photovoltaic in this state.

6. The method for regulating and controlling the voltage of the distributed photovoltaic multi-main-body cluster according to claim 4, wherein the reactive voltage control optimization model based on cooperation of a distribution network aggregator and a distributed photovoltaic user is specifically as follows:

photovoltaic user benefits mainly consider the benefits obtained by participating in voltage regulation and the cost required in the regulation process, as follows:

I _u,i,t ＝I _u,gro,i,t -C _i,t

I _u,gro,i,t ＝w _q,i,t ΔQ _i,t

wherein I is _u,i,t The total voltage regulation income of the user at the node i at the moment t is obtained; i _u,gro,i,t Compensation income obtained for the user due to participation in voltage regulation; c (C) _i,t Is the required cost in the pressure regulating process;

for the required cost C in the pressure regulating process _i,t The reactive power adjustment amount is classified into the following two types according to the magnitude of the reactive power adjustment amount:

1) Only reactive power regulation, namely reactive power regulation quantity does not exceed the reactive power regulation upper limit, and photovoltaic active power normally outputs power, only the cost occupied by the capacity of the converter is considered:

wherein c _p The photovoltaic online electricity price is that of photovoltaic; xi is the utilization rate of the capacity occupied by reactive power regulation of the photovoltaic converter, and the higher the utilization rate is under the condition of occupying the same capacity, the higher the reactive power regulation cost is;

2) Active and reactive combined regulation, i.e. the reactive regulation quantity exceeds the reactive regulation upper limit, the photovoltaic active has the function of reducing to release more reactive capacity, and the cost also comprises the electricity selling benefits of reducing part of the active power:

the corresponding photovoltaic active reduction and reactive absorption are constrained by self-regulating capacity:

0≤ΔP _i,t ≤P _PV,i,t -P _load,i,t

7. the method for regulating and controlling the voltage of the distributed photovoltaic multi-main-body cluster according to claim 4, wherein the solving is based on a reactive voltage control optimization model which is cooperated by a distribution network aggregator and a distributed photovoltaic user, and specifically comprises the following steps:

the DQN and DDPG deep reinforcement learning algorithm is adopted to realize the control of voltage, firstly, aiming at the DQN, the voltage fluctuation quantity index is used as a rewarding function, the specific action of the voltage regulator is used as an action space, and a new state space is obtained and stored in a data value experience pool D _DQN The method comprises the steps of carrying out a first treatment on the surface of the In addition, the DDPG algorithm is similar, an action set is established based on continuous action space considering the output of photovoltaic active and reactive power, the node voltage threshold and the power adjustment are taken as rewarding functions, a new state space is obtained through iteration, and experience data are stored in an experience pool D _DDPG At this time, the outer layer circularly obtains the pressure regulating strategy and result of DQN and DDPG as parameters to be input into the inner layer; and the inner layer takes the comprehensive benefits of the DSO and the distributed photovoltaic user clusters as an objective function, and utilizes a firefly algorithm to carry out voltage regulation compensation electricity price optimization on a given voltage regulation strategy, so that the optimal voltage regulation strategy with the maximum upper and lower layer cooperative benefits is finally obtained.

8. Distributed photovoltaic multi-body cluster voltage regulation and control system, characterized by comprising:

the multi-agent collaborative optimization structure construction module comprises: based on a DSO and a system frame of a multi-body photovoltaic user, constructing a multi-agent collaborative optimization structure of a deep reinforcement learning frame;

the calculation module: calculating the voltage support and the photovoltaic utilization rate based on the multi-agent cooperative optimization structure;

and the photovoltaic hierarchical model building module: comprehensively representing the voltage regulation and digestion performances of the photovoltaic by using two indexes of voltage support and photovoltaic utilization rate, and constructing a photovoltaic grading model;

the upper and lower layer collaborative optimization model establishment module between the DSO and the photovoltaic user: based on the photovoltaic grading model, establishing an upper-layer and lower-layer collaborative optimization model between DSO and a photovoltaic user;

based on an upper layer and lower layer collaborative optimization model between DSO and photovoltaic users, establishing a reactive voltage control optimization model based on cooperation of a distribution network aggregator and distributed photovoltaic users;

and solving a reactive voltage control optimization model based on cooperation of a distribution network aggregator and a distributed photovoltaic user to obtain an optimal voltage regulation strategy with maximized upper and lower layer cooperation benefits.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the distributed photovoltaic multi-body cluster voltage regulation method according to any of claims 1 to 7 when the computer program is executed.

10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the distributed photovoltaic multi-body cluster voltage regulation method according to any one of claims 1 to 7.