+

WO2018133569A1 - Procédé et système de traitement d'informations neuronales ayant un fenêtrage temporel profond - Google Patents

Procédé et système de traitement d'informations neuronales ayant un fenêtrage temporel profond Download PDF

Info

Publication number
WO2018133569A1
WO2018133569A1 PCT/CN2017/114663 CN2017114663W WO2018133569A1 WO 2018133569 A1 WO2018133569 A1 WO 2018133569A1 CN 2017114663 W CN2017114663 W CN 2017114663W WO 2018133569 A1 WO2018133569 A1 WO 2018133569A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
neuron
pulse
current
current time
Prior art date
Application number
PCT/CN2017/114663
Other languages
English (en)
Chinese (zh)
Inventor
裴京
邓磊
施路平
吴臻志
李国齐
Original Assignee
清华大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学 filed Critical 清华大学
Priority to CN201780084032.6A priority Critical patent/CN110730971B/zh
Publication of WO2018133569A1 publication Critical patent/WO2018133569A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present invention relates to the field of artificial neural network technology, and in particular to a neuron information processing method and system with deep time windowing.
  • the output information of the current pulsed neuron can only affect the pulse neurons connected to the back end in the next time step, ignoring the time domain depth effect between the biological neurons.
  • the front end pulse neuron output information including pulse tip information output by the front end pulse neuron
  • Reading first current pulse neuron information the first current pulse neuron information including a pulse tip information history sequence in a current time window;
  • the pulse sequence information history sequence in the current time window includes: a sequence consisting of output information of each front-end pulse neuron received by N time steps before the current time step, stored in time step order,
  • the pulse tip information of the first time step in the history sequence of the pulse tip information in the current time window is the front end pulse neuron output information received at the first time step before the current time step, the current time window
  • the pulse tip information of the Nth time step in the history sequence of the internal pulse tip information is the front end pulse neuron output information received at the Nth time step before the current time step, where N is a natural number;
  • the pulse tip information update sequence in the current time window including:
  • the pulsed tip information of the updated first time step to the Nth time step constitutes a pulse tip information update sequence in the current time window.
  • the front-end pulse neuron outputs information, and further includes: a connection weight index of the front-end neuron and the current neuron;
  • the current pulsed neuron information further includes: a current time window width, historical membrane potential information, and membrane potential leakage information;
  • the calculating, according to the front-end pulse neuron information and the second current pulse neuron information, the current pulse neuron output information including:
  • the current pulse nerve is output at the Before the step of outputting information, the method further includes:
  • the issuing triggering flag information includes: issuing a trigger or issuing a non-trigger; and when the issuing triggering flag information is issued When triggered,
  • the refractory period timer is reset, and the historical membrane potential information is updated to a preset reset membrane potential information.
  • the method further includes:
  • the issue trigger flag information When the issue trigger flag information is not triggered by the issue, the current time step of the refractory period width and the refractory period timer is read;
  • the refractory period timer is cumulatively counted for one time step, and the historical membrane potential information is updated to be the current pulsed neuron output information.
  • the obtaining a threshold potential includes:
  • the threshold potential is determined based on the threshold random amount and the threshold offset.
  • the outputting the current pulse neuron output information comprises:
  • Reading an issue enable identifier where the issue enable identifier includes allowing data to be issued or not allowing data to be issued; when the issue enable identifier is allowed to issue data,
  • the neuron information processing method with deep time windowing obtains the pulse tip information update sequence in the current time window according to the front end pulse neuron output information and the current tip window pulse tip information history sequence, and is used for current
  • the output information of the pulsed neurons is calculated such that the output information of the current pulsed neuron is correlated with the pulse tip information history sequence in the current time window and the front-end pulse neuron output information received at the current time step. Breaking through the limitation that only the front and back time steps are related to each other, the historical activity information of the cached larger time depth can be flexibly set according to the need, and the biological neuron is closer to the depth effect in the time domain.
  • the front-end pulse neuron output information received in the first N time steps is stored according to the time step sequence, and the information sequence in the historical sequence is sequenced after receiving the front-end pulse neuron output information of the current time step. Move one bit backward, fill the pulse information received by the current time step into the first place, and obtain the updated pulse tip information sequence for the calculation of the current pulse neuron output information. Slide according to time, history pulse tip information and current time step The combination of the pulse tip information enables the output information of the received front-end pulse neurons to be flexibly set according to the required time depth, and the sliding method according to the chronological order is more in line with the biological neuron time depth effect.
  • the threshold potential is determined by reading a random threshold mask potential and a threshold bias and receiving a configuration value given by a configuration register such that the neuron issues pulse tip information with a probability of randomness.
  • the current pulse neuron output information is determined by setting the release enable identifier and the issue trigger flag, so that the output of the pulse neuron is more controllable, and the release enable flag can be configured with neurons that are not configured. It is allowed to issue data, but only as an intermediate auxiliary computing neuron, which is necessary for some functions that require multiple neurons to work together.
  • the invention also provides a neuron information processing system with deep time windowing, comprising:
  • a front-end pulse neuron output information receiving module configured to receive front-end pulse neuron output information, where the front-end pulse neuron output information includes pulse tip information output by the front-end pulse neuron;
  • a first current pulse neuron information reading module configured to read first current pulse neuron information, where the first current pulse neuron information includes a pulse tip information history sequence in a current time window;
  • a current time window in-time pulse tip information update sequence acquisition module configured to acquire a pulse tip information update sequence in a current time window according to pulse tip information output by the front end pulse neuron and a pulse tip information history sequence in the current time window ;
  • a second current pulse neuron information determining module configured to determine second current pulse neuron information according to the current time window intrapulse information update sequence
  • the current pulse neuron output information calculation module is configured to calculate current pulse neuron output information according to the front end pulse neuron information and the second current pulse neuron information;
  • the current pulse neuron output information output module is configured to output the current pulse neuron output information.
  • the pulse sequence information history sequence in the current time window includes: a sequence consisting of output information of each front-end pulse neuron received by N time steps before the current time step, stored in time step order,
  • the pulse tip information of the first time step in the history sequence of the pulse tip information in the current time window is the front end pulse neuron output information received at the first time step before the current time step, the current time window
  • the pulse tip information of the Nth time step in the history sequence of the internal pulse tip information is the front end pulse neuron output information received at the Nth time step before the current time step, where N is a natural number;
  • the pulse tip information update sequence acquisition module in the current time window is used to:
  • the pulsed tip information of the updated first time step to the Nth time step constitutes a pulse tip information update sequence in the current time window.
  • the front-end pulse neuron outputs information, and further includes: a connection weight index of the front-end neuron and the current neuron;
  • the current pulsed neuron information further includes: a current time window width, historical membrane potential information, and membrane potential leakage information;
  • the current pulse neuron output information calculation module includes:
  • a pulsed neuron connection weight reading unit configured to read a connection weight of the front end neuron and the current neuron according to a connection weight index of the front end neuron and the current neuron;
  • a front-end pulse neuron input information calculation unit configured to calculate front-end pulse neuron input information by using an attenuation function according to the current time window width, the pulse tip information update sequence in the current time window;
  • a current pulse neuron output information calculation unit configured to input, according to the front end pulse neuron input information, a connection weight of the front end pulse neuron and a current pulse neuron, the historical membrane potential information, and the membrane potential leakage information,
  • the current pulsed neuron output information is calculated by a pulsed neuron calculation model.
  • the method further includes:
  • a threshold potential acquisition module configured to acquire a threshold potential
  • the issuance triggering flag information determining module is configured to compare the current pulsed neuron output information with the threshold potential, and determine the issuance triggering flag information according to the comparison result, where the issuing triggering flag information includes: issuing a trigger or issuing a trigger; When the issuing trigger flag information is an issue trigger,
  • the refractory timer reset module is configured to reset the refractory period timer and update the historical membrane potential information to a preset reset membrane potential information.
  • the method further includes:
  • the refractory timer reading module is used to read the current time step of the refractory width and the refractory period timer;
  • the refractory period determining module is configured to determine whether the current time is within the refractory period according to the refractory period width and the current time step of the refractory period timer, and if the current time is within the refractory period, The refractory period timer accumulates a time step, does not update the historical membrane potential information; if the current time is not within the due period, accumulates the refractory period timer for one time step, and updates the historical film
  • the potential information is the current pulse neuron output information.
  • the threshold potential acquisition module includes:
  • a threshold information reading unit configured to read a random threshold mask potential, a threshold offset, and a random threshold
  • a random superposition amount acquiring unit configured to perform bitwise AND operation on the random threshold and the random threshold mask potential, Obtain a threshold random stack amount
  • a threshold potential determining unit configured to determine the threshold potential according to the threshold random amount and the threshold offset.
  • the current pulse neuron information output module includes:
  • the identification identifier reading unit is configured to read an issue enable identifier, where the release enable identifier includes allowing data to be released or not allowing data to be issued; and when the issue enable identifier is allowed to issue data,
  • a trigger flag information reading unit configured to read the issue trigger flag information, when the issue trigger flag information is an issue trigger
  • the current pulse neuron information output unit is configured to output the current pulse neuron output information.
  • the neuron information processing system with deep time windowing acquires the pulse tip information update sequence in the current time window according to the front end pulse neuron output information and the current time window pulse tip information history sequence, and is used for current
  • the output information of the pulsed neurons is calculated such that the output information of the current pulsed neuron is correlated with the pulse tip information history sequence in the current time window and the front-end pulse neuron output information received at the current time step. Breaking through the limitation that only the front and back time steps are related to each other, the historical activity information of the cached larger time depth can be flexibly set according to the need, and the biological neuron is closer to the depth effect in the time domain.
  • the front-end pulse neuron output information received in the first N time steps is stored according to the time step sequence, and the information sequence in the historical sequence is sequenced after receiving the front-end pulse neuron output information of the current time step. Move one bit backward, fill the pulse information received by the current time step into the first place, and obtain the updated pulse tip information sequence for the calculation of the current pulse neuron output information. Sliding according to time, combining the historical pulse tip information with the pulse tip information of the current time step, so that the output information of the received front-end pulse neurons can be flexibly set according to requirements, and the sliding is performed according to the time sequence. The method is more in line with the biological neuron time depth effect.
  • the threshold potential is determined by reading a random threshold mask potential and a threshold bias and receiving a configuration value given by a configuration register such that the neuron issues pulse tip information with a probability of randomness.
  • the current pulse neuron output information is determined by setting the release enable identifier and the issue trigger flag, so that the output of the pulse neuron is more controllable, and the release enable flag can be configured with neurons that are not configured. It is allowed to issue data, but only as an intermediate auxiliary computing neuron, which is necessary for some functions that require multiple neurons to work together.
  • FIG. 1 is a schematic flow chart of an adaptive leakage value neural network information processing method according to an embodiment
  • FIG. 2 is a schematic flow chart of an adaptive leak value neural network information processing method according to another embodiment
  • FIG. 3 is a schematic flow chart of an adaptive leakage value neural network information processing method according to still another embodiment
  • FIG. 4 is a schematic flow chart of an adaptive leakage value neural network information processing method according to still another embodiment
  • FIG. 5 is a schematic structural diagram of a pulse tip information history sequence in a current time window in an adaptive leakage value neural network information processing method according to an embodiment
  • FIG. 6 is a schematic structural diagram of an adaptive leakage value neural network information processing system according to another embodiment
  • FIG. 7 is a schematic structural diagram of an adaptive leakage value neural network information processing system according to still another embodiment.
  • FIG. 8 is a schematic structural diagram of an adaptive leakage value neural network information processing system according to still another embodiment.
  • FIG. 1 is a schematic flowchart of an adaptive leakage value neural network information processing method according to an embodiment, and the adaptive leakage value neural network information processing method shown in FIG. 1 includes:
  • Step S100 Receive front end pulse neuron output information, where the front end pulse neuron output information includes pulse tip information output by the front end pulse neuron.
  • the pulse tip information output by the front end pulse neuron is pulse tip information output by the pulse neuron of the front end of the connection with the current pulse neuron.
  • Step S200 reading first current pulse neuron information, the first current pulse neuron information including a pulse tip information history sequence in a current time window.
  • the pulse tip information history sequence in the current time window refers to a sequence of information in which the pulse tip information received in a time range within a certain range in the current time window width is sequentially cached in time series. .
  • Step S300 acquiring a pulse tip information update sequence in the current time window according to the pulse tip information output by the front end pulse neuron and the pulse tip information history sequence in the current time window.
  • the pulse tip information received at the current time step and the pulse tip information received at the time step in the past range are integrated into a new current time window pulse tip information update sequence to be received in the past.
  • the pulse tip information is still involved in the calculation of the pulse neuron output information for the current time step.
  • Step S400 determining second current pulse neuron information according to the pulse tip information update sequence in the current time window.
  • the current pulse neuron information further includes other information
  • the pulse tip information update sequence in the pre-time window acquired after the update is replaced, and the second current pulse neuron is acquired after replacing the pulse tip information history sequence in the current time window. information.
  • Step S500 calculating current pulse neuron output information according to the front end pulse neuron information and the second current pulse neuron information.
  • the front-end pulse neuron information further includes other information, such as a connection weight index of the front-end pulse neuron and the current pulse neuron, and the front-end pulse neuron information and the second current pulse neuron information are calculated.
  • the current pulse neuron output information is obtained.
  • Step S600 outputting the current pulse neuron output information.
  • the neuron information processing method with deep time windowing obtains the pulse tip information update sequence in the current time window according to the front end pulse neuron output information and the current tip window pulse tip information history sequence, and is used for current
  • the output information of the pulsed neurons is calculated such that the output information of the current pulsed neuron is correlated with the pulse tip information history sequence in the current time window and the front-end pulse neuron output information received at the current time step. Breaking through the limitation that only the front and back time steps are related to each other, the historical activity information of the cached larger time depth can be flexibly set according to the need, and the biological neuron is closer to the depth effect in the time domain.
  • FIG. 2 is a schematic flowchart of an adaptive leakage value neural network information processing method according to another embodiment.
  • the adaptive leakage value neural network information processing method shown in FIG. 2 is a detailed step of step S300 in FIG. 1, and includes:
  • Step S310 deleting the pulse tip information of the Nth time step in the history sequence of the pulse tip information in the current time window, and changing the pulse tip information of the first time step to the N-1 time step to the sequence Pulse tip information from the second time step to the Nth time step.
  • the pulse tip information history sequence in the current time window includes: a sequence consisting of outputting information of each front-end pulse neuron received by N time steps before the current time step, stored in a time step order, wherein
  • the pulse tip information of the first time step in the history sequence of the pulse tip information in the current time window is the front end pulse neuron output information received at the first time step before the current time step, and the pulse tip information in the current time window
  • the pulse tip information of the Nth time step in the history sequence is the front end pulse neuron output information received at the Nth time step before the current time step.
  • the pulse tip information history sequence in the current time window stores, from the right hand to the left, the output information of each front-end pulse neuron received by the 15 time steps before the current time step in time step order, since each Each front-end pulse neuron information received by the time step includes a plurality of front-end pulse neuron information according to actual conditions, and is represented by a vertical column in FIG. 5, wherein each circle represents a single front-end pulse neuron output. Pulse tip information.
  • the pulse tip information of the 15th column on the leftmost side in FIG. 5 is deleted, and the pulse tip information of the 1st to 14th columns is sequentially changed to the pulse tip information of the 2nd to 15th columns.
  • Step S320 setting the front end pulse neuron output information received by the current time step as the pulse tip information of the first time step in the pulse tip information history sequence in the current time window.
  • the front end pulse tip information received at the current time step is placed in the first column of FIG.
  • Step S330 the pulse tip information of the updated first time step to the Nth time step is formed into a pulse tip information update sequence in the current time window.
  • the new pulse tip information of the first to the 15th time steps is the pulse tip in the updated current time window.
  • Information update sequence is the pulse tip in the updated current time window.
  • the shift register array can be used to store the pulse tip sequence, or it can be implemented by a common memory. Since the memory can only be read and written according to the address, it cannot be automatically shifted according to the clock signal like the shift register.
  • the bit operation can only be completed through the series of “read ⁇ data splicing ⁇ writing”.
  • the principle of data splicing is: based on the history information and the latest release information currently read, and then intercept the lower 15 bits of the historical release information. The latest release information is spliced and sent to the subsequent calculation module (the input produces the adder) and rewritten back into the spike cache RAM.
  • the row address and write enable signals of the memory are given by the system's read/write control module.
  • the front-end pulse neuron output information received in the first N time steps is stored according to the time step sequence, and after the front-end pulse neuron output information of the current time step is received, the information in the historical sequence is sequentially sequenced. After moving one bit, the pulse information received at the current time step is filled in the first place, and the updated pulse tip information sequence is obtained for the calculation of the current pulse neuron output information. Sliding according to time, combining the historical pulse tip information with the pulse tip information of the current time step, so that the output information of the received front-end pulse neurons can be flexibly set according to requirements, and the sliding is performed according to the time sequence. The method is more in line with the biological neuron time depth effect.
  • FIG. 3 is a schematic flowchart diagram of an adaptive leakage value neural network information processing method according to still another embodiment, and the adaptive leakage value neural network information processing method shown in FIG. 3 includes:
  • Step S100b Receive pulse tip information output by the front end pulse neuron, and a connection weight index of the front end pulse neuron and the current pulse neuron.
  • connection weight index of the front-end pulse neuron and the current pulse neuron is a weight index sent by the front-end neuron together with the output information of the front-end pulse neuron, and is used to indicate the extraction of the current neuron weight.
  • the pulse tip information output by the front end pulse neuron is a pulse tip signal sent by the front end pulse neuron.
  • Step S200b reading the current time window width, the pulse tip information update sequence in the current time window, the historical membrane potential information, and the membrane potential leakage information.
  • the current time window width, the historical film potential information, and the membrane potential leakage information are information in the first current pulse neuron information.
  • Step S300b The connection weight of the front-end neuron and the current neuron is read according to the connection weight index of the front-end neuron and the current neuron.
  • connection weight index of the front-end pulse neuron and the current pulse neuron is an address information
  • the current neuron is indexed according to the received connection weight of the front-end pulse neuron and the current pulse neuron.
  • the connection weight of the front-end pulse neuron and the current pulse neuron is read, and according to the connection weight information, the output information of the front-end neuron can be used in the calculation process of participating in the current neuron output information. More accurately reflects the weight of the output information of the front-end neurons, carrying more abundant information.
  • Step S400b calculating front-end pulse neuron input information by using an attenuation function according to the current time window width and the current time window intra-pulse tip information update sequence.
  • the pulse tip information update sequence in the current time window includes historical pulse tip information of N time steps before the current time step, and uses an attenuation factor K i in the calculation of the current pulse neuron output information.
  • K i the attenuation factor
  • Step S500b calculating, according to the front-end pulse neuron input information, the connection weight of the front-end pulse neuron and the current pulse neuron, the historical membrane potential information, and the membrane potential leakage information, by calculating a pulse neuron calculation model Current pulsed neuron output information.
  • the calculation of the input information of the front-end pulse neuron is represented by the following formula:
  • W ij is the connection weight of the front-end pulse neuron j and the current pulse neuron i
  • T w is the time window width
  • ⁇ j is the front-end neuron j after the spike is issued in the current time window, at the current
  • t is the current time
  • K( ⁇ t) is an attenuation function that decreases rapidly as ⁇ t increases.
  • V SNN f(V+V input +V leak )
  • V is the historical membrane potential information stored in the memory
  • V input is the input of the current beat accumulation
  • V leak is the membrane potential leakage value information
  • the front end pulse neuron input is calculated by the attenuation function according to the current time window pulse tip information update sequence, the current time window width, the connection weight of the front end pulse neuron and the current pulse neuron, and the attenuation function.
  • the information can support the spatio-temporal pulse neural network model with time depth. Compared with the neural network technology scheme with only one time depth, the spatio-temporal information coding ability of the pulse neural network can be greatly improved, and the application space of the pulse neural network is enriched.
  • FIG. 4 is a schematic flowchart of an adaptive leakage value neural network information processing method according to still another embodiment, and the adaptive leakage value neural network information processing method shown in FIG. 4 includes:
  • step S100c the current pulse neuron output information and the threshold potential are calculated.
  • step S200c it is determined whether the current pulse neuron output information is greater than or equal to the threshold potential, and the issuing trigger flag information is determined according to the comparison result, where the issuing trigger flag information includes issuing a trigger or issuing a non-trigger, if When the triggering flag information is issued, the process proceeds to step S300c. When it is determined that the issuance trigger flag information is not triggered, the process proceeds to step S400c.
  • the threshold pulse potential is compared with the current pulse neuron output information, and the issue trigger flag information is determined according to the comparison result.
  • the current pulse neuron output information is transmitted only when the current pulse neuron output information is greater than the threshold potential.
  • Step S300c resetting the refractory period timer, and updating the historical membrane potential information to a preset reset membrane potential information.
  • the issue trigger flag information is an issue trigger
  • the current pulse neuron output information is sent, and after the refractory period timer is reset, the refractory period is recalculated, and the historical membrane potential information is updated.
  • the preset membrane potential information, and the historical membrane potential information is updated, selectively resetting the membrane potential to the current membrane potential, the current membrane potential and the threshold potential difference, or a fixed reset voltage according to the configured reset type.
  • Step S400c reading the current time step of the refractory period width and the refractory period timer.
  • the current pulse neuron output information is not sent, and further determining whether the current period is within the refractory period.
  • the refractory period width is a range of durations of the refractory period, and the refractory period timer is timed by means of a time step.
  • Step S500c determining whether the current time is within the refractory period according to the refractory period width and the current time step of the refractory period timer. If the current time is within the refractory period, proceeding to step S600c, otherwise jumping Go to step S700c.
  • the cumulative calculation of the current time step of the refractory period timer it can be determined whether the current time step is still in the refractory period.
  • Step S600c accumulating the refractory period timer for one time step, and not updating the historical film potential information.
  • the pulsed neurons of the time step need to read the information, that is, during the refractory period, the pulsed neuron output information calculated this time does not participate in the calculation of the next time step.
  • Step S700c accumulating the refractory period timer for one time step, and updating the historical membrane potential information as the current pulse neuron output information.
  • the historical membrane potential information is the current pulse neuron output information, and participates in the calculation of the next time step.
  • the obtaining a threshold potential comprises: reading a random threshold mask potential, a threshold offset, and a random threshold; performing a bitwise AND operation on the random threshold and the random threshold mask potential to obtain a threshold random superposition amount; determining the threshold potential according to the threshold random superimposition amount and the threshold offset.
  • the pseudo random number generator generates a random threshold V rand , and uses the random threshold and the preset random threshold mask potential V mask to perform a bitwise sum operation to generate a threshold random superposition amount, and then randomly superimpose the threshold value. The amount is added to the preset threshold offset Vth0 to produce a true threshold potential Vth .
  • the initial seed of the pseudo random number generator is given by the configuration register V seed .
  • the threshold potential is determined by reading the random threshold mask potential and the threshold offset and receiving the configuration value given by the configuration register, so that the neuron issues the pulse tip information with a certain probability of randomness.
  • the outputting the current pulse neuron output information comprises: reading an issue enable identifier, wherein the issue enable identifier includes allowing data to be issued or not allowing data to be issued; When the data is allowed to be issued, the issuing trigger flag information is read, and when the issuing trigger flag information is an issue trigger, the current pulse neuron output information is output.
  • the release enable identifier and the issue trigger flag by setting the release enable identifier and the issue trigger flag, the current pulse neuron output information is determined, so that the output of the pulse neuron is more controllable, and the release enable flag can be configured with neurons that are not allowed. Issuing data, but only as an intermediate auxiliary computing neuron, is necessary for some functions that require multiple neurons to work together.
  • FIG. 6 is a schematic structural diagram of an adaptive leakage value neural network information processing system according to another embodiment.
  • the adaptive leakage value neural network information processing system shown in FIG. 6 includes:
  • the front-end pulse neuron output information receiving module 100 is configured to receive front-end pulse neuron output information, where the front-end pulse neuron output information includes pulse tip information output by the front-end pulse neuron; and the front-end pulse neuron output information includes : The weight of the connection between the front-end neuron and the current neuron.
  • the first current pulse neuron information reading module 200 is configured to read first current pulse neuron information, where the first current pulse neuron information includes a pulse tip information history sequence in a current time window; and the current time window a sequence of pulse tip information history, comprising: a sequence consisting of output information of each front-end pulse neuron received by N time steps before the current time step stored in a time step sequence, wherein the pulse tip information history sequence in the current time window
  • the pulse tip information of the first time step in the first time step is the front end pulse neuron output information received at the first time step before the current time step, and the Nth time step in the history sequence of the pulse tip information in the current time window
  • the pulse tip information is the front-end pulse neuron output information received at the Nth time step before the current time step; the current pulse neuron information further includes: a current time window width, a historical membrane potential information, and a membrane potential leakage information. .
  • the current time window pulse tip information update sequence acquisition module 300 is configured to acquire a pulse tip in the current time window according to the pulse tip information output by the front end pulse neuron and the pulse tip information history sequence in the current time window.
  • An information update sequence for deleting pulse tip information of the Nth time step in the history sequence of pulse tip information in the current time window, and cutting the pulse tip information from the first time step to the N-1th time step, The sequence is changed to the pulse tip information of the second time step to the Nth time step; the front end pulse neuron output information received by the current time step is set as the first in the history sequence of the pulse tip information in the current time window
  • the pulse tip information of the time step; the pulse tip information of the updated first time step to the Nth time step constitutes a pulse tip information update sequence in the current time window.
  • the second current pulse neuron information determining module 400 is configured to determine second current pulse neuron information according to the current time window intrapulse information update sequence
  • the current pulse neuron output information calculation module 500 is configured to calculate current pulse neuron output information according to the front end pulse neuron information and the second current pulse neuron information;
  • the current pulse neuron output information output module 600 is configured to output the current pulse neuron output information.
  • the method includes: an enable identifier reading unit, configured to read an issue enable identifier, where the issue enable identifier includes allowing data to be issued or not allowed to be issued; and when the issue enable identifier is allowed to issue data, issuing a trigger flag.
  • the information reading unit is configured to read the issue trigger flag information, and when the issue trigger flag information is an issue trigger, the current pulse neuron information output unit is configured to output the current pulse neuron output information.
  • the neuron information processing system with deep time windowing acquires the pulse tip information update sequence in the current time window according to the front end pulse neuron output information and the current time window pulse tip information history sequence, and is used for current
  • the output information of the pulsed neurons is calculated such that the output information of the current pulsed neuron is correlated with the pulse tip information history sequence in the current time window and the front-end pulse neuron output information received at the current time step. Breaking through the limitation that only the front and back time steps are related to each other, the historical activity information of the cached larger time depth can be flexibly set according to the need, and the biological neuron is closer to the depth effect in the time domain.
  • the information calculation module 500 includes:
  • the pulse neuron connection weight reading unit 100b is configured to read the connection weight of the front end neuron and the current neuron according to the connection weight index of the front end neuron and the current neuron.
  • the front end pulse neuron input information calculation unit 200b is configured to calculate front end pulse neuron input information by using an attenuation function according to the current time window width and the current time window intrapulse information update sequence.
  • the current pulse neuron output information calculation unit 300b is configured to input information according to the front end pulse neuron, a connection weight of the front end pulse neuron and the current pulse neuron, the historical membrane potential information, and the membrane potential leakage information.
  • the current pulsed neuron output information is calculated by a pulsed neuron calculation model.
  • the pulse tip information update sequence, the current time window width, the connection weight of the front-end pulse neuron and the current pulse neuron, and the front-end pulse neuron input information are calculated by the attenuation function. It can support the spatio-temporal pulse neural network model with time depth. Compared with the neural network technology scheme with only one time depth, it can greatly improve the spatio-temporal information coding ability of the pulse neural network and enrich the application space of the pulse neural network.
  • FIG. 8 is a schematic structural diagram of an adaptive leakage value neural network information processing system according to still another embodiment.
  • the adaptive leakage value neural network information processing system shown in FIG. 8 includes:
  • the threshold potential acquisition module 700 is configured to acquire a threshold potential, and includes: a threshold information reading unit, configured to read a random threshold mask potential, a threshold offset, and a random threshold; and a random superposition amount acquiring unit configured to use the random threshold And performing a bitwise AND operation with the random threshold mask potential to obtain a threshold random superposition amount; and a threshold potential determining unit configured to determine the threshold potential according to the threshold random superposition amount and the threshold offset.
  • the issue trigger flag information determining module 800 is configured to compare the current pulse neuron output information with the threshold potential, and determine the issue trigger flag information according to the comparison result, where the issue trigger flag information includes: issuing a trigger or issuing a trigger When the issue trigger flag information is an issue trigger.
  • the refractory timer reset module 900 is configured to reset the refractory period timer and update the historical membrane potential information to a preset reset membrane potential information.
  • the refractory timer reads module 1000 for reading the current time step of the refractory period width and the refractory period timer.
  • the refractory period determining module 1100 is configured to determine, according to the refractory period width and the current time step of the refractory period timer, whether the current time is within the refractory period, and if the current time is within the refractory period, And accumulating the refractory period timer for one time step, not updating the historical membrane potential information; if the current time is not within the due period, accumulating the refractory period timer for one time step, and updating the history
  • the membrane potential information is the current pulsed neuron output information.
  • the threshold potential is determined by reading the random threshold mask potential and the threshold offset and receiving the configuration value given by the configuration register, so that the neuron issues the pulse tip information with a certain probability of randomness.
  • the release enable flag and issuing the trigger flag the current pulse neuron output information is determined, so that the output of the pulse neuron is more controllable, and the release enable flag can be configured to allow the neuron to not issue data, but only As an intermediate aided computational neuron, this is necessary for some functions that require multiple neurons to work together.
  • an embodiment of the present invention further provides a computer device including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer
  • a computer device including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer
  • Non-volatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM is available in a variety of formats, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization chain.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDRSDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • Synchlink DRAM SLDRAM
  • Memory Bus Radbus
  • RDRAM Direct RAM
  • DRAM Direct Memory Bus Dynamic RAM
  • RDRAM Memory Bus Dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Theoretical Computer Science (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Neurology (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Feedback Control In General (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)
  • Medical Treatment And Welfare Office Work (AREA)

Abstract

La présente invention concerne un procédé et un système de traitement d'informations neuronales ayant un fenêtrage temporel profond. Le procédé consiste à : recevoir des informations de sortie de neurone impulsionnel d'extrémité, les informations de sortie de neurone impulsionnel d'extrémité comprenant des informations de pointe d'impulsion délivrées par un neurone impulsionnel d'extrémité (S100) ; lire les premières informations de neurone impulsionnel actuelles, les premières informations de neurone impulsionnel actuelles comprenant une séquence historique d'informations de pointe d'impulsion dans la fenêtre temporelle actuelle (S200) ; acquérir une séquence mise à jour d'informations de pointe d'impulsion dans la fenêtre temporelle actuelle selon les informations de pointe d'impulsion délivrées par le neurone impulsionnel d'extrémité et selon la séquence historique d'informations de pointe d'impulsion dans la fenêtre temporelle actuelle (S300) ; déterminer des secondes informations de neurone impulsionnel actuelles selon la séquence mise à jour d'informations de pointe d'impulsion dans la fenêtre temporelle actuelle (S400) ; calculer les informations de sortie de neurone impulsionnel actuelles selon les informations de neurone impulsionnel d'extrémité et les secondes informations de neurone impulsionnel actuelles (S500) ; et délivrer les informations de sortie de neurone impulsionnel actuelles (S600). Le procédé met fin à la restriction selon laquelle seules des étapes temporelles adjacentes sont en relation, peut définir de manière souple des informations d'activités historiques pour cacher une plus grande profondeur de temps selon des exigences, et est plus proche d'un neurone biologique quant à l'aspect de l'effet de profondeur du domaine temporel.
PCT/CN2017/114663 2017-01-20 2017-12-05 Procédé et système de traitement d'informations neuronales ayant un fenêtrage temporel profond WO2018133569A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201780084032.6A CN110730971B (zh) 2017-01-20 2017-12-05 具有深度时间划窗的神经元信息处理方法和系统

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710041894.2 2017-01-20
CN201710041894.2A CN106897769A (zh) 2017-01-20 2017-01-20 具有深度时间划窗的神经元信息处理方法和系统

Publications (1)

Publication Number Publication Date
WO2018133569A1 true WO2018133569A1 (fr) 2018-07-26

Family

ID=59197968

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/114663 WO2018133569A1 (fr) 2017-01-20 2017-12-05 Procédé et système de traitement d'informations neuronales ayant un fenêtrage temporel profond

Country Status (2)

Country Link
CN (2) CN106897769A (fr)
WO (1) WO2018133569A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190102669A1 (en) * 2017-09-29 2019-04-04 Intel Corporation Global and local time-step determination schemes for neural networks
KR20200029661A (ko) * 2018-09-07 2020-03-19 삼성전자주식회사 뉴럴 프로세싱 시스템
CN110059812B (zh) * 2019-01-26 2021-09-14 中国科学院计算技术研究所 脉冲神经网络运算芯片及相关运算方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1262912A2 (fr) * 2001-05-31 2002-12-04 Canon Kabushiki Kaisha Circuit de traitement de signal avec comportement local synchronisé
CN101052979A (zh) * 2004-03-17 2007-10-10 佳能株式会社 并行脉冲信号处理设备、模式识别设备和图像输入设备
CN103279958A (zh) * 2013-05-31 2013-09-04 电子科技大学 一种基于Spiking神经网络的图像分割方法
CN103679265A (zh) * 2013-11-21 2014-03-26 大连海联自动控制有限公司 一种多特性仿生统一神经元模型

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001013991A1 (fr) * 1999-08-26 2001-03-01 Med-El Elektromedizinische Geräte GmbH Stimulation nerveuse electrique fondee sur des sequences d'echantillonnage specifiques a une voie
US9147156B2 (en) * 2011-09-21 2015-09-29 Qualcomm Technologies Inc. Apparatus and methods for synaptic update in a pulse-coded network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1262912A2 (fr) * 2001-05-31 2002-12-04 Canon Kabushiki Kaisha Circuit de traitement de signal avec comportement local synchronisé
CN101052979A (zh) * 2004-03-17 2007-10-10 佳能株式会社 并行脉冲信号处理设备、模式识别设备和图像输入设备
CN103279958A (zh) * 2013-05-31 2013-09-04 电子科技大学 一种基于Spiking神经网络的图像分割方法
CN103679265A (zh) * 2013-11-21 2014-03-26 大连海联自动控制有限公司 一种多特性仿生统一神经元模型

Also Published As

Publication number Publication date
CN106897769A (zh) 2017-06-27
CN110730971B (zh) 2023-11-28
CN110730971A (zh) 2020-01-24

Similar Documents

Publication Publication Date Title
WO2018133568A1 (fr) Procédé et système de traitement d'informations neuronales en mode composé, et dispositif informatique
US11710571B2 (en) Long short-term memory model-based disease prediction method and apparatus, and computer device
CN106875003B (zh) 自适应泄漏值神经元信息处理方法和系统
WO2018133569A1 (fr) Procédé et système de traitement d'informations neuronales ayant un fenêtrage temporel profond
Alejo et al. Tests for normality in linear panel-data models
Jin et al. Implementing spike-timing-dependent plasticity on SpiNNaker neuromorphic hardware
US11551074B2 (en) Self-adaptive threshold neuron information processing method, self-adaptive leakage value neuron information processing method, system computer device and readable storage medium
CN106875005B (zh) 自适应阈值神经元信息处理方法和系统
CN114676795B (zh) 一种深度学习模型的训练方法、装置、设备及存储介质
CN108229677A (zh) 用于使用循环模型执行识别和训练循环模型的方法和设备
CN108229714A (zh) 预测模型构建方法、门诊量预测方法及装置
KR20160068823A (ko) 스파이킹 신경세포들의 망들에서의 정체 회피
Bryer TriMatch: An R package for propensity score matching of non-binary treatments
CN118280450A (zh) 基于双通道异构网络特征表示策略的microRNA-药物敏感性预测方法及设备
CN116864143A (zh) 基于人工智能的传染病数据分析方法、装置及相关设备
CN106933834A (zh) 一种数据匹配方法及装置
CN106815638B (zh) 输入权重拓展的神经元信息处理方法和系统
CN119452373A (zh) 校正机器学习模型训练数据
CN111738407B (zh) 一种基于深度学习的钟差预测方法、装置、介质及终端
WO2018133567A1 (fr) Procédé et système de traitement d'informations de poids de neurone, procédé et système de traitement d'informations neuronales, et dispositif informatique
Roman et al. Modeling cross correlations within a many-assets market
Castle et al. A Novel Approach to Forecasting UK Inflation over 2021-2024
CN118734975B (zh) 大语言模型训练方法、装置、设备及介质
WO2018137411A1 (fr) Procédé et système de conversion d'informations de réseau neuronal, et dispositif informatique
Nance et al. Applying the causal roadmap to longitudinal national Danish registry data: a case study of second-line diabetes medication and dementia

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17893167

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17893167

Country of ref document: EP

Kind code of ref document: A1

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载