JP2025514007A - Neural network circuits and devices - Google Patents

Abstract

The present invention relates to a neural network circuit and device.
[Solution] In one embodiment, a neural network circuit includes a synapse memory cell arranged along an output line and including a resistive memory element having one of a first resistance value and a second resistance value, and in response to receiving an input signal via an input line, generates a column signal based on the resistive memory element and the input signal, a reference memory cell arranged along a reference line and including a reference memory element having the first resistance value or the second resistance value in a ratio predetermined by an application, and generates a reference signal based on the reference memory element and the input signal, and an output circuit that generates an output signal for the output line from the column signal and the reference signal.
[Selected Figure] Figure 1

Description

The following disclosure relates to neural network circuits and devices.

Currently, in von Neumann computer architectures, the frequent movement of huge amounts of data between the processor and memory causes long delays and high power consumption, limiting the performance of the chip. In machine learning applications, for example, the movement of data between layers of a neural network is a major aspect of the computational load. Considering the computational cost when implementing machine learning in von Neumann architectures, current software-based deep neural network operations can use artificial intelligence (AI) accelerator hardware. Currently, software-based deep neural network operations use AI accelerator hardware such as high-performance CPUs, GPUs, and ASICs.

Neuromorphic architectures can perform calculations immediately at the location of the memory element where data is stored, and store and update the connection strengths between neuronal circuits (e.g., synaptic weights) in the memory element. Neuromorphic calculation methods can be applied to artificial intelligence, big data, sensor networks, pattern/object recognition, etc. Neuromorphic architectures can be realized as hardware using analog memory.

The neural network circuit of one embodiment includes a synapse memory cell arranged along an output line, including a resistive memory element having one of a first resistance value and a second resistance value, and generating a column signal based on the resistive memory element and the input signal in response to receiving an input signal via an input line; a reference memory cell arranged along a reference line, including a reference memory element having the first resistance value or the second resistance value in a ratio predetermined by an application, and generating a reference signal based on the reference memory element and the input signal; and an output circuit that generates an output signal for the output line from the column signal and the reference signal.

The synapse memory cell includes a resistive memory element having a bit number for expressing a synapse weight value assigned to the synapse memory cell, and the resistive memory elements having the bit number can be arranged along the same input line.

The reference memory cell includes reference memory elements having a number of bits for representing the synapse weight value, and the reference memory elements having the number of bits can be arranged along the same input line.

The reference memory cell may include a reference memory element for representing a code bit.

The output circuit can generate, as the output signal, a current corresponding to a positive integer multiple of a net current that is the difference between a first current based on the resistive memory element having the first resistance value and a second current based on the resistive memory element having the second resistance value.

The output circuit can generate, as the output signal, a current corresponding to a negative integer multiple of a net current that is the difference between a first current based on a resistive memory element having the first resistance value and a second current based on a resistive memory element having the second resistance value.

The output circuit may further include a leaky integration and fire circuit that fires other neuron circuits based on the result of comparing the output signal with a threshold.

The LIF circuit can increase the output voltage in response to the output signal corresponding to a positive integer multiple of the net current.

The LIF circuit can reduce the output voltage in response to the output signal corresponding to a negative integer multiple of the net current.

The LIF circuit can leak the output signal in response to a voltage accumulating based on the output signal not reaching a threshold voltage within a critical time.

The LIF circuit can fire the other neuron circuit in response to a voltage accumulated based on the output signal reaching a threshold voltage within a critical time corresponding to the threshold.

The neural network circuit according to one embodiment may further include a threshold memory array including a plurality of memory elements, in which at least one memory element selected from the plurality of memory elements based on the set threshold has the first resistance value.

In one embodiment, the neural network circuit may further include an additional reference memory cell that shares a reference word line with the threshold generation circuit, is disposed along the reference word line, and has an additional reference memory element having the second resistance value.

The output circuit can set a critical time corresponding to the threshold based on a signal generated based on the threshold memory cell and a signal generated based on the additional reference memory cell.

The output circuit can begin accumulating a current corresponding to the difference between the signal generated based on the threshold memory cell and the signal generated based on the additional reference memory cell, and output a signal indicating a threshold time corresponding to the threshold when a voltage corresponding to the accumulated current exceeds a threshold voltage.

The output circuit can apply a critical time corresponding to the threshold determined based on the threshold memory cell and the additional reference memory cell to the output signal for the output line and other output signals for other output lines.

The resistive memory elements of synapse memory cells connected to the same output line can be connected in parallel with each other.

In one embodiment, the neural network circuit further includes a different synapse memory cell arranged along an output line different from the output line, and the output circuit can generate output signals individually for each of the output line and the different output line using the same reference memory cell.

The output circuit may include a read circuit that generates a column integrated signal by accumulating the column signals of the synapse memory cells for each bit, and generates a reference integrated signal by accumulating the reference signals of the reference memory cells for each bit.

The read circuit may include a current mirror that copies the column signal for each bit of the synapse memory cell and the reference memory cell with a current multiple corresponding to that bit.

The output circuit can generate the output signal corresponding to the difference between the column accumulation signal and the reference accumulation signal.

The output circuit may include a capacitor through which a current corresponding to the difference between the column accumulation signal and the reference accumulation signal flows by causing a current corresponding to the reference accumulation signal to flow in toward a node and a current corresponding to the column accumulation signal to flow out of the node.

The output circuit can obtain a multiply and accumulate (MAC) between the input signal received along the input line and the synaptic weight value based on the result of interpreting the output signal, and transmit a node value determined based on the obtained multiply and accumulate (MAC) to another neuron circuit.

An operating method of a neural network circuit according to one embodiment includes the steps of: generating a column signal based on a resistive memory element of a synapse memory cell to which an input signal is applied via an input line among one or more memory cells arranged along an output line, and the input signal; generating a reference signal based on a reference memory element having a reference resistance value of a reference memory cell to which the input signal is applied among one or more memory cells arranged along a reference line, and the input signal; and generating an output signal for the output line from the column signal and the reference signal, and the reference resistance value can be determined based on a predetermined combination of a first resistance value and a second resistance value different from the first resistance value according to an application.

1 illustrates a neural network circuit according to one embodiment. 1 illustrates a synapse memory cell and readout circuit according to an embodiment. 1 illustrates a Leaky Integration and Fire (LIF) circuit for an output circuit according to one embodiment. 1 illustrates a threshold generation circuit and an additional reference circuit included in a neural network circuit according to an embodiment. 1 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to one embodiment. 1 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to one embodiment. 1 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to one embodiment. 10 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to another embodiment. 10 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to another embodiment. 10 illustrates an example of a firing comparison using a net signal generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to another embodiment. 4 illustrates a critical time generation circuit of an output circuit according to an embodiment. 1 shows elements included in an LIF circuit in an output circuit according to an embodiment, and a timing diagram for each element. An example of a firing operation of a neural network circuit according to an embodiment will be described. 4 is a flowchart illustrating a method of operating a neural network circuit according to an embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified in various forms. Therefore, the embodiments are not limited to the specific disclosed forms, and the scope of this specification includes modifications, equivalents, or alternatives within the technical concept.

Although terms such as first or second may be used to describe multiple components, such terms should be construed only for the purpose of distinguishing one component from the other components. For example, a first component may be named a second component, and similarly, a second component may be named a first component.

When a component is referred to as being "connected" to another component, it is understood that it is directly connected or connected to the other component, but that there may be other components in between.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "include" or "have" indicate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Terms commonly used and predefined should be interpreted to have a meaning consistent with the meaning they have in the context of the relevant art, and should not be interpreted as having an ideal or overly formal meaning unless expressly defined in this specification.

Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. When describing the embodiments with reference to the drawings, the same components are given the same reference numerals regardless of the drawing numerals, and duplicate descriptions thereof will be omitted.

Figure 1 shows a neural network circuit according to one embodiment.

The neural network circuit 100 according to one embodiment can be realized as a circuit that replicates synapses connecting neuron circuits of a previous layer and neuron circuits of a target layer in a neural network. An electronic device including the neural network circuit 100 may be referred to as a neural network device. The neuron circuit may include a circuit that realizes activation of neurons and/or activation functions of a neural network. The neural network includes a number of layers each having a number of nodes, and the previous layer represents a previously connected layer of the target layer in the neural network. The nodes of the neural network correspond to the neuron circuits of the neural network circuit 100. The neural network circuit 100 can transmit node values (e.g., input values) output from the neuron circuits of the previous layer to the neuron circuits of the target layer based on the connection strength (e.g., synapse weight value) between the corresponding neuron circuits. A target neuron circuit of a target layer may have, as a node value, a result of applying an activation function to a weighted sum based on a node value and a synaptic weight value received from a previous neuron circuit of a previous layer connected to the target neuron circuit. In this specification, a neuron circuit may be realized in the output circuit 150, and as described below, a neuron circuit of a neural network circuit 100 that realizes a spiking neural network may include a leaky integration and fire (LIF) circuit. However, without being limited thereto, the activation function of a neural network may be realized by a circuit designed according to another method. Here, the activation function or the activation value from the result of the activation function may be distinguished according to whether a line or an operator is in an active state or an inactive state, for example, whether a line is activated or inactivated. However, for example, if a non-zero activation value output from a previous layer is applied to an input line, the input line is considered active or activated, and conversely, if no zero activation value or signal is applied, the line is considered inactive or deactivated.

The neural network circuit 100 includes a synapse memory array 110, a reference memory array 120, and an output circuit 150. The synapse memory array 110 and the reference memory array 120 may be referred to as memory arrays.

The synapse memory array 110 includes a plurality of synapse memory cells 111 arranged along input lines and output lines. The plurality of synapse memory cells 111 may be arranged in a crossbar array configuration. The input lines are shown in FIG. 1 as K word lines WL0 to WLK-1 that receive inputs. K may be an integer equal to or greater than 1. The signals from each word line indicate "0" or "1", which are the outputs of the K nodes of the previous layer. The output line may output an output signal 159 indicating a sum of the synapse weight value indicated by the resistive memory element of the synapse memory cell 111 arranged along the output line and the operation result (e.g., the product result) between the input values indicated by each input signal (e.g., the result of multiplication and accumulation (MAC) value, or weighted sum). The output lines are illustrated as J lines in FIG. 1, where J may be an integer equal to or greater than 1. Thus, the synapse memory array 110 may include K×J synapse memory cells 111. Given a synapse memory cell 111 in a jth cell column (j is 1 to J) and a kth cell row (k is 1 to K), the value stored in the given synapse memory cell 111 is a synapse weight value between a jth node of a previous layer and a kth node of a current/next layer. In other words, the neural network circuit 100 illustrated in FIG. 1 may be interpreted as a circuit in which synaptic connections connecting K previous nodes and J next nodes are embodied. If the synapse weight value is expressed as a bits, each output line may include a bit lines. Here, a may be an integer equal to or greater than 1. An output signal 159 output from any one output line is a difference signal between a signal obtained by accumulating column signals (e.g., bit column signals) generated by the bit lines included in the output line and a signal obtained by accumulating a reference signal generated by a reference line.

The input line may receive an input signal via a pre-synaptic circuit. The pre-synaptic circuit may receive the output of a previous node (e.g., a previous neuron circuit) of the neural network and transmit it to a synaptic memory element of the current node. The pre-synaptic circuit may also be referred to as an axon circuit.

The output line is connected to a post-synaptic circuit and transmits a signal corresponding to the sum of products between the synaptic weights of the synaptic memory elements connected to the output line and the input signal to the post-synaptic circuit. The post-synaptic circuit may fire or transmit an output signal 159 corresponding to the sum of products between the synaptic weights and the input signal to the next node (e.g., the next synaptic circuit). The post-synaptic circuit may be represented as a dendrites circuit, and may be realized, for example, as a LIF circuit described below.

The input signal indicates a signal received through an input line. In this specification, when the neural network circuit 100 receives an input signal corresponding to a bit value "1" from any one of the input lines, an input voltage may be applied to the input line. As a different example, when the input signal of any one of the input lines indicates a bit value "0", the neural network circuit 100 may inactivate the input line. In other words, the neural network circuit 100 may apply no voltage or a voltage of 0V to the input line. However, the input voltage applied for each bit value indicated by the input signal is not limited to the above. In the example shown in FIG. 1, among the K input lines, an activation value (e.g., a voltage corresponding to a bit value "1") may be applied to k input lines. k is an integer between 0 and K. That is, for a given input, some input lines become activated "1" and other lines become inactivated "0".

The synapse memory cell 111 may include a resistive memory element arranged along an output line and having one of a first resistance value and a second resistance value. The synapse memory cell 111 may include a resistive memory element having a bit number for expressing a synapse weight value assigned to the synapse memory cell 111. The resistive memory elements having the above-mentioned bit number may be arranged along the same input line.

In response to receiving an input signal via an input line, the synapse memory cell 111 may generate a column signal based on the resistive memory element and the input signal. The first resistance value and the second resistance value may each be mapped to a binary coded value. For example, the first resistance value and the second resistance value may each indicate a bit value of "0" or "1". Exemplarily, the first resistance value being a low resistance indicates a bit value of "0", and the second resistance value being a high resistance indicates a bit value of "1". However, without being limited thereto, the bit values mapped to the first resistance value and the second resistance value may vary depending on the design.

When the synapse weight value is expressed as a bit sequence with multiple bits, the synapse memory cell 111 may include a number of subcells. For example, the synapse weight value may be expressed as a bit sequence of a bits. For a synapse weight value of a bits, the synapse memory cell 111 may include a number of subcells. The first subcell may have a resistance value of a bit value corresponding to a least significant bit (LSB) digit of the bit sequence indicating the synapse weight value, and the a-th subcell may have a resistance value of a bit value corresponding to a most significant bit (MSB) digit. Each of the a subcells may include a resistive memory element having a resistance value corresponding to a bit value of a corresponding bit digit of the synapse weight value. The resistive memory element included in each of the a subcells may be set to have a resistance value mapped to the bit value of the corresponding bit digit. It may be set to have one of a first resistance value and a second resistance value. FIG. 2 below illustrates an example in which the synapse memory cell 111 is implemented as a 3-bit cell.

The column signal is a signal obtained by accumulating signals output from the synapse memory cells 111 arranged along any one of the column lines in the synapse memory array 110. For example, the column signal may be a signal corresponding to the sum of products between the bit values corresponding to the resistive memory elements of the subcells connected to the same bit line in the synapse memory array 110 and the input value of the input signal. In other words, the column signal of any one of the bit lines may be a signal corresponding to the sum of products of the bit digits indicated by the bit line in the output line. As will be described later, the neural network circuit 100 can obtain a signal obtained by multiplying each of the column signals of the bit lines included in the same output line by a weighting value (hereinafter, "bit weighting value") corresponding to the bit digit of the bit line. For example, the bit weighting value corresponding to the LSB may be 1/4, the bit weighting value corresponding to the first bit digit from the LSB may be 1/2, and the bit weighting value corresponding to the second bit digit from the LSB may be 1. However, this is designed assuming that the bit weight value of the second bit digit from the LSB, which is the MSB, is 1, and the bit weight value can be changed according to the design rather than being input thereto. For convenience of explanation, columns are shown mainly as vertical lines in this specification, but are not limited to this and the shape of the column lines can be changed according to the design.

As described above, the resistive memory element is an element having one of a first resistance value and a second resistance value. For example, the resistive memory element may be an MRAM (magnetic ram). The MRAM may have one of two states, MTJ (magnetic tunnel junction). The MRAM in the P state (parallel state) may have a first resistance value, and the MRAM in the AP state (anti-parallel state) may have a second resistance value. The second resistance value may be greater than the first resistance value. The second resistance value may be referred to as a high resistance value, and the first resistance value may be referred to as a low resistance value. The AP state may be referred to as a high resistance state, and the P state may be referred to as a low resistance state. The resistance ratio of the first resistance value to the second resistance value of the MRAM may be 2. However, the resistance ratio of the resistive memory element is not limited to the above, and the resistance ratio may vary depending on the type of element. The neural network circuit 100 according to an embodiment may generate a net signal using a reference memory array 120 described below in order to distinguish signals output according to the first resistance value and the second resistance value of the resistive memory element.

For reference, for convenience of explanation in this specification, resistive memory elements in the P state and AP state are mainly described, but are not limited thereto. Unless contrary to the explanation, the P state may be interpreted as a state having a first resistance value, and the AP state may be interpreted as a state having a second resistance value.

The reference memory array 120 includes a plurality of reference memory cells 121 arranged along the reference lines. As an example shown in FIG. 1, the reference memory cells 121 included in the reference memory array 120 may be arranged along the reference lines. The reference memory array 120 may include the same number of reference memory cells 121 as the number of input lines.

The reference memory cell 121 may include a reference memory element arranged along a reference line and having a programmable resistance value according to an application. Here, having a "programmable" resistance value means that the resistance value can be dynamically changed. The reference memory cell 121 may include a reference memory element with a number of bits for expressing a synapse weight value. The reference memory elements with the above-mentioned number of bits may be arranged along the same input line.

Each reference memory cell 121 according to an embodiment may include as many reference sub-cells as the number of bits, in other words, as many resistive memory elements as the number of bits, similar to the synapse memory cell 111 described above. Here, the reference memory cell 121 may include a resistive memory element set to a predetermined resistance value among the first resistance value and the second resistance value.

In other embodiments, each reference memory cell 121 may be a one-bit cell that stores a single bit of information (e.g., a sign bit), regardless of the number of bits in the synapse memory cell 111.

The reference memory cells 121 may generate a reference signal based on the reference memory elements and the input signal. The reference signal is an accumulated signal of signals output from the reference memory cells 121 arranged along the reference lines in the reference memory array 120. The reference lines may also include reference bit lines as many as the number of bits. For example, the reference signal is a signal corresponding to the sum of products between the bit values corresponding to the resistive memory elements of the reference sub-cells connected to the same reference bit line in the reference memory array 120 and the input value of the input signal. In other words, the reference signal of any one of the reference bit lines may be a signal corresponding to the sum of products of the bit digits indicated by the reference bit line in the reference line.

The output circuit 150 generates an output signal 159 for the output line from the column signal and the reference signal. For reference, the neural network circuit 100 may further include different synapse memory cells arranged along different output lines from the output line. In this case, the output circuit 150 can generate output signals separately for each of the output line and the different output lines using the same reference memory cells. In other words, the output circuit 150 can use the same reference column signal to generate different output signals for each output line. The operation of the output circuit 150 will be described later.

The neural network circuit 100 according to an embodiment integrates all signals (e.g., current signals) generated for each column when input to K word lines for accessing each synapse memory element during synapse operation. The output lines include bit lines corresponding to bit digits, and the column accumulation signal I _Cells output from the output lines may be a signal obtained by integrating a signal to which a bit weight is applied to the column signals of the individual bit lines. The column accumulation signal is expressed as the following Equation (1).

In the above formula (1), I _P represents a current flowing through a resistive memory element in a P state (e.g., a first resistance value), and I _AP represents a current flowing through a resistive memory element in an AP state (e.g., a second resistance value). The column accumulation signal I _Cells is a linear combination of I _P and I _AP . The synapse readout circuit 151 of the output circuit 150 generates the column accumulation signal I _Cells . The reference accumulation signal I _REF of the reference memory array 120 can be calculated as shown in formula (2).

The above-mentioned Equation (2) is a signal in which a bit weighting value for each bit digit is applied to a current signal flowing through a resistive memory element to which an input signal is applied among resistive memory elements included in the reference memory array 120. The reference accumulation signal _IREF is a signal in which a bit weighting value is applied to a reference signal for each bit digit and integrated. Here, X may be determined to be any one of an integer smaller than or equal to N and larger than or equal to 0 depending on an application. When the reference memory cell 121 is a 1-bit cell, X may have a value of 0 or N.

A reference read circuit 152 of the output circuit 150 generates a reference accumulation signal I _REF . In the above-mentioned formulas (1) and (2), coefficients N and M of the linear combination are expressed as the following formulas (3) and (4).

N indicates a value determined based on the subcells connected to the activated word line. k indicates the number of activated word lines (e.g., input lines) as described above. a indicates the number of subcells included in each memory cell, and the number of bits of the synapse weight value. For example, N is the sum of powers of 2 with the bit digits of each subcell included in the synapse memory cells 111 connected to the activated word line among the synapse memory cells 111 connected to any one output line as an exponent. M is an integer equal to or less than N, and is a value determined based on the subcells in the P state (e.g., the first resistance value) among the synapse memory cells 111 connected to the activated word line. m _ij is 1 when the i-th synapse memory cell of the activated j-th word line has the first resistance value, and is 0 when the i-th synapse memory cell has the second resistance value. For example, M is the sum of k powers of 2, the exponent being the bit digit indicated by the P-state subcell among the synapse memory cells 111 connected to any one output line and the synapse memory cells 111 connected to the activated word line. In summary, N is proportional to the number of bits of the activated input line, and M is proportional to the number of bits of the activated input line in the P state (e.g., "1").

The output signal 159 _{I net,column} , ie, the net signal of the integrated column accumulated signal I _Cells and the integrated reference accumulated signal I _REF, is calculated with reference to Equation (5).

The output circuit 150 of the neural network circuit 100 according to an embodiment of the present invention may generate an output signal 159 I _net,column that corresponds to the difference between the column accumulated signal I _Cells and the reference accumulated signal I _REF according to Equation (5) above.

The difference (I _P -I _AP ) between the first current I _P based on the resistive memory element having the first resistance value and the second current I _AP based on the resistive memory element having the second resistance value may be referred to as a valid signal or a net signal. Alternatively, since I _P -I _AP is a current-type signal, it may be referred to as a net current. Here, the output signal 159 _{I net,column} corresponds to (M -X) times the net signal, where (M -X) may be a positive integer or a negative integer depending on the value of X.

In this specification, the neural network circuit 100 is mainly described as a spiking neural network (SNN) that realizes a function of simulating a biological neural network that accumulates current according to synaptic weights, but is not limited thereto. The neural network circuit 100 according to one embodiment may be applied to a system that realizes a multiply-accumulate (MAC) operation, such as a computation in memory (CIM) circuit using a memory with a low resistance ratio and a vector matrix multiplication (VMM) circuit. Even if the resistance ratio between the low resistance and the high resistance of the resistive memory element included in the neural network circuit 100 is not large, the output value can be distinguished through the above-mentioned net signal. In the neural network circuit 100 according to an embodiment, even if a resistive memory element is used in which the resistance ratio of the first resistance value to the second resistance value exceeds 1, the output can be distinguished via a net signal. Therefore, constraints on the design for detecting and summing the currents output from the memory array are reduced.

Thus, the neural network circuit 100 according to one embodiment can have a memory array of increased size by offsetting the high resistance state. Furthermore, the neural network circuit 100 can be applied to artificial intelligence processors that handle large amounts of data.

Furthermore, as will be described in detail below, the neural network circuit 100 according to an embodiment can realize two typical reactions of biological neural networks, an EPSP (Excitatory Post Synaptic Potential) in which the potential increases due to an input stimulus, and an IPSP (Inhibitory Post Synaptic Potential) in which the potential decreases, in one synapse memory array 110. More specifically, the neural network circuit 100 can realize an ESPS that increases the voltage in response to a stimulus when the output signal 159I _{net, column} is proportional to a positive multiple of the net signal, _and an IPSP that reduces the voltage in response to a stimulus when the output signal 159I net, column is proportional to a negative multiple of the net signal.

For reference, the present specification mainly describes an example in which the output circuit 150 includes an LIF circuit as an activation function circuit, but is not limited thereto. For example, the neural network circuit 100 may include an analog-to-digital converter instead of the LIF circuit to convert the aforementioned output signal from an analog signal to a digital value (e.g., output data). An electronic device including the neural network circuit 100 can apply an activation function to the output data to determine a value to be propagated to the next node. In other words, the electronic device can also perform a calculation corresponding to the activation function at a digital level.

Figure 2 shows a synapse memory cell and readout circuit according to one embodiment.

The output circuit of the neural network circuit according to one embodiment includes a read circuit 251 and an LIF circuit connected to the synapse memory array 210. Although omitted in FIG. 2, the output circuit may also include a reference read circuit 251 connected to a reference memory array. Although FIG. 2 mainly describes the synapse read circuit 251, the reference read circuit 251 may also be configured in the same manner as the synapse read circuit 251.

As shown in FIG. 2, the resistive memory elements of the synapse memory cells 211 connected to the same output line may be connected in parallel to each other. For example, among the synapse memory cells 211 connected to the same output line, the resistive memory elements connected to the same bit line may be connected in parallel to each other. Thus, the current signals output from the resistive memory elements of the same bit line may be integrated along the output line. In FIG. 2, the resistance of the resistive memory elements is indicated as R _DATA . As described above, among the resistive memory elements arranged along the bit lines BL0 to BL2, the resistive memory elements to which the input signal is applied via the word lines WL ₀ to WL _K-1 may generate a current signal according to the voltage applied to the resistive memory elements. When the input signal is applied along the word lines WL ₀ to WL _K-1 , the switch SW _data is turned on.

The switch SW _sense (e.g., a transistor) can connect the resistive memory element to a sensing line. When a Sel_data signal is applied, the switch SW _sense (e.g., a transistor) is turned on, and the bit lines BL0 to BL2 and the sensing line SL are connected to a supply voltage and ground, respectively.

The operational amplifier OP has a high advantage in that it can fix the voltage of the bit lines BL0-BL2 to V _{amp_ref} regardless of the resistance seen between the sensing line SL and the bit lines BL0-BL2. In other words, the voltage difference between the sensing line SL and the bit lines BL0-BL2 is held constant. The resistive memory element may generate a current (e.g., a column signal) in the bit lines BL0-BL2 that is determined by the resistance R _DATA with respect to the same fixed voltage V _{amp_ref} applied to both ends of each.

The read circuit 251 of the output circuit includes a current mirror that copies a column signal for each bit of the synapse memory cell 211 and the reference memory cell at a multiple of the current corresponding to the bit. For example, the current mirror of the read circuit 251 may copy a column signal generated in a bit line to other bit lines BL'0 to BL'2. The magnitude of the copied current may vary depending on the width of the transistor. As described above, the current mirror may copy the column signal at a current copy ratio corresponding to the bit weight value. For example, in FIG. 2, the first bit line BL0 is copied at 1/4, the second bit line BL1 is copied at 1/2, and the third bit line BL3 is copied at 1.

The read circuit 251 of the output circuit can generate a column integrated signal I _CELLS by accumulating column signals for each bit of the synapse memory cell 211, and can generate a reference integrated signal I _REF by accumulating reference signals for each bit of the reference memory cell. For example, the output circuit can generate the column integrated signal I _CELLS by summing currents copied from the column signals. The output circuit can generate the reference integrated signal I _REF for the reference signal by the same read circuit 251 and operation as described above. For reference, when the resistive memory element is realized as an MRAM, an amplifier capable of processing a relatively low common input is required because the current flowing through the MTJ is small and no read disturbance occurs. The output circuit may further include an LIF circuit 260 for generating an output signal based on the column accumulation signal _{I_Cells} and the reference accumulation signal _{I_REF} and for processing the generated output signal, the structure and operation of which will be described later.

The output circuit can accurately distinguish signals when there is one P-state resistive memory element and multiple AP-state resistive memory elements by using an output signal that is the difference between the column accumulation signal I _Cells and the reference accumulation signal I _REF from the above-mentioned read circuit 251. In addition, since the reference accumulation signal I _REF is generated using a reference memory element having a programmable resistance value, the output signal has values corresponding to not only positive integer multiples of the net signal but also negative integer multiples, and the output circuit can use this to express not only EPSP but also IPSP.

In other words, since the net signal is always positive, when the output signal is a positive integer multiple of the net signal, it is understood that the column accumulated signal I _Cells is greater than the reference accumulated signal I _REF , and when the output signal is a negative integer multiple of the net signal, it is understood that the column accumulated signal I _Cells is less than the reference accumulated signal I _REF .

Figure 2 shows the output signal obtained by reading the resistance value of the resistive memory element included in the synapse memory array using the read circuit 251 of the output circuit, and Figure 3 below shows the firing process using the output signal.

Figure 3 shows a leaky integration and fire (LIF) circuit for an output circuit according to one embodiment.

The output circuit 150 according to an embodiment includes a LIF circuit 260. The LIF circuit 260 is a circuit for replicating neuron operation, and can fire other neuron circuits based on the result of comparing the output signal 159 with a threshold value. The output signal 159 is the I _{net, column} described above in the above-mentioned formula (5), so a description thereof will be omitted. The LIF circuit 260 includes a leakage partial circuit 361 and a firing partial circuit 362.

Post-synaptic potentials include EPSPs, which are potentials that increase in voltage in response to a stimulus, and IPSPs, which are potentials that decrease in voltage in response to a stimulus. EPSPs may be called excitatory postsynaptic potentials, and IPSPs may be called inhibitory postsynaptic potentials. In neurons, when a stimulus that exceeds the threshold is transmitted from a pre-neuron to a post-neuron, or when multiple stimuli below the threshold are transmitted several times within a short period of time, an action potential may be generated. The membrane potential of the neuron can then be reset to the base voltage. However, if a stimulus below the threshold is transmitted from a pre-neuron to a post-neuron, a post-synchronous potential (PSP) occurs instead of an action potential, and leakage may occur little by little in the PSP until the next stimulus is received. The LIF circuit 260 is a circuit that realizes the aforementioned neuronal firing or leakage.

The leakage circuit 361 may receive the column accumulation signal and the reference accumulation signal described above with reference to FIG. 2. In FIG. 3, the column accumulation signal and the reference accumulation signal are modeled as current sources, but this is for convenience of explanation and is actually interpreted as a current supplied by the readout circuit shown in FIG. 2. The leakage circuit 361 includes a capacitor. For example, the output signal 159 flows in a current corresponding to the reference accumulation signal toward a node of the capacitor, and flows out a current corresponding to the column accumulation signal from a node such as a capacitor. Due to the inflow of the reference accumulation signal and the outflow of the column accumulation signal described above, a current corresponding to the difference between the column accumulation signal and the reference accumulation signal may flow in the capacitor.

The output circuit 150 generates as the output signal 159 a current corresponding to an integer multiple of a net current that is a difference between a first current I _P based on the resistive memory element having a first resistance value and a second current I _AP based on the resistive memory element having a second resistance value. The leakage portion circuit 361 is inactivated while the reset signal RESET is applied and is activated while the reset signal RESET is not applied. In other words, the leakage portion circuit 361 can leak an output current according to the output signal 159 for a critical time after the reset signal RESET is not applied.

The output current of the output signal 159 is converted into an output voltage by flowing through a capacitor. In the leakage partial circuit 361, the leakage operational amplifier OP _integ is connected at its output end to a window switch, and the window switch is connected to a capacitor connected to ground. While the window signal WINDOW is applied to the window switch, the leakage operational amplifier OP _integ and the capacitor connected to ground are connected. Therefore, the output current of the output signal 159 flows into the capacitor connected to ground located at the output end of the leakage operational amplifier OP _integ . The capacitor connected to ground may convert the output current into an output voltage by leaking charge due to the output current. Here, the voltage charged in the capacitor connected to ground leaks through the leakage transistor LEAKAGE.

An activation voltage based on _{VCM_COLUMN_IN} may be applied to the leakage transistor LEAKAGE. The window switch is turned on for a certain critical time (e.g., leakage time) by the window signal WINDOW. In other words, a capacitor connected to ground leaks output current until a critical time has elapsed after reset.

According to one embodiment, the leakage subcircuit 361 may transmit a leakage output voltage to the firing subcircuit 362 during the critical time. The comparator OP _fire of the firing subcircuit 362 compares a preset threshold voltage V _TH with the output voltage V _COLUMN . For example, the comparator OP _fire may output a firing signal (e.g., 1) if the output voltage V _COLUMN accumulated during the critical time exceeds the threshold voltage V _TH , and may output a leakage signal (e.g., 0) if the output voltage V _COLUMN is equal to or lower than the threshold voltage V _TH .

For reference, the critical time is set as the time required to reach the threshold voltage _VTH through leakage of a threshold current corresponding to the threshold in a separate circuit (e.g., a critical time generating circuit), as described later. Since both the output current and the threshold current are currents, it is difficult to compare one current with the other. Therefore, the comparator OP _fire of the firing subcircuit 362 can compare the voltage converted from the current leaked during the critical time from the moment the RESET signal becomes 0 according to the WINDOW signal generated in FIG. 7 to the moment the WINDOW signal becomes 0 with the threshold voltage. If the current is small, the amount converted to voltage is also small, and the converted voltage is smaller than the threshold voltage. On the contrary, if the current is large, the amount converted to voltage is large, so the converted voltage rises above the threshold voltage, and the comparator OP _fire outputs a firing pulse.

According to one embodiment, the LIF circuit can leak the output signal 159 in response to the voltage accumulated based on the output signal 159 not reaching the threshold voltage within a critical time. For example, the firing subcircuit 362 of the LIF circuit 260 can transmit the converted output voltage to the leaky operational amplifier _OPintegr if the leakage and converted output voltage are less than the threshold voltage. The analog MUX can provide the converted output voltage as a new common mode input _{VCM_COLUMN} of the leaky operational amplifier _OPintegr . The leakage circuit can obtain a leaky integrated output voltage by continuing as a leakage due to the output current with respect to the aforementioned common mode input when the output signal 159 based on the next neuron input is generated.

According to one embodiment, the LIF circuit 260 can fire other neuron circuits in response to the voltage accumulating based on the output signal 159 reaching a threshold voltage within a critical time corresponding to the threshold. For example, if the leakage output voltage exceeds the threshold voltage or if the leakage voltage of several small output signals 159 exceeds the threshold voltage, the comparator OP _fire of the firing subcircuit 362 may generate a firing signal. The firing signal may be a pulse output instructing firing. The firing signal may be generated as a signal synchronized to the clock signal through an analog amplifier-based comparator OP _fire and a flip-flop circuit synchronized to the clock signal.

The firing partial circuit 362 can initialize the common mode voltage by controlling the selection signal SEL of the analog MUX circuit and transmitting the V _{CM_COLUMN_IN} input from the outside to the leakage operational amplifier _{OP_integ} .

In one embodiment, the output circuit 150 performs integration, comparison, and leakage operations separately for each circuit. Therefore, the output circuit 150 can minimize errors due to constraints on the size of the capacitor and leakage current. Furthermore, the output circuit 150 can also mimic firing in units of ms, which is the stimulation period of an actual living organism.

In FIG. 3, the RESET signal and the SW signal may be generated by a timing generator to which an external clock Clk is input as a control signal. A critical time generation circuit, which will be described later, may generate a window signal WINDOW indicating a critical time based on the presence or absence of firing. The window signal WINDOW provides a critical time (e.g., integration time) that is robust to variations in the set threshold voltage, resistive memory element, and capacitor.

Figure 4 illustrates a threshold generation circuit and an additional reference circuit included in a neural network circuit according to one embodiment.

The neural network circuit according to one embodiment may further include a circuit for setting a threshold value. For example, the neural network circuit may further include a threshold memory array 480 and an additional reference memory array 490. The synapse memory array 110 and the reference memory array 120 have been described above with reference to FIG. 1, and therefore will not be described here.

The threshold memory array 480 includes a plurality of memory elements. In the threshold memory array 480, at least one memory element designated based on a set threshold among the plurality of memory elements may have a first resistance value (e.g., P state). In FIG. 4, the memory cells included in the threshold memory array 480 are threshold memory cells and are illustrated as Th. For example, a memory element for expressing a threshold value may be selected taking into consideration a bit digit. In other words, the threshold value may be expressed based on the number of memory elements having the first resistance value and the bit digit represented by the memory element.

The threshold memory array 480 includes threshold memory cells arranged along any one of the column lines. The column line may include a plurality of bit lines representing multiple bits. Each of the threshold memory cells may include a sub-cell for each bit line. For example, when the threshold is set as a threshold current of 6I 1 _P , in order to represent (2+4)*I 1 _P , a memory element corresponding to the first bit digit from the LSB (e.g., 2 ¹ =2) and a memory element corresponding to the second bit digit from the LSB (e.g., 2 ² =4) in the threshold memory array 480 may have a first resistance value.

The additional reference memory cells share the reference word lines (Ref WL0 to Ref WL-1) with the threshold generation circuit. The additional reference memory cells may include additional reference memory elements arranged along the reference word lines and having a second resistance value. The memory elements of the additional reference memory cells of the additional reference memory array 490 may have a programmable resistance value depending on the application. The additional reference memory cells, described below, may be used to represent a specified threshold value as the threshold memory array 480 described above in a net signal.

For example, the signal accumulated along the columns of the threshold memory array 480 described above (hereinafter, the "threshold accumulation signal") can be expressed as the following equation (6).

In the above equation (6), I _P represents the current through the resistive memory element in the P state (e.g., a first resistance value), and I _AP represents the current through the resistive memory element in the AP state (e.g., a second resistance value). The threshold accumulation signal I _Th is a linear combination of I _P and I _AP . The second read circuit 452 of the output circuit 450 generates the threshold accumulation signal I _Th .

The above-mentioned Equation (7) is a signal in which a bit weighting value for each bit digit is applied to a current signal flowing through a resistive memory element to which a reference word signal is applied among resistive memory elements included in the additional reference memory array 490. The additional reference accumulated signal _IREF,Th is a signal obtained by applying a bit weighting value to a reference signal for each bit digit and integrating it. Here, Y may be determined to be any one of an integer smaller than or equal to R and larger than or equal to 0 according to an application. When the additional reference memory array 490 is composed of 1-bit cells, Y has a value of 0 or Y.

The second read circuit 452 of the output circuit 450 generates an additional reference accumulated signal _{I_REF,Th} . In the above-mentioned equations (6) and (7), the coefficients T and R of the linear combination can be expressed as the following equations (8) and (9).

R indicates a value determined based on the subcells connected to the activated reference word line. L indicates the number of activated reference word lines. An example in which all reference word lines are activated will be described with reference to FIG. 4. a indicates the number of subcells included in each memory cell, and the number of bits of the synapse weight value. For example, R is a sum of powers of 2 using the bit digits indicated by the subcells included in the reference memory cells connected to the activated reference word line among the threshold memory cells connected along one column as an exponent. T is an integer equal to or less than R, and is a value determined based on the subcells in the P state (e.g., the first resistance value) among the threshold memory cells connected to the activated reference word line. T corresponds to a preset threshold. t _ij is 1 if the i-th threshold memory cell of the activated j-th word line has a first resistance value, and is 0 if the i-th threshold memory cell has a second resistance value. For example, T is a sum of k powers of 2 using the bit digits indicated by the subcells in the P state among the threshold memory cells connected in the same column as an exponent.

The output circuit 450 of the neural network circuit according to an embodiment generates a threshold net signal I net,Th corresponding to the difference between the threshold accumulated signal I _Th and the additional reference accumulated signal I _{REF, Th according to the above-mentioned Equation (10). The output circuit 450 compares the output signal I net,} column obtained from the first readout circuit 451 (e.g., the circuit including the synapse readout circuit 151 and the reference readout circuit 152 of FIG. 1) with the threshold net signal I _net,Th obtained according to the above-mentioned Equation (10) in _the above-mentioned Equation (5). However, as described above, the comparison between current and current is difficult to realize, and further, the output circuit 450 may further include a critical time generation circuit for indirectly comparing the output signal I _net,column and the threshold net signal I _net,Th . The critical time generation circuit will be described with reference to FIG. 7 below.

The output circuit 450 according to one embodiment applies a critical time corresponding to a threshold determined based on the threshold memory cell and the additional reference memory cell to an output signal for an output line and another output signal for another output line. In other words, the output circuit 450 can apply a critical time corresponding to a common threshold to output signals obtained from the output lines of the synapse memory array.

Figures 5A to 5C illustrate an example of a firing comparison using net signals generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to one embodiment.

In FIG. 5A, one column of the synapse memory array is illustrated as synapse memory column 510. One column of the reference memory array is illustrated as reference column 520. One column of the threshold memory array is illustrated as threshold generation column 580. One column of the additional reference memory array is illustrated as additional reference column 590. For ease of explanation, each column includes 3-bit bit lines, and an example is shown in which an input signal is applied to one input line.

For example, the third subcell of the synapse memory column 510 may have a resistive memory element set to the second resistance value, and the first and second subcells may have a resistive memory element set to the first resistance value. The first and third subcells of the reference column 520 may have a resistive memory element set to the second resistance value, and the second subcell may have a resistive memory element set to the first resistance value. However, the resistance value of each subcell of the reference column 520 is not limited to the above example, and may be dynamically changed by the application.

Each sub-cell may generate a current signal to which a bit weighting value corresponding to the LSB (e.g., 1x), a bit weighting value of the first bit digit from the LSB (e.g., 2x), and a bit weighting value of the second bit digit from the LSB (e.g., 4x) are applied. The column accumulation signal I _Cells of the synapse memory column 510 may be (2+4)*I _P +1*I _AP . The reference accumulation signal I _REF of the reference column 520 may be 2*I _P +(1+4)*I _AP . The output signal I _net,column may be 4 (I _P -I _AP ). Since the output signal I _net,column is a positive number, it can be seen that the output signal I _net,column is an output signal for performing an EPSP operation.

Similarly, the threshold accumulation signal _{I_Th} generated in the threshold generation column 580 may be 2* _{I_AP} + (1+4)* _{I_P} = _{5I_P} + _{2I_AP} . The additional reference accumulation signal I_REF _,Th generated in the additional reference column 590 may be 2* _{I_P} + (1+4)* _{I_AP} = _{2I_P} + _{5I_AP} . In FIG. 5A, the threshold net signal _{I_net,Th} may be determined to be 3 ( _{I_P} - _{I_AP} ). The output circuit may generate a fire signal because the output signal _{I_net,column} is greater than the threshold net signal I_net _,Th .

Referring to FIG. 5B, the third subcell of the synapse memory column 510 has a resistive memory element set to a first resistance value, and the first and second subcells have a resistive memory element set to a second resistance value. In the case of the reference column 520, similar to FIG. 5A, the first and third subcells may have a memory element set to a second resistance value, and the second subcell may have a memory element set to a first resistance value. However, the resistance value of each subcell of the reference column 520 is not limited to the above example, and may be dynamically changed by the application.

Each sub-cell generates a current signal to which a bit weighting value corresponding to the LSB (e.g., 1x), a bit weighting value of the first bit digit from the LSB (e.g., 2x), and a bit weighting value of the second bit digit from the LSB (e.g., 4x) are applied. The column accumulation signal I _Cells of the synapse memory column 510 may be 1*I _P +(2+4)*I _AP . The reference accumulation signal I _REF of the reference column 520 may be 2*I _P +(1+4)*I _AP . The output signal I _net,column may be -1 (I _P -I _AP ). Since the output signal I _net,column is a negative number, it can be seen that the output signal I _net,column is an output signal for performing an IPSP operation.

Figure 5C shows an example where an input signal is applied to two input lines.

The output signal I _net,column in Figure 5C may be 4(I _P -I _AP )-(I _P -I _AP )=3(I _P -I _AP ) which is the sum of the output signals of Figures 5A and 5B. Similarly, the threshold net signal I _net,Th may be determined to be 6(I _P -I _AP ). The output circuit may perform leakage operation because the output signal I _net,column is less than the threshold net signal I _net,Th .

Figures 6A-6C illustrate an example of a firing comparison using net signals generated by a synaptic memory column, a reference column, a threshold generation column, and an additional reference column according to another embodiment.

In FIG. 6A, one of the columns of the synapse memory array is shown in synapse memory column 610. A column of the reference memory array is shown in reference column 620. A column of the threshold memory array is shown in threshold generation column 680. One of the columns of the additional reference memory array is shown in additional base column 690. For ease of explanation, an example is shown in which each column includes three bit lines and an input signal is applied to one input line.

Exemplarily, the third subcell of the synapse memory column 610 may have a resistive memory element set to a first resistance value, and the first and second subcells may have a resistive memory element set to a second resistance value. The reference column 620 may be a 1-bit cell that stores a single bit of information (e.g., a sign bit) regardless of the number of bits of the synapse memory column 610. More specifically, the reference column 620 may have a resistive memory element set to a second resistance value or a first resistance value in a 1-bit cell depending on whether the operation to be replicated is EPSP or IPSP. In FIG. 6A, the operation to be replicated is EPSP, and the 1-bit cell of the reference column 620 may have a resistive memory element set to a second resistance value.

Each sub-cell of the synapse memory column 610 may generate a current signal to which a bit weighting value corresponding to the LSB (e.g., 1x), a bit weighting value of the first bit digit from the LSB (e.g., 2x), and a bit weighting value of the second bit digit from the LSB (e.g., 4x) are applied. The column accumulation signal I _Cells of the synapse memory column 610 may be (4+2)*I _P +1*I _AP .

A current signal to which a predetermined bit weight (e.g., 7 times) is applied is generated for the 1-bit cell of the reference column 620. The reference accumulation signal I _REF of the reference column 620 may be 7*I _AP . The output signal I _net,column may be 6 (I _P -I _AP ). Since the output signal I _net,column is a positive number, it can be seen that the output signal I _net,column is EPSP.

Similarly, the threshold accumulation signal _{I_Th} in the threshold generation column 680 may be 2* _{I_AP} + (1+4)* _{I_P} = _{5I_P} + _{2I_AP} . The additional reference accumulation signal _{I_REF,Th} generated in the additional reference column 690 may be 7* _{I_AP} . In FIG. 6A, the threshold net signal I_net _,Th may be determined to be 5 ( _{I_P} - _{I_AP} ). The output circuit may generate a fire signal because the output signal I_net _,column is greater than the threshold net signal I_net _,Th .

Referring to FIG. 6B, the third subcell of the synapse memory column 610 may have a resistive memory element set to a second resistance value, and the first and second subcells may have a resistive memory element set to a first resistance value. In FIG. 6B, the operation to be replicated is IPSP, and the 1-bit cell of the reference column 620 may have a resistive memory element set to a first resistance value.

Each sub-cell of the synaptic memory column 610 generates a current signal to which a bit weighting value corresponding to the LSB (e.g., 1x), a bit weighting value corresponding to the first bit digit from the LSB (e.g., 2x), and a bit weighting value corresponding to the second bit digit from the LSB (e.g., 4x) are applied. The column accumulation signal I _Cells of the synaptic memory column 510 may be (2+4)*I _P +1*I _AP .

A current signal to which a predetermined bit weight (e.g., 7 times) is applied is generated for the 1-bit cell of the reference column 620. The reference accumulation signal I _REF of the reference column 620 may be 7*I _P. The output signal I _{net, column} may be −1 (I _P −I _AP ).

Figure 6C illustrates an example where signals are applied to two input lines.

The output signal I _net,column in Figure 6C may be 6(I _P -I _AP )-(I _P -I _AP )=5(I _P -I _AP ) which is the sum of the output signals of Figures 6A and 6B. Similarly, the threshold net signal I _net,Th may be determined to be 10(I _P -I _AP ). The output circuit may perform leakage operation because the output signal I _net,column is less than the threshold net signal I _net,Th .

Figure 7 shows a critical time generation circuit for an output circuit according to one embodiment.

An output circuit according to an embodiment can set a critical time 791 corresponding to a threshold based on a signal generated based on a threshold memory cell and a signal generated based on an additional reference memory cell. FIG. 7 describes a critical time generation circuit 753 that sets the critical time 791 in the output circuit. The first read circuit 451, the second read circuit 452, and the LIF circuit have been described above, and therefore will not be described here.

As described above, since it is difficult to directly compare the output current with the current set as the threshold, the critical time generation circuit 753 can set a critical time 791 for current comparison. The critical time generation circuit 753 may set the time taken to reach the threshold voltage through the leakage of the current corresponding to the threshold as the critical time 791. As described above, the output circuit compares whether the time taken to reach the same threshold voltage through the leakage of the output current is smaller than the critical time 791. For example, the output circuit can indirectly compare the threshold current with the output current by determining whether the output voltage converted through the leakage of the output current exceeds the threshold voltage within the set critical time 791.

The output circuit discloses an accumulation of currents corresponding to the difference between the signal generated based on the threshold memory cell and the signal generated based on the additional reference memory cell, and outputs a signal indicating a critical time 791 corresponding to the threshold when the voltage corresponding to the accumulated current exceeds the threshold voltage.

The critical time generating circuit 753 includes a configuration similar to that of the leakage portion circuit described above with reference to Fig. 3. For example, the critical time generating circuit 753 can input the threshold accumulation signal I _Th to a node of a capacitor and input the additional reference accumulation signal I _REF,Th from the node. Thus, the threshold net signal I _net,Th flows to the capacitor. The capacitor and the operational amplifier may be designed to have the same type and size.

The reset signal RESET and the hold signal HOLD shown in the timing diagram 790 are generated by a timing generator. When the reset signal RESET=1, the voltages across the capacitor through which the threshold net signals I _{net and Th} flow are all VCM and TWG, so they are initialized. When the reset signal RESET=0 and the hold signal HOLD=1, the converted voltage V _TWG is stored in the capacitor through accumulation by the threshold net signals I _{net and Th} . The converted voltage V _TWG gradually increases due to the accumulation of the threshold net signals I _{net and Th} . The window comparator OP _window outputs 0 when the converted voltage V _TWG exceeds the threshold voltage V _TH . Therefore, the critical time 791 is defined as the time from when the reset signal RESET=0 to when the window signal WINDOW=0. _{V CM and TWG may be set smaller than VCM_COLUMN_IN} . For example, when the output signal I _net,column and the threshold net signal I _net,Th are the same (e.g., I _net,column = 7I _P - 7I _AP , I _net,Th = 7I _P - 7I _AP ), since the converted voltage V TWG based on V _{CM_COLUMN_IN} is higher than the threshold voltage V _TH based on V _{CM,TWG ,} it is possible to confirm firing even in the case of 7I _P - 7I _AP .

Figure 8 shows the elements included in the LIF circuit in an output circuit according to one embodiment, and a timing diagram for each element.

The reset flip-flop 810 may generate a reset signal RESET in response to a clock signal divided by 4 and a neuron input (input signal). The hold reset flip-flop 820 may generate a hold signal HOLD in response to a clock signal divided by 8 and a neuron input (e.g., an input signal). The compare reset flip-flop 830 may generate a compare signal COMPARE in response to a clock signal, a supply voltage VDD, and an inverted hold signal HOLDB. The compare signal COMPARE is a signal that starts the comparison operation of the comparator when the accumulated output signal is completely transmitted to the input of the comparator. The leakage reset flip-flop 840 generates a leakage signal LEAKAGE in response to a clock signal, a supply voltage VDD, and an inverted compare signal COMPAREB. The leakage signal LEAKAGE is a signal for performing a leakage operation of the output voltage after the above-mentioned comparison operation. If there is a leakage voltage, the SW pull ripple loop 870 can generate a SW signal for transmitting the leakage voltage to the leakage subcircuit so that the leakage voltage is added to the output based on the next input. As shown in the timing diagram 890 for the above-mentioned signals, the critical time 891 is the time period after reset before the window signal WINDOW is inactivated.

Figure 9 illustrates an example of the firing behavior of a neural network circuit according to one embodiment.

The neural network circuit includes a memory array 910, an output circuit 950, a divider and timing generator 940, a word line driver 970, and a write driver 980. The memory array 910 includes a synapse memory array, a reference memory array, a threshold memory array, and an additional reference memory array. The output circuit 950 may include a read circuit 951, a critical time generation circuit 953, and an LIF circuit 960. The memory array 910 and the output circuit 950 have been described above, so their description will be omitted. The divider and timing generator 940 generates signals used in the circuit (e.g., clock signals and control signals for individual elements). The word line driver 970 drives the word lines (e.g., input lines) of the memory array 910. The write driver 980 can set the resistance value of the resistive memory elements of the memory cells arranged along the bit lines of the memory array 910 and drive the bit lines and sensing lines.

9 illustrates an exemplary operation of the neural network circuit with a threshold set as T=6. The threshold net signal I _net,Th may be 6(I _P -I _AP ). The current through column <0> changes from 7(I P -I _AP )→2(I _P -I AP )→7(I _{P -I AP )} +(-3)(I _P -I _AP ) and the current through column <1> changes from 7(I _P -I _AP )→-2(I _P -I _AP )→3(I _P -I _AP )+(-3)(I _P -I _AP ). As described above, if a stimulus below the threshold is delivered, the firing signal is not output, and the accumulated voltage V_integrated is delivered in the section where the leak pulse is high. The level of the accumulated voltage gradually decreases during the section where the accumulated pulse is high. If a stimulus above the threshold is delivered, the firing signal is output, and the accumulated voltage V_integrated is initialized. In addition, if a stimulus below the threshold is delivered and leaks, and then a stimulus is delivered and the sum of the two stimuli exceeds the threshold, the firing signal is output. In the example shown in FIG. 9, since the threshold T=6, firing → leakage → leakage occurs in the order of the current flowing through column <0>. Firing → leakage → leakage occurs in the order of the current flowing through column <1>.

Figure 10 is a flowchart showing a method of operation of a neural network circuit according to one embodiment.

First, in step 1010, the neural network circuit generates a column signal based on the input signal and the resistive memory element of a synapse memory cell to which an input signal is applied via an input line, among one or more memory cells arranged along an output line. For example, when an input signal (e.g., a spike signal) is received via a pre-synapse circuit, the neural network circuit accesses a word line of the memory array. A read circuit of the neural network circuit can generate a column signal (e.g., a current) based on the synapse weight value and the input signal.

Then, in step 1020, the neural network circuit generates a reference signal based on the input signal and a reference memory element having a reference resistance value of a reference memory cell to which an input signal is applied, among one or more memory cells arranged along a reference line. The reference resistance value may be determined based on a predetermined combination of the first resistance value or the second resistance value by an application.

Next, in step 1030, the neural network circuit generates an output signal for the output line from the column signal and the reference signal. The neural network circuit generates an output signal corresponding to the difference between the column accumulated signal and the reference accumulated signal to cancel the current in the high resistance state.

Then, in step 1040, the neural network circuit fires or accumulates based on the output signal. For example, the neural network circuit may leak the output signal for a critical time. The neural network circuit fires or accumulates depending on the result of comparing the leaked output voltage with a threshold voltage. The neural network circuit may fire if the output voltage exceeds the threshold voltage. If the output voltage is equal to or less than the threshold voltage, the neural network circuit accumulates and holds the voltage until the next neuron input arrives.

However, the output signal processing of the neural network is not limited to the above-mentioned step 1040. The output circuit of the neural network circuit can obtain the multiplication and accumulation (MAC) between the input signal received along the input line and the synaptic weight value based on the result of interpreting the output signal (e.g., an analog-digital converted value), and can transmit a node value (e.g., an activation value) determined based on the obtained multiplication and accumulation to another neuron circuit.

The embodiments described above may be implemented with hardware components, software components, or a combination of hardware and software components. For example, the devices and components described in the present embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or a different device that executes and responds to instructions. The processing device executes an operating system (OS) and one or more software applications that run on the operating system. The processing device also accesses, stores, manipulates, processes, and generates data in response to the execution of the software. For ease of understanding, the description may assume that a single processing device is used, but one skilled in the art will appreciate that a processing device may include multiple processing elements and/or multiple types of processing elements. For example, a processing device may include multiple processors or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

Software may include computer programs, codes, instructions, or any combination of one or more thereof, to configure or instruct a processing device to operate as desired, either independently or in combination. The software and/or data may be embodied permanently or temporarily in any type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, to be interpreted by or provide instructions or data to a processing device. The software may be distributed across computer systems coupled to a network, and may be stored and executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to the present invention is embodied in the form of program instructions to be executed by various computer means and recorded on a computer-readable recording medium. The recording medium includes program instructions, data files, data structures, and the like, either alone or in combination. The recording medium and program instructions may be specially designed and constructed for the purposes of the present invention, or may be known and available to those skilled in the art of computer software technology. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include not only machine language code, such as that generated by a compiler, but also high-level language code executed by a computer using an interpreter, etc.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations described in the present invention, and vice versa.

Although the embodiments have been described above with reference to limited drawings, a person having ordinary skill in the art may apply various technical modifications and variations based on the above description. For example, the described techniques may be performed in a different order than described, and/or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, and may be replaced or substituted with other components or equivalents to achieve suitable results.

Therefore, other realizations, other embodiments, and equivalents to the claims are also within the scope of the claims set forth below.

100 Neural network circuit 110 Synapse memory array 111 Synapse memory cell 120 Reference memory array 121 Reference memory cell 150 Output circuit 151 Synapse readout circuit 152 Reference readout circuit

Claims (27)

A neural network circuit, comprising:
a synapse memory cell disposed along the output line, the synapse memory cell including a resistive memory element having one of a first resistance value and a second resistance value, the synapse memory cell responsive to receiving an input signal via an input line to generate a column signal based on the resistive memory element and the input signal;
a reference memory cell arranged along a reference line, the reference memory cell including a reference memory element having the first resistance value or the second resistance value in a ratio predetermined by an application, the reference memory element and the reference memory cell generating a reference signal based on the reference memory element and the input signal;
an output circuit for generating an output signal for the output line from the column signal and the reference signal;
A neural network circuit comprising: The synapse memory cell includes a resistive memory element having a number of bits for expressing a synapse weight value assigned to the synapse memory cell,
The neural network circuit of claim 1 , wherein the resistive memory elements of said number of bits are arranged along the same input line. the reference memory cell includes a reference memory element having a number of bits for expressing the synapse weight value;
3. The neural network circuit of claim 2, wherein the reference memory elements of said number of bits are arranged along the same input line. The neural network circuit of claim 2, wherein the reference memory cell includes a reference memory element for representing a sign bit. The neural network circuit of claim 1, wherein the output circuit generates as the output signal a current corresponding to a positive integer multiple of a net current that is a difference between a first current based on a resistive memory element having the first resistance value and a second current based on a resistive memory element having the second resistance value. The neural network circuit of claim 1, wherein the output circuit generates as the output signal a current corresponding to a negative integer multiple of a net current that is a difference between a first current based on a resistive memory element having the first resistance value and a second current based on a resistive memory element having the second resistance value. The neural network circuit of claim 1, wherein the output circuit further includes a leaky integration and fire (LIF) circuit that fires other neuron circuits based on a result of comparing the output signal with a threshold. The neural network circuit of claim 7, wherein the LIF circuit increases the output voltage in response to the output signal corresponding to a positive integer multiple of the net current. The neural network circuit of claim 7, wherein the LIF circuit reduces the output voltage in response to the output signal corresponding to a negative integer multiple of the net current. The neural network circuit of claim 7, wherein the LIF circuit leaks the output signal in response to a voltage accumulating based on the output signal not reaching a threshold voltage within a critical time. The neural network circuit of claim 7, wherein the LIF circuit fires the other neuron circuit in response to a voltage accumulated based on the output signal reaching a threshold voltage within a critical time corresponding to the threshold. The neural network circuit of claim 7 further includes a threshold memory array including a plurality of memory elements, at least one memory element of the plurality of memory elements designated based on the set threshold value has the first resistance value. The neural network circuit of claim 12, further comprising an additional reference memory cell that shares a reference word line with the threshold generation circuit, is disposed along the reference word line, and has an additional reference memory element having the second resistance value. The neural network circuit of claim 13, wherein the output circuit sets a critical time corresponding to the threshold based on a signal generated based on the threshold memory cell and a signal generated based on the additional reference memory cell. The neural network circuit of claim 14, wherein the output circuit starts accumulating a current corresponding to a difference between a signal generated based on the threshold memory cell and a signal generated based on the additional reference memory cell, and outputs a signal indicating a threshold time corresponding to the threshold when a voltage corresponding to the accumulated current exceeds a threshold voltage. The neural network circuit of claim 14, wherein the output circuit applies a critical time corresponding to the threshold determined based on the threshold memory cell and the additional reference memory cell to the output signal for the output line and other output signals for other output lines. The neural network circuit of claim 1, wherein the resistive memory elements of synapse memory cells connected to the same output line are connected in parallel with each other. further comprising a different synapse memory cell disposed along a different output line than the output line;
2. The neural network circuit of claim 1, wherein the output circuit generates output signals using the same reference memory cell for the output line and each of the different output lines individually. The neural network circuit of claim 1, wherein the output circuit includes a read circuit that generates a column accumulation signal by accumulating the column signals for each bit of the synapse memory cells, and generates a reference accumulation signal by accumulating the reference signals for each bit of the reference memory cells. The neural network circuit of claim 19, wherein the readout circuit includes a current mirror that copies the column signal for each bit of the synapse memory cell and the reference memory cell with a current multiple corresponding to that bit. The neural network circuit of claim 19, wherein the output circuit generates the output signal corresponding to a difference between the column accumulation signal and the reference accumulation signal. The neural network circuit of claim 21, wherein the output circuit includes a capacitor through which a current corresponding to the difference between the column accumulation signal and the reference accumulation signal flows by causing a current corresponding to the reference accumulation signal to flow into a node and a current corresponding to the column accumulation signal to flow out of the node. The neural network circuit of claim 1, wherein the output circuit obtains a sum of products between the input signal received along the input line and the synaptic weight value based on a result of interpreting the output signal, and transmits a node value determined based on the obtained sum of products to another neuron circuit. The neural network circuit of claim 1, wherein the ratio is predetermined based on the application. 1. A method of operating a neural network circuit, comprising:
generating a column signal based on a resistive memory element of a synapse memory cell to which an input signal is applied via an input line, among one or more memory cells arranged along the output line; and
generating a reference signal based on a reference memory element having a reference resistance value of a reference memory cell to which the input signal is applied, among one or more memory cells arranged along a reference line, and the input signal;
generating an output signal for the output line from the column signal and the reference signal;
Including,
A method of operating a neural network circuit, wherein the reference resistance value is determined by an application based on a predetermined combination of a first resistance value and a second resistance value different from the first resistance value. The method of claim 25, wherein the combination is predetermined based on the application. 1. A processor comprising:
a memory array including rows and columns of resistive memory cells including respective resistors;
each resistive memory cell of the memory array having a resistance programmable to change between a first resistance value and a second resistance value, and a column of the memory array provides an integrated thermal current based on an input signal provided to the resistive memory elements of the column and the resistance of each memory element of the column;
a reference array of resistive memory cells each including a resistor;
A processor, wherein each resistive memory cell of the reference array is programmed to vary between a first resistance value and a second resistance value, and the reference array provides an integrated reference current based on an input signal provided to the resistive memory elements of the reference array and the resistance of each memory element.

Images