CN106297772A - Detection method is attacked in the playback of voice signal distorted characteristic based on speaker introducing - Google Patents

Abstract

The invention discloses a playback attack detection method based on the distortion characteristics of a speech signal introduced by a loudspeaker. First, the speech to be detected is preprocessed, and the voiced sound frame is retained; feature extraction is performed for each voiced sound frame in the preprocessed speech signal, Obtain the eigenvector based on the linear distortion and nonlinear distortion characteristics of the speech signal; average the eigenvectors of all voiced frames to form a statistical eigenvector and obtain the feature model of the speech to be tested; then extract the eigenvector of the training speech sample to obtain the training Speech feature model, and use the trained speech feature model to train the SVM model to obtain a speech model library; finally, perform SVM pattern matching on the feature model of the speech to be tested and the trained speech model library, and output the judgment result. The invention can realize real-time and effective detection of playback voice.

Description

Playback Attack Detection Method Based on the Distortion Characteristics of Speech Signal Introduced by Loudspeaker

technical field

The invention belongs to the field of digital media processing, and relates to a playback attack detection method, in particular to a method for judging whether a voice is a voice content safe for a playback attack.

Background technique

The advantage of biological characteristics as an inherent attribute of living organisms has led to the emergence of biometric technology. Speaker recognition is part of biometrics, which is based on the user's voice samples to achieve identity authentication. Compared with other biological characteristics, voice has the advantages of simple voice pickup equipment, usable anytime and anywhere, and small data volume. Voiceprint verification technology has been developed for more than 60 years since it was proposed, and has made great progress, and has been widely used. However, the current identity authentication system for voiceprints is facing various masquerade attacks, including recording playback, speech synthesis, speech conversion, and voice imitation. The playback attack refers to the attacker using a recording device to record a legitimate user entering the authentication system. The real-time voice is then played back through the loudspeaker at the pickup end of the system, so as to achieve the purpose of disguising the user and entering the system. Due to the low cost and portability of existing recording equipment, this attack is easy to operate and easy to implement. Recording and playback attacks have become the most widespread and most threatening means of masquerading attacks. The existing mainstream speaker recognition platform has a very high false acceptance rate for replay attacks, which shows that recording replay attacks pose a great threat to the security of the voiceprint authentication platform. This shows how to implement recording replay attack detection based on voice It is an important problem that urgently needs to be solved in the identity authentication system of the pattern.

Since the recording playback attack appeared, only a few research teams at home and abroad have studied it, and their main technical achievements were concentrated before 2011, and the development has been slow in recent years. Moreover, the existing research results have strict requirements and restrictions on the voice sampling frequency, system storage space, voice collection environment and other conditions, and cannot achieve high accuracy and real-time recognition effects, so they cannot be widely applied to existing voiceprint recognition platforms. .

The spectrogram of the voice signal can accurately and intuitively reflect the changes and differences before and after the voice signal is modified. Compared with the original voice, the playback attack process introduces three links: microphone collection, digital compression, and speaker playback, and each link may be damaged. cause a change in the speech signal. Therefore, according to the analysis of the changes in the speech signal spectrum in the three links, a replay attack detection algorithm based on the speech signal spectrum characteristics can be designed and realized with good universality, real-time performance and high accuracy.

Contents of the invention

Aiming at the safety loophole that the existing voiceprint recognition system cannot resist the replay attack, the present invention provides a replay attack detection method based on the distortion characteristics of the voice signal introduced by the loudspeaker.

The technical solution adopted in the present invention is: a playback attack detection method based on the distortion characteristics of the speech signal introduced by the loudspeaker, which is characterized in that, comprising the following steps:

Step 1: Preprocessing the speech to be detected, retaining voiced frames;

Step 2: perform feature extraction for each voiced sound frame in the preprocessed speech signal, and obtain a feature vector based on the linear distortion and nonlinear distortion characteristics of the speech signal;

Step 3: the feature vectors of all voiced frames are averaged to form a statistical feature vector to obtain a feature model of the speech to be tested;

Step 4: extract the feature vector of the training speech sample, obtain the training speech feature model, and use the training speech feature model to train the SVM model, and obtain the speech model library;

Step 5: Perform SVM pattern matching between the feature model of the speech to be tested and the trained speech model library, and output the judgment result.

Preferably, the preprocessing of the speech to be detected in step 1 is to use a Hamming window to perform frame division and window processing on the speech signal, the frame length is 70 ms, and the voiced sound frames are retained.

Preferably, the feature extraction for each voiced sound frame in the preprocessed speech signal in step 2 is to extract a 26-dimensional feature vector based on the linear distortion and nonlinear distortion characteristics of the speech signal.

Preferably, the extraction is based on the speech signal linear distortion feature vector, consisting of five features of low frequency ratio, low frequency variance, low frequency difference variance, low frequency fitting and global low frequency ratio, consisting of a total of 10 dimensional vectors;

The low frequency ratio Wherein X(f) is the fast Fourier transform of each frame;

The low frequency variance in

The low frequency difference variance in

The low-frequency fitting is to use the 6-dimensional fitting feature to fit the FFT sampling points of 0-500 Hz, and the fitting formula is Where x is the FFT sampling point from 0 to 500Hz, and a _i represents the fitting coefficient;

The global low frequency ratio

Preferably, the extraction is based on the nonlinear distortion feature vector of the speech signal, including three features of total harmonic distortion, clipping ratio and timbre vector, with a total of 16 dimensional feature vectors;

The total harmonic distortion in X(f) is the fast Fourier transform of each frame, f ₁ is the pitch frequency, and f _i is i times the pitch frequency;

The clipping ratio in x is the time-domain spectrum, len is the length of the time-domain spectrum;

The Timbre Vector

Preferably, the statistical feature vector in step 3 is a 26-dimensional statistical feature vector.

Preferably, the training speech samples in step 4 come from several devices and several recorders, including playback speech and original speech.

Preferably, in step 4, after extracting the feature vectors of the training speech samples, LIBSVM is used to perform binary classification training on the feature database in the training speech sample set, the feature database is composed of the feature vectors of the training speech samples.

The beneficial effects of the present invention are: the present invention can be integrated into the existing voiceprint recognition platform to realize real-time and effective detection of the playback voice, and provide safe and effective identity authentication for judicial evidence collection, e-commerce, financial systems and other fields in the current information age Technical Support.

Description of drawings

Fig. 1 is the overall flowchart of the algorithm of the embodiment of the present invention;

Fig. 2 is a feature extraction flowchart of an embodiment of the present invention;

Fig. 3 is a comparison diagram of differences introduced by replay attacks according to an embodiment of the present invention;

Fig. 4 is the acceleration frequency response graph of the embodiment of the present invention;

FIG. 5 is a spectrum diagram describing low-frequency attenuation distortion according to an embodiment of the present invention;

FIG. 6 is a spectrum diagram describing high-frequency harmonic distortion according to an embodiment of the present invention.

detailed description

In order to facilitate those of ordinary skill in the art to understand and implement the present invention, the present invention will be described in further detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described here are only used to illustrate and explain the present invention, and are not intended to limit this invention.

Related terms involved in the embodiments of the present invention are explained as follows:

1) Playback attack: Use recording equipment to record the speaker's voice, and then play this recording to the speaker recognition system, so that the speaker recognition system can judge that it is the speaker.

2) Signal spectrum: The amplitude or phase of each component of the signal is a function of frequency.

3) Linear distortion: the amplitude or phase distortion caused by the linear reactance components of the circuit responding differently to different frequencies, and there will be no new frequency components in the output signal

4) Non-linear distortion: New harmonic components are generated in the output signal, which shows that the output signal is not linearly related to the input signal.

5) Pitch: In polyphony, the sound with the lowest frequency is called the pitch, and the pitch of the musical tone is determined by the frequency of the pitch.

The present invention is a replay attack detection algorithm based on the spectral characteristics of voice signals. It uses the linear distortion and nonlinear distortion characteristics caused by the loudspeaker to the voice signal as the technical principle, extracts the corresponding feature vectors, and uses SVM for classification and judgment, which can realize real-time playback of voice effective detection.

See also Fig. 1, is the algorithm flowchart of the present invention, with reference to shown in this figure, the playback attack detection process to a section of voice has the following steps:

Step 1: For the speech to be detected, first use the Hamming window to frame and window the signal, the frame length is 70ms, and retain the voiced sound frame.

Step 2: Perform feature extraction for each voiced sound frame in the preprocessed speech signal to obtain a 26-dimensional feature vector based on the linear distortion and nonlinear distortion characteristics of the speech signal.

As shown in Figure 3, compared with the original voice, the playback attack process introduces three links: microphone acquisition, digital compression, and speaker playback. Among them, the impact of the speaker on the voice signal is the most significant, and it has multiple performance evaluation indicators. The influence of the sound process on the speech signal can be divided into two types: linear distortion and nonlinear distortion.

Linear distortion is due to the existence of linear components in the circuit, and its impedance varies with frequency, which causes the system to have different amplification factors and delay times for signal components of different frequencies. Linear distortion will change the magnitude and relative time relationship of different frequency signal components, but will not produce new frequency components that the input signal does not have.

As shown in Figure 5, the top is the original voice, and the bottom is the playback voice. The linear distortion is mainly reflected in the attenuation of the low frequency part on the speaker. As shown in Figure 4, since the sound radiation is directly proportional to the acceleration, the natural frequency of the speaker cone is designed to be lower than the operating frequency, and the speaker works in the mass working area. When Qm=1, the frequency response is relatively flat. In this working state, the loudspeaker will experience obvious low-frequency attenuation.

Non-linear distortion is caused by non-linear components in the circuit or entering non-linear regions. The main characteristic of nonlinear distortion is the generation of new frequency components that were not present in the input signal. Can be divided into harmonic distortion and transient intermodulation distortion.

Harmonic distortion refers to the harmful interference of various multiples of the original frequency. As shown in Figure 6, it is a section of original voice signal and corresponding playback voice signal. Since the amplifier is not ideal enough, the output signal not only contains the amplified input component, but also adds some frequency components (harmonics) that are integer multiples of the original signal. ), causing the output waveform to be out of shape.

Due to the unstable working characteristics of the transistor and the distortion caused by factors such as temperature, a deep negative feedback is adopted. In order to reduce the high-frequency oscillation caused by deep negative feedback, the transistor amplifier generally needs to add a small capacitor between the base and collector of the pre-driver transistor, so that the phase of the high-frequency band is slightly delayed, which is called lagging price or Weigh sub-price. When the input signal contains a high-speed transient pulse, the capacitor has no time to charge, and the circuit is in a state of no negative feedback. Because the input signal is not subtracted from the negative feedback input signal, the signal is too strong. These too strong signals will momentarily overload the amplification circuit, resulting in clipping of the output signal.

Please see Figure 2, the feature extraction process of this embodiment based on the principle of linear distortion and the principle of nonlinear distortion is as follows:

The features proposed based on the linear distortion phenomenon are all processed in the range of 500Hz, so as to achieve better discrimination effect. Here we propose five features: low frequency ratio, low frequency variance, low frequency difference variance, low frequency fitting and global low frequency ratio, with a total of 10 dimensional vectors to describe the low frequency attenuation characteristics in linear distortion.

①Low Spectral Ratio

The spectral peak distribution of the playback speech signal is lower than the original speech in the range of 250-350Hz, and higher than the original speech in the range of 500Hz, so the characteristic parameters of 250-350Hz can be compared with the characteristic parameters of 400-500Hz. distinguish between the two.

Equation 1, where X(f) is the fast Fourier transform for each frame.

②Low Spectral Variance

The low-frequency variance is used to describe the fluctuation of the signal in the low-frequency region. First, count the FFT sampling points within 500Hz. In the case of a frame length of 70ms, there are a total of 1120 sampling points at 16kHz, and a total of 35 sampling points within 0-500Hz;

③Low Spectral Difference Variance

First-order differences are often used to describe the degree of change in data. Here, the variance value of the first-order difference is used to more accurately describe the degree of data fluctuation in the low-frequency part.

④Low Spectral Curve Fit

The 6-dimensional fitting feature is used to fit the FFT sampling points from 0 to 500 Hz.

Where x is the FFT sampling point from 0 to 500Hz, and a _i represents the fitting coefficient;

⑤Global Low Spectral Ratio

This feature is proposed based on the existing frequency band feature detection algorithm and the attenuation effect of the speaker on the speech signal, and it has wide applicability through the improvement of the original algorithm. The extraction of low-frequency proportional features verifies the attenuation characteristics of the overall speech signal in the low-frequency part.

Among them, X(f) is the fast Fourier transform of each frame. The audio signal sampling frequency used in this experiment is 16kHz, and the attenuation part mainly occurs below 500Hz.

For nonlinear distortion phenomena, three features, total harmonic distortion, clipping ratio and timbre vector, are extracted, with a total of 16-dimensional feature vectors, which are used to describe high-frequency harmonic distortion and transient intermodulation distortion in nonlinear distortion. ①Total Harmonic Distortion (Total Harmonic Distortion)

The proposal of this feature is based on the harmonic distortion phenomenon of the loudspeaker to the high-frequency part of the voice. The ratio of the root mean square value of each harmonic to the root mean square value of the fundamental is called the harmonic content of the harmonic. The ratio of the root sum of the root mean square values of all harmonics to the root mean square value of the fundamental is called total harmonic distortion

where X(f) is the fast Fourier transform of each frame. f ₁ is the pitch frequency, and f _i is i times the pitch frequency.

② Clipping Ratio

The average value of the absolute value of the time-domain spectrum is compared with the maximum value to quantify the clipping phenomenon caused by the transient intermodulation distortion.

Where x is the time-domain spectrum, and len is the length of the time-domain spectrum.

③Timbre Vector

The playback signal is significantly different from the original signal in terms of harmonics. Timbre is mainly determined by the relative size of each harmonic (overtone). Timbre vectors can describe the relative size relationship of harmonics.

Step 3: After extracting the feature vector for each voiced sound frame, average the feature vectors of all the voiced sound frames to form a 26-dimensional statistical feature vector.

Step 4: extract the feature vector of the training speech sample, obtain the training speech feature model, and use the training speech feature model to train the SVM model, and obtain the speech model library;

Step 4.1: Input the training sample set, the training audio in the training sample set comes from multiple devices and multiple recorders, and includes playback voice and original voice; as shown in Figure 2, extract 26-dimensional statistics for all voice samples in the training sample set Feature vector.

Step 4.2: The speech determination problem is actually a binary classification problem, so the model used is SVM; after extracting the feature vector, use LIBSVM to perform binary classification training on the feature database in the training sample set.

Step 5: Perform SVM pattern matching on the feature model of the speech sample to be tested and the trained speech model library, and further output the judgment result.

Step 5.1: extracting the speech feature vector to be tested;

Step 5.2: Match the feature vector of the sample to be tested with the existing speech model library to obtain the judgment standard, and further output the judgment result.

The feature vector of the sample to be tested is pattern-matched with the existing speech model library. The trained SVM model has a classification boundary that distinguishes the original speech from the playback speech, which can realize the binary classification of the sample to be tested, and further output the judgment result, and the judgment is playback /original.

In order to verify the effectiveness of the algorithm, three experiments are set up for testing;

Experiment 1: Users of different age groups and different genders have great differences in voice characteristics such as frequency and intonation. Therefore, different user groups are classified into three age groups, namely under 18 years old, 18-40 years old and over 40 years old. , there are male recorders and female recorders in each age group; please see Table 1 below for the test results of different user groups;

Table 1 Classification test results of different user groups

test group Age1 (<18) Age2(18-40) Age3(>40) average Test indicators AR AR AR AR male 100% 99.2054% 98.2% 99.14% Female 97.7941% 98.3% 98.8372% 98.32% average 98.08% 98.69% 98.525% 98.68%

Experiment 2: Different speakers have different physical structures, and their frequency response curves are relatively different. The test for speakers can verify the recognition of different mainstream devices. The test devices are Huawei, iPhone, Samsung, Meizu, and Google nexus; different speakers are classified The test results are shown in Table 2 below;

Table 2 Classification test results of different loudspeakers

Equipment type Number of samples FAR AR iPhone5s 172 8.55% 91.45% Huawei 171 2.34% 97.66% Nexus 155 0.65% 99.35% meizu 175 1.15% 98.85% Samsung 254 3.15% 96.85% average 185.4 3.17% 96.83%

Experiment 3: The algorithm in document [1] is an excellent replay attack detection algorithm currently proposed, so the method of the present invention is compared with the algorithm in document [1] to verify the improvement of the recognition rate of this algorithm. The comparison test results are shown in Table 3 below;

Table 3 Algorithm comparison test

Experimental results show that the algorithm provided by the present invention has good detection versatility for different user groups and different loudspeaker devices, and the average recognition accuracy rate of the algorithm is as high as 98%, compared with the average recognition rate of 82% of the existing algorithm There has been a significant improvement.

Literature [1] Villalba, Jesús, and Eduardo Lleida. "Detecting replay attacks from far-field recordings on speaker verification systems." European Workshop on Biometrics and Identity Management. Springer Berlin Heidelberg, 2011.

It should be understood that the parts not described in detail in this specification belong to the prior art.

It should be understood that the above-mentioned descriptions for the preferred embodiments are relatively detailed, and should not therefore be considered as limiting the scope of the patent protection of the present invention. Within the scope of protection, replacements or modifications can also be made, all of which fall within the protection scope of the present invention, and the scope of protection of the present invention should be based on the appended claims.

Claims (8)

1. a playback attack detection method based on the speech signal distortion characteristics introduced by loudspeaker, it is characterized in that, comprising the following steps: Step 1: Preprocessing the speech to be detected, retaining voiced frames; Step 2: perform feature extraction for each voiced sound frame in the preprocessed speech signal, and obtain a feature vector based on the linear distortion and nonlinear distortion characteristics of the speech signal; Step 3: the feature vectors of all voiced frames are averaged to form a statistical feature vector to obtain a feature model of the speech to be tested; Step 4: extract the feature vector of the training speech sample, obtain the training speech feature model, and use the training speech feature model to train the SVM model, and obtain the speech model library; Step 5: Perform SVM pattern matching between the feature model of the speech to be tested and the trained speech model library, and output the judgment result. 2. the playback attack detection method based on the speech signal distortion characteristic that loudspeaker introduces according to claim 1, it is characterized in that: the speech to be detected described in step 1 is preprocessed, is to use Hamming window to carry out framing adding to speech signal Window processing, the frame length is 70ms, and the voiced frames in it are reserved. 3. the playback attack detection method based on the speech signal distortion characteristic that loudspeaker introduces according to claim 1, is characterized in that: the described step 2 carries out feature extraction for each voiced sound frame in the preprocessed speech signal, is to extract based on 26-dimensional eigenvectors of the linear distortion and nonlinear distortion characteristics of speech signals. 4. the playback attack detection method based on the speech signal distortion characteristic that loudspeaker introduces according to claim 1 or 3, is characterized in that: described extraction is based on speech signal linear distortion feature vector, by low frequency ratio, low frequency variance, low frequency differential variance , low-frequency fitting and global low-frequency ratio five features, a total of 10-dimensional vector composition; The low frequency ratio Wherein X(f) is the fast Fourier transform of each frame; The low frequency variance in The low frequency difference variance in The low-frequency fitting is to use the 6-dimensional fitting feature to fit the FFT sampling points of 0-500 Hz, and the fitting formula is Where x is the FFT sampling point from 0 to 500Hz, and a _i represents the fitting coefficient; The global low frequency ratio 5. the playback attack detection method based on the speech signal distortion characteristic that loudspeaker introduces according to claim 1 or 3, is characterized in that: described extraction is based on speech signal non-linear distortion feature vector, comprises total harmonic distortion, clipping ratio and timbre vector, a total of 16 dimensional feature vectors; The total harmonic distortion in X(f) is the fast Fourier transform of each frame, f ₁ is the pitch frequency, and f _i is i times the pitch frequency; The clipping ratio in x is the time-domain spectrum, len is the length of the time-domain spectrum; The Timbre Vector 6. The playback attack detection method based on the distortion characteristics of the speech signal introduced by the loudspeaker according to claim 1, characterized in that: the statistical feature vector in step 3 is a 26-dimensional statistical feature vector. 7. the playback attack detection method based on the speech signal distortion characteristics introduced by loudspeaker according to claim 1, characterized in that: the training speech samples described in step 4, from several devices and several recorders, including playback speech and original speech . 8. the playback attack detection method based on the speech signal distortion characteristic that speaker introduces according to claim 1 or 7, it is characterized in that: in step 4, after extracting training speech sample feature vector, utilize LIBSVM to the feature in training speech sample set The database is trained for binary classification, and the feature database is composed of feature vectors of training speech samples.