CN116721678B - Audio data monitoring method, electronic equipment and medium - Google Patents

Abstract

The present application relates to the field of computer technology, and discloses a method for monitoring audio data, an electronic device, and a medium, the method comprising: obtaining audio data to be monitored, the audio data to be monitored having a first audio format; based on the first audio format, using a preset processing method to divide the audio data to be monitored into a plurality of audio data frames; determining the channel type to which each audio data frame belongs; and printing the information of the audio data frames whose values in the audio data frames corresponding to each channel meet preset conditions into the log of the electronic device. Based on this, it can be avoided that the audio data detection result obtained by the audio data stream detection module is erroneous due to the different formats of the audio data to be monitored and the different formats processing methods.

Description

Audio data monitoring method, electronic device and medium

Technical Field

The present application relates to the field of computer technology, and in particular to a method, electronic device and medium for monitoring audio data.

Background technique

Audio data can be used as a carrier for information transmission. In life, users can use mobile phones to play audio data. For example, when listening to music, users use mobile phones to play media audio data, and when making a call, users use mobile phones to play telephone audio data. However, during the user's use, the mobile phone may be silent or the volume of the sound may be very different from the expected. Therefore, a method for monitoring audio data is needed to print out the information of the current audio data stream in real time, so that the maintenance personnel in the background can solve the problem of silence or the volume of the sound being very different from the expected according to the information of the printed audio data stream.

In the prior art, in the process of real-time printing of the monitored audio data stream, since the audio data monitoring method is developed based on a specific platform, the monitored audio data will be processed in a fixed audio data processing manner, for example, all audio data will be processed in the same manner as audio data in the PCM 32bit LE format. However, different platforms support different audio data formats, for example, there are PCM 24bit LE, PCM 16bit LE and other formats. When audio data of different formats are processed in a fixed audio data processing manner, the information of the printed audio data will be obviously wrong, for example, the size of the printed audio data is wrong, thereby affecting the efficiency of the back-end maintenance personnel in solving the problem of silence and the sound volume being very different from the expected.

Summary of the invention

The purpose of the embodiments of the present application is to provide an audio data monitoring method, electronic device and medium, so that when monitoring audio data, it can be processed in a processing method corresponding to the audio data to be monitored, thereby avoiding the situation where the audio data detection results are erroneous due to the different formats and processing methods of the audio data to be monitored, thereby improving the efficiency of back-end maintenance personnel in solving problems such as silence and sound volume being very different from expectations.

In a first aspect, an embodiment of the present application provides a method for monitoring audio data, which is applied to an electronic device, comprising:

Acquire audio data to be monitored, where the audio data to be monitored has a first audio format;

Based on the first audio format, the audio data to be monitored is divided into a plurality of audio data frames by using a preset processing method;

Determine the channel type to which each audio data frame belongs;

Information of the audio data frames corresponding to each channel whose values meet the preset conditions is printed into the log of the electronic device.

It can be understood that after the electronic device obtains the audio data to be monitored in the first audio format, the preset processing method corresponding to the first audio format is used to divide the audio data to be monitored into multiple audio data frames, which avoids the problem of using the same processing method for different audio formats, and then determines the channel type to which each audio data frame belongs, and prints the information of the audio data frame whose value meets the preset conditions in the audio data frame corresponding to each channel into the log of the electronic device. At this time, the information of the printed audio data frame is correct, so that the maintenance personnel in the background can solve the problem of the sound volume being very different from the expected or the sound being silent during the audio data playback process according to the monitoring results, and improve the efficiency of the maintenance personnel in solving the problem.

In a possible implementation of the first aspect, based on the first audio format, the audio data to be monitored is divided into a plurality of audio data frames using a preset processing method, including:

The frame size N bits of the audio data frame corresponding to the first audio format is the same as the preset data reading amount M bits of the preset processing method. From the audio data to be monitored, the audio data read each time with the preset data reading amount is taken as an audio data frame.

In a possible implementation of the first aspect, based on the first audio format, the audio data to be monitored is divided into a plurality of audio data frames using a preset processing method, including:

The frame size N bits of the audio data frame corresponding to the first audio format is greater than the preset data reading amount M bits of the preset processing method. Each time, M bits of high-order data are selected from the N bits of audio data to be monitored as a frame of audio data.

In a possible implementation of the first aspect, based on the first audio format, the audio data to be monitored is divided into a plurality of audio data frames using a preset processing method, including:

The frame size N bits of the audio data frame corresponding to the first audio format is smaller than the preset data reading amount M bits of the preset processing method, and N bits of audio data are read from the audio data to be monitored each time as one audio data frame.

In a possible implementation of the first aspect, the first audio format includes any one of PCM 16bit LE, PCM 24bitLE, and PCM 32bit LE; and

The preset data reading amount includes any one of 8 bits, 16 bits, 24 bits, and 32 bits.

In a possible implementation of the first aspect, the channel type to which the audio data frame belongs includes a left channel and a right channel; and

Printing information of the audio data frames corresponding to each channel whose values meet the preset conditions to the log of the electronic device, including:

Print information of audio data frames belonging to the left channel in the data to be monitored, whose values meet the preset conditions, into the log of the electronic device, and,

The information of the audio data frames belonging to the right channel in the data to be monitored, whose values meet the preset conditions, is printed into the log of the electronic device.

In a possible implementation of the first aspect, the channel type to which the audio data frame belongs is mono; and

Printing information of the audio data frames corresponding to each channel whose values meet the preset conditions to the log of the electronic device, including:

Information of audio data frames whose values meet preset conditions in the audio data frames is printed into a log of the electronic device.

In a possible implementation of the first aspect above, the information of the audio data frame includes an audio data frame that meets a preset condition and/or a decibel value corresponding to the audio data frame that meets the preset condition.

In a possible implementation of the first aspect, the value satisfies a preset condition including: the value in the audio data frame is a maximum value.

In a second aspect, an embodiment of the present application provides a method for monitoring audio data, which is applied to an electronic device, wherein the electronic device includes an audio stream detection module, and the audio stream detection module executes the following method, including:

Acquire audio data to be monitored, where the audio data to be monitored has a first audio format;

Based on the first audio format, the audio data to be monitored is divided into a plurality of audio data frames by using a preset processing method;

Determine the channel type to which each audio data frame belongs;

Information of the audio data frames corresponding to each channel whose values meet the preset conditions is printed into the log of the electronic device.

In a third aspect, an embodiment of the present application provides an electronic device, including:

one or more processors;

One or more memories; one or more memories store one or more programs, and when one or more programs are executed by one or more processors, the electronic device executes the audio data monitoring method described in the first aspect and any one of the various implementations of the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that instructions are stored on the storage medium, and when the instructions are executed on a computer, the computer executes the audio data monitoring method described in the first aspect and any one of the various implementations of the first aspect.

In a fifth aspect, an embodiment of the present application provides a computer program product, characterized in that the computer program product includes instructions, which, when executed, enable a computer to execute the audio data monitoring method described in the first aspect and any one of the various implementations of the first aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram showing a scenario in which a mobile phone 100 plays music according to some embodiments of the present application;

FIG2A shows a data schematic diagram of a PCM 32-bit LE format according to some embodiments of the present application;

FIG2B shows a schematic diagram of an audio data frame in PCM 32-bit LE format according to some embodiments of the present application;

FIG2C shows a schematic diagram of a PCM 32-bit LE format audio data frame corresponding to the number of channels according to some embodiments of the present application;

FIG2D shows a data schematic diagram of a PCM 16-bit LE format according to some embodiments of the present application;

FIG2E is a schematic diagram showing a result printed into a log according to some embodiments of the present application;

FIG2F shows another schematic diagram of a result printed into a log according to some embodiments of the present application;

FIG3 is a schematic diagram showing a method of converting a PCM 24-bit LE format into audio data with 16 bits per frame according to some embodiments of the present application;

FIG4A is a schematic diagram showing a flow chart of a method for monitoring audio data according to some embodiments of the present application;

FIG4B shows a schematic diagram of a PCM 16-bit LE format audio data frame corresponding to the number of channels according to some embodiments of the present application;

FIG4C is a schematic diagram showing a specific method of processing the audio data to be monitored in a preset processing manner based on the acquired format information according to some embodiments of the present application;

FIG. 4D shows a method for taking the audio data read at a preset data reading amount of 16 bits each time as an audio data frame for 24-byte PCM 16-bit LE audio data according to some embodiments of the present application;

FIG5A shows a schematic diagram of an audio system software structure according to some embodiments of the present application;

FIG5B shows an interactive schematic diagram of monitoring audio data according to some embodiments of the present application;

FIG6 shows a schematic diagram of the hardware structure of a mobile phone 100 according to an embodiment of the present application.

Detailed ways

The illustrative embodiments of the present application include, but are not limited to, a method for monitoring audio data, an electronic device, and a medium.

For ease of understanding, the terms used in this application are first introduced below.

PCM (Pulse Code Modulation): also known as pulse code modulation. PCM audio data is an uncompressed audio sampling data stream. It is a standard digital audio data converted from analog signals through sampling, quantization, and encoding.

Log: Usually a record of a certain process completed by the system or some software for future reference. It does not have a fixed format and is usually a text file that can be opened with Notepad to view the content. Of course, it may be in other formats and will be garbled if opened directly.

The technical solution of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

FIG. 1 shows a scene of a mobile phone 100 playing music according to some embodiments of the present application. In some cases, when a user uses the mobile phone 100 to play music, silence may occur suddenly. In other cases, when a user switches an audio playback device, such as switching a headset to a speaker, the volume may suddenly decrease. In order to solve the audio problems involved in the above two cases, the audio stream detection (Audio Stream Detection, ASD) module in the mobile phone 100 can obtain the audio data to be output while the mobile phone 100 outputs the audio data, and then print out the audio data to be output and save it in a log. Thus, the maintenance personnel in the background can monitor the changes in the audio data in real time according to the information of the printed audio data stream, thereby solving the problems of silence and the sound size being very different from the expected.

In some embodiments, after receiving the audio data to be monitored, the ASD module processes the audio data to be monitored in a fixed manner, for example, in a manner of processing audio data in a PCM 32bit LE format, and processes all audio data. FIG. 2A shows audio data in a PCM 32bit LE format, wherein in this format, the data size of each frame of audio data is 32bit. Moreover, the data size of the audio data shown in FIG. 2A is 48 bytes, and the number of channels is 2. When the audio data is processed in the processing format of PCM 32bit LE, each time 32bit of data is read from the audio data to be monitored, the data shown in FIG. 2A needs to be read 12 times in total, and finally the audio data frame 0, audio data frame 1, ..., audio data frame 11, i.e., 12 frames of audio data, are obtained as shown in FIG. 2B.

Then, according to the number of channels being 2, the left and right channels corresponding to the 12 frames of audio data can be determined. For example, as shown in FIG2C , the audio data frame 0 corresponds to the left channel as L0, the audio data frame 1 corresponds to the right channel as R1, ..., the audio data frame 10 corresponds to the left channel as L10, and the audio data frame 11 corresponds to the right channel as R11.

Since the maximum value of the audio data of each channel can represent the size of all the audio data of the channel, the maximum audio data value can be found from the audio data of the left channel, i.e., L0, L2...L10, and converted into decibel values for printing, and the maximum audio data volume can be found from the audio data of the right channel, i.e., R1,...R11, and converted into decibel values for printing, so that the maintenance personnel can more directly observe the size changes of the audio data according to the decibel values of the audio data. It needs to be explained that when the maximum audio data value printed is 0, it means that there is no sound at this time, and the decibel value printed out of the audio data under normal circumstances should be within the range of -100 to 0dB.

However, the audio data to be monitored may have different formats on different mobile phone platforms, or audio data of different formats may exist on the same platform. For example, there are formats such as PCM 24bit LE and PCM 16bit LE. However, audio data of different formats will be processed in the same format. For example, if the format of the audio data to be monitored is PCM 16bit LE, then each 16 bits is a frame of audio data, the total amount of data is 24 bytes, and the number of channels is 2. In the specific process of processing the audio data, if it is still processed according to the PCM 32bit LE format, that is, each time 32bit audio data is read from the audio data in the PCM 16bit LE format, the data of two audio frames will be read as the data of one audio frame each time, which will cause the read data to be erroneous.

For example, when processing the audio data of the PCM 16bit LE format as shown in Figure 2 D, read the data of 32bit from the audio data to be monitored at each time, can read the data of two 16bit audio data frames as an audio frame at each time, thereby read error.Then, find the maximum audio data value in the left and right channels respectively from the data read, and convert it into decibel value and print it out, the print result can be incorrect.For example, Fig. 2 E shows a kind of audio data of processing PCM 16bit LE with the audio format of PCM 32bit LE, the result printed out, for example in frame 2001a, the maximum audio data value in left channel " L " is " 8388485 ", corresponding decibel value is " 48.164672db ", the maximum audio data value in right channel " R " is " 8388608 ", corresponding decibel value is " 48.16799db ". It is understandable that the maximum decimal value corresponding to 16-bit data is 32767, while "8388485" and "8388608" are greater than 32767, exceeding the maximum range of data that can be represented by 16 bits, and the decibel values corresponding to the left and right channels exceed the range of -100 to 0dB, so the printed value is wrong and does not actually reflect the size of the current audio data, which affects the background maintenance personnel in solving the silent problem or solving the problem of changes in the volume of the sound played when switching devices without adjusting the volume.

In order to solve the above problems, the present application proposes a method for monitoring audio data. While obtaining the audio data to be monitored, the format information of the audio data to be monitored is also obtained, and then the audio data to be monitored is processed using a format processing method corresponding to the obtained format, thereby avoiding the problem of incorrect numerical values in the size of the printed audio data due to inappropriate format processing method.

For example, after obtaining the audio data to be monitored as shown in Figure 2D and the corresponding information, including the format, the number of channels and the total amount of data information, the audio data to be monitored is processed in a format processing method corresponding to the PCM 16bit LE format, and the audio data of 16bit size is read from the audio data to be monitored each time, and a total of 12 times are read. Then, according to the number of channels 2, the maximum audio data values in the left and right channels are found respectively, and the maximum audio data values found are converted into decibel values and printed out in real time. For example, Figure 2F shows a result printed out after processing the audio data of PCM 16bit LE in the audio format of PCM 16bit LE. For example, in box 2002, the maximum audio data in the left channel "L" is "1", and the corresponding decibel value is "-90.308999db"; the maximum audio data in the right channel "R" is also "1", and the corresponding decibel value is also "-90.308999db". It can be understood that the values of the five sets of data printed out in FIG. 2F and the audio data printed out in FIG. 2E are different and vary greatly.

In addition, a data reading amount can be set, and the size of each frame of data corresponding to the obtained audio data format is compared with the preset data reading amount. If the size of each frame of data is larger than the preset data reading amount, when the obtained audio data is divided into multiple audio frames, the high-order data corresponding to the preset data reading amount in each frame of audio data is intercepted to obtain an audio data frame corresponding to the preset data reading amount. Then, the audio data value that meets the preset conditions in the corresponding channel is found according to the number of channels, and the audio data information that meets the preset conditions is printed out in real time. This processing method makes the audio data values of different formats within a certain range after processing, so as to be printed uniformly. Among them, the preset conditions include any one of the maximum value, the second largest value or the average value.

For example, taking the preset condition as the maximum value as an example, if the preset data reading amount is 16 bits, if the format of the currently acquired audio data to be monitored is PCM 24bit LE, the number of channels is 2, and the total amount of data is 36 bytes, then it can be determined that the data size of each frame of the acquired audio data to be monitored is greater than the preset data reading amount. First, according to the acquired audio data format PCM24bit LE, 16 bits of each 24bit data in the audio data to be monitored are intercepted as new audio data frames. The data in Figure 3 intercepts a total of 12 frames of 16bit audio data, which are audio data 0', audio data 1'...audio data 11'. Then, according to the number of channels 2, the channels corresponding to each audio data can be determined, and L0', R1'...R11' are obtained, and then the maximum values in the left and right channels are determined respectively, and the maximum value and the decibel value converted from the maximum value are printed out.

It is understandable that using different processing methods for audio data in different formats can avoid a certain platform using a specific format, which may cause errors in the printed audio data. This can improve the ability of back-end maintenance personnel to solve problems such as the sound volume being very different from expectations or silence during audio data playback based on monitoring results, thereby improving the efficiency of maintenance personnel in solving problems.

It can be understood that the technical solution of the present application can be applicable to different platforms, for example, HiSilicon platform ^TM , MTK ^TM (MediaTek) platform, Qualcomm ^TM platform, TI ^TM (Tencent Cloud TI Platform), Motorola ^TM , Philips ^TM , ADI ^TM (Analog Devices, Inc.), Spreadtrum ^TM , Infineon ^TM , Kaiming ^TM , etc.

It is understandable that the technical solution of the present application is applicable not only to the mobile phones mentioned above, but also to other electronic devices, for example, tablet computers, smart screens, augmented reality (AR)/virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. It is understandable that the embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

FIG4A shows a flowchart of a technical method for processing the audio data to be monitored in a processing method corresponding to the received format according to some embodiments of the present application. With the ASD module as the execution body, the method includes:

S401, obtaining audio data to be monitored and corresponding information, including format, number of channels and total amount of data.

For example, the format of the acquired audio data to be monitored is PCM 24bit LE, the number of channels is 2, and the total amount of data is 36 bytes.

For another example, the format of the acquired audio data to be monitored is PCM 16bit LE, the number of channels is 2, and the total amount of data is 24 bytes.

S402: Based on the acquired format, the audio data to be monitored is divided into a plurality of audio data frames by using a preset processing method.

It can be understood that the preset processing method is determined based on the preset data reading amount M bits. Different processing methods can be set according to the different size relationship between the frame size N bits of the audio data frame corresponding to the acquired format and the preset data reading amount M bits in the preset processing method. For example, when the frame size N bits of the audio data frame corresponding to the acquired format is the same as the preset data reading amount M bits of the preset processing method, the audio data read each time with the preset data reading amount is determined from the audio data to be monitored as a frame of audio data frame; it is also possible that when the frame size N bits of the audio data frame corresponding to the acquired format is greater than the preset data reading amount M bits of the preset processing method, each time from the N bits of the audio data to be monitored, M bits of high-order data are selected as a frame of audio data frame; it is also possible that the frame size N bits of the audio data frame corresponding to the first audio format is less than the preset data reading amount M bits of the preset processing method, and N bits of audio data are read from the audio data to be monitored each time as a frame of audio data frame.

The specific execution process of the above-mentioned S402 example will be specifically described in FIG. 4C below, and will not be repeated here.

S403: Determine the audio data corresponding to each channel according to the number of channels.

It can be understood that when the number of channels and the number of audio frames are known, the arrangement rule of the audio data frames can be known, so that the position of the audio data frames of the corresponding channels can be determined. For example, if the number of channels is 2, the channels include the left channel and the right channel, and the arrangement rule of the audio data frames is that the left channel and the right channel are arranged alternately in sequence. Therefore, according to the number of channels, the position of the audio data frame of the left channel and the position of the audio data frame of the right channel can be known, so as to determine the audio data corresponding to each channel. If the number of channels is 1, it is mono.

For example, taking the case where the frame size N bits of the audio data frame corresponding to the acquired format is the same as the preset data reading amount M bits of the preset processing method, Figure 4B shows that after the PCM 16bit LE audio data is divided into 12 audio data frames using the preset processing method, the audio data frames corresponding to each channel are determined according to the channel number 2, audio data 0 corresponds to the left channel as L0, audio data 1 corresponds to the right channel as R1..., and audio data 11 corresponds to the right channel as R11.

S404, finding the maximum audio data value in the corresponding channel, and printing out the found maximum value and the decibel value converted from the maximum value.

When the audio data is dual-channel, find the maximum audio data value in the left and right channels respectively, and print out the maximum value found and the decibel value converted from the maximum value; when the audio data is monophonic, find the maximum audio data value in the audio data, and print out the maximum value found and the decibel value converted from the maximum value.

For example, find out that the maximum audio data value of L0, L2, ..., L10 in the left channel shown in Figure 4B is L10, and print out the maximum value L10 and the decibel value converted from the maximum value L10; find out that the maximum audio data value of R1, R3, ..., R11 in the right channel shown in Figure 4B is R3, and print out the maximum value R3 and the decibel value converted from the maximum value R3.

It can be understood that the execution order of the above steps S401 to S404 is only an example. In other embodiments, other execution orders may be adopted, and some steps may be split or combined, which is not limited here.

FIG4C shows a specific method of dividing the monitored audio data into multiple audio data frames using a preset processing method based on the acquired format information according to some embodiments of the present application. It can be understood that the aforementioned method is a specific description of some embodiments in the above S402, and the method includes:

S4021, determining whether the frame size N bits of the audio data frame corresponding to the acquired format is the same as the preset data reading amount M bits of the preset processing method.

It can be understood that when the frame size N bits of the audio data frame corresponding to the acquired format is the same as the preset data reading amount M bits of the preset processing method, the preset data reading amount M bits can be directly used as the data amount read each time, thereby entering S4022; otherwise, enter S4023, continue to judge the size relationship between the frame size N bits of the audio data frame corresponding to the acquired format and the preset data reading amount M bits of the preset processing method, and then process according to the corresponding processing method.

It can be understood that the preset data reading amount M bits can be 8 bits, 16 bits, 24 bits, 32 bits, etc., which will not be repeated here.

For example, the preset data reading amount M bits is 16 bits, and the format of the acquired audio data to be detected is PCM16bit LE, then the frame size N bits of the audio data frame corresponding to the acquired format is 16 bits, and it can be judged that the frame size N bits of the audio data frame corresponding to the acquired format is the same as the preset data reading amount M bits of the preset processing method, and the process goes to S4022; for another example, the format of the acquired audio data to be detected is PCM 8bit LE or PCM 24bit LE, then the frame size N bits of the audio data frame corresponding to the acquired format are 8bit and 24bit respectively, and the frame size N bits of the audio data frame corresponding to the acquired format are different from the preset data reading amount M bits of the preset processing method, and the process goes to S4023.

S4022: From the audio data to be monitored, audio data read each time with a preset data reading amount is taken as an audio data frame.

For example, as shown in FIG. 4D , for 24-byte PCM 16-bit LE audio data, the audio data read each time with a preset data reading amount of 16 bits is taken as one audio data frame.

S4023, determining whether the frame size N bits of the audio data frame corresponding to the acquired format is greater than the preset data reading amount M bits of the preset processing method.

It can be understood that when the frame size N bits of the audio data frame corresponding to the acquired format is greater than the preset data reading amount M bits of the preset processing method, since the accuracy of the audio data values viewed by maintenance personnel does not need to be particularly high, S4025 can be entered, and each time from the N bits of audio data to be monitored, M bits of high-order data are selected as a frame of audio data frame; otherwise, S4024 is entered, and N bits of audio data are read from the audio data to be monitored each time as a frame of audio data frame.

For example, when the frame size N bits of the audio data frame corresponding to the acquired format is 24 bits, which is greater than the preset data reading amount M bits of the preset processing method, which is 16 bits, then enter S4025; for another example, when the frame size N bits of the audio data frame corresponding to the acquired format is 8 bits, which is less than the preset data reading amount M bits of the preset processing method, which is 16 bits, then enter S4024.

S4024, reading N bits of audio data from the audio data to be monitored each time as one frame of audio data.

For example, for PCM 8-bit LE audio data, 8-bit audio data is read from the audio data to be monitored each time as one audio data frame.

S4025, each time, select M bits of high-order data from the N bits of audio data to be monitored as a frame of audio data.

For example, for the PCM 24-bit LE audio data to be monitored, as shown in FIG3 , each time, high 16-bit data is selected from the 24-bit data as a frame of audio data.

It can be understood that the execution order of the above steps S4021 to S4025 is only an example. In other embodiments, other execution orders may be adopted, and some steps may be split or combined, which is not limited here.

Further, to facilitate a deeper understanding of the technical solutions provided by the embodiments of the present application, FIG5A shows a schematic diagram of the audio system software structure of an electronic device according to some embodiments of the present application. As shown in FIG5A , the audio system structure is a multi-layer structure, which includes, from top to bottom, an audio application layer 01, an audio framework layer 02, and an audio hardware abstraction layer 03.

In some embodiments of the present application, the audio application layer 01 includes application software with audio playback functions, such as music, media players, etc. (not shown in the figure), and audio playback function modules that can be used for application software to play audio data, such as audio tracks (AudioTrack) 011, media players (Media Player) 012, etc. Specifically, audio tracks (AudioTrack) 011 can be used to manage and play PCM data raw streams, and other audio formats (such as AAC, AMR, OPUS, FLAC, etc.) need to be decoded into PCM data raw stream formats before they can be used. Media Player (Media Player) 012 can play audio format data such as MP3, WAV, OGG, AAC, MIDI, etc.

The audio framework layer 02 can call the audio function library module (Audio Lib) to play audio data, for example, the audio mixer (AudioMixer) module 022, and the audio cache module 021 for storing the audio data to be played transmitted by the received audio application layer 01. Among them, the audio cache module 021 is used to store audio data as an interactive channel between the audio application layer 01 and the audio framework layer 02. The audio mixer (AudioMixer) module 022 serves as an interactive channel between the audio framework layer 02 and the audio hardware abstraction layer 03, and can mix and process multiple audio stream data sent to the audio framework layer 02 through the audio track 012.

The audio hardware abstraction layer 03 is a layer that is independently encapsulated to adapt to different hardware, so as to output the audio data to be played. The audio hardware abstraction layer 03 shown in the figure includes an audio output module 031 and an audio stream detection (Audio Stream Detection, ASD) module 032. Among them, the audio output module 031 is used to receive the audio data to be played sent by the audio framework layer 02, and output the audio data to be played. The audio stream detection (Audio Stream Detection, ASD) module 032 is used to print out the audio data to be played, so as to monitor the audio data to be played in real time, and then assist the maintenance personnel to deal with the problem.

It is understandable that in the process of monitoring audio data, the audio data to be played described in Figure 5A is the audio data to be monitored. The arrow in Figure 5A shows, in some embodiments, the flow relationship of the audio data to be monitored. The audio track (AudioTrack) 011 sends the audio data to be monitored to the audio cache module 021. After receiving, the audio cache module 021 sends the audio data to be monitored to the audio mixing module 022 for mixing. The audio mixing module 022 sends the processed audio data to the audio output module 031 for subsequent output. After receiving the audio data to be monitored, the audio output module 031 outputs and sends the audio data to be monitored to the audio stream detection module 032. The audio stream detection module 032 processes the received audio data to be monitored, thereby successfully printing it out.

It can be understood that the audio system software structure shown in FIG. 5A is only an example. In other embodiments, the electronic device may also adopt other forms of audio system software structure, which is not limited in the embodiments of the present application.

The following further describes the solution of the present application in conjunction with the modules in the audio system software structure shown in FIG5A. As shown in FIG5B, the audio output module 031 sends the audio data to be monitored to the audio stream detection module 032 while outputting the audio data to be monitored, and then the audio stream detection module 032 prints out the audio data to be monitored. The specific interaction process is as follows:

S501 , the audio output module 031 sends an initialization thread instruction to the audio stream detection module 032 .

It can be understood that the audio stream detection module 032 needs to start a thread to complete the task of processing the audio data to be monitored and printing it out.

S502, the audio stream detection module 032 initializes a thread.

After receiving the instruction to initialize the thread, the audio stream detection module 032 mounts the interface and then creates a thread.

S503, the audio output module 031 sends an instruction to the audio stream detection module 032 to start processing the audio data to be monitored.

It is understandable that the audio stream detection module 032 needs to prepare for the process of printing the audio data, such as preparing storage space, etc. Therefore, the audio output module 031 needs to send an instruction to the audio stream detection module 032 to start processing the audio data to be monitored.

S504: the audio stream detection module 032 prepares to process the audio data to be monitored.

It is understandable that after receiving the instruction to process the audio data to be monitored sent by the audio output module 031, the audio stream detection module 032 will prepare for processing the audio data to be monitored, such as preparing storage space.

S505 , the audio output module 031 sends the audio data to be monitored and corresponding information, including the format, the number of channels and the total amount of data, to the audio stream detection module 032 .

It can be understood that the audio output module 031 has sent an instruction to the audio stream detection module 032 to start processing the audio data to be monitored, so the audio output module 031 can send the audio data to be monitored and corresponding information to the audio stream detection module 032, including the format, number of channels and total amount of data.

For example, the audio output module 031 sends the audio data to be monitored, which has a format of PCM 24bit LE, a total data amount of 24 bytes, and a number of channels of 2, to the audio stream detection module 032 .

S506: The audio stream detection module 032 receives the audio data to be monitored and the corresponding information.

S507: The audio stream detection module 032 divides the audio data to be monitored into multiple audio data frames based on the acquired format using a preset processing method.

S508: The audio stream detection module 032 determines the audio data corresponding to each channel according to the number of channels.

S509, the audio stream detection module 032 prints out the maximum value and the decibel value converted from the maximum value according to the maximum audio data value found in the corresponding channel.

It can be understood that the manner in which the maximum value of the corresponding channel in the audio data to be monitored is printed out in S506 to S509 is the same as the process of S401 to S405 described in FIG. 4A and FIG. 4C , and thus will not be described in detail here.

S510, the audio output module 031 sends an instruction to the audio stream detection module 032 to stop processing audio data.

It is understandable that the process of the audio stream detection module 032 printing the audio data to be monitored is a real-time process, and the audio output module 031 will continuously send the audio data to be monitored to the audio stream detection module 032. Therefore, when the audio output module 031 receives the end of outputting audio data, or receives the need to stop using the audio stream detection module 032 to monitor the output audio data, it can send an instruction to stop monitoring the audio data to the audio stream detection module 032, thereby terminating the thread processing process and saving resources.

It can be understood that the execution order of the above steps S501 to S510 is only an example. In other embodiments, other execution orders may be adopted, and some steps may be split or combined, which is not limited here.

Fig. 6 shows a schematic diagram of the hardware structure of a mobile phone 100 according to an embodiment of the present application. The schematic diagram of the hardware structure of the mobile phone 100 shown in Fig. 6 may be a schematic diagram of the hardware structure of the mobile phone 100.

The mobile phone 100 can execute the audio data monitoring method provided in the embodiment of the present application. As shown in Figure 6, the mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (Subscriber Identification Module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the mobile phone 100. In other embodiments of the present application, the mobile phone 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a microprocessor (MCU), an AI (Artificial Intelligence) processor, or a processing module or processing circuit of a programmable logic device (FPGA). Different processing units may be independent devices or integrated in one or more processors.

The memory can be used to store data, software programs and modules, and can be a volatile memory (Volatile Memory), such as a random-access memory (Random-Access Memory, RAM); or a non-volatile memory (Non-Volatile Memory), such as a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, HDD) or a solid-state drive (Solid-State Drive, SSD); or a combination of the above types of memory, or it can also be a removable storage medium, such as a secure digital (Secure Digital, SD) memory card.

The charging management module 140 may include a battery 142, a power management module 141, etc. The power management module 141 is used to manage the charging of the power source and the power supply of the power source to other modules. The charging management module 140 is used to receive charging input from the charger; the power management module 141 is used to connect the battery 142, the charging management module and the processor 110.

The mobile communication module 150 may include, but is not limited to, an antenna, a power amplifier, a filter, a low noise amplifier (LNA), etc. The mobile communication module 150 may provide solutions for wireless communications including 2G/3G/4G/5G applied to the mobile phone 100. The mobile communication module 150 may receive electromagnetic waves by an antenna, filter, amplify, and process the received electromagnetic waves, and transmit them to a modulation and demodulation processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be arranged in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be arranged in the same device as at least some of the modules of the processor 110.

The wireless communication module 160 may include an antenna, and transmit and receive electromagnetic waves via the antenna. The wireless communication module 160 may provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc., which are applied to the mobile phone 100. The mobile phone 100 may communicate with the network and other devices through wireless communication technology.

In some embodiments, the mobile communication module 150 and the wireless communication module 160 of the mobile phone 100 may also be located in the same module.

The camera 193 is used to capture still images or videos. The optical image generated by the lens is projected onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP (Image Signal Processor) to be converted into a digital image signal. The mobile phone 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU (Graphic Processing Unit), the display screen and the application processor. Among them, there can be N cameras, where N is a positive integer.

The display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a Micro LED, a Micro OLED, a quantum dot light-emitting diode (QLED), etc. The number of display screens may be N, where N is a positive integer.

The audio module 170 can convert digital audio information into analog audio signal output, or convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110. Among them, the audio module 170 can include a speaker 170A, a receiver 170B, a microphone 170C, and an earphone interface 170D.

The interface module includes an external memory interface 120, a Universal Serial Bus (USB) interface 130, and a Subscriber Identification Module (SIM) card interface 195, etc. Among them, there can be N SIM card interfaces, where N is a positive integer. The external memory interface can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of the mobile phone 100. The external memory card communicates with the processor 110 via the external memory interface to implement a data storage function. The universal serial bus interface is used for the mobile phone 100 to communicate with other mobile phones. The Subscriber Identification Module card interface is used to communicate with a SIM card installed in the mobile phone 100, for example, to read a phone number stored in the SIM card, or to write a phone number into the SIM card.

In some embodiments, the mobile phone 100 further includes a button 190, a motor 191, and an indicator 192. The button may include a volume button, a power on/off button, etc. The motor is used to make the mobile phone 100 vibrate, for example, when the user's mobile phone 100 receives a new IM APP message, vibration is generated to prompt the user to check the message. The indicator may include a laser indicator, a radio frequency indicator, an LED indicator, etc.

It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the mobile phone 100. In other embodiments of the present application, the mobile phone 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and/or storage elements), at least one input device and at least one output device.

Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Therefore, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.

In the accompanying drawings, some structural or method features may be shown in a specific arrangement and/or order. However, it should be understood that such a specific arrangement and/or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and/or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

It should be noted that the units/modules mentioned in the various device embodiments of the present application are all logical units/modules. Physically, a logical unit/module can be a physical unit/module, or a part of a physical unit/module, or can be implemented as a combination of multiple physical units/modules. The physical implementation method of these logical units/modules themselves is not the most important. The combination of functions implemented by these logical units/modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units/modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units/modules in the above-mentioned device embodiments.

It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims (11)

1. A method for monitoring audio data is applied to electronic equipment and is characterized by comprising the following steps of,

Acquiring audio data to be monitored, wherein the audio data to be monitored has a first audio format;

Based on the first audio format, dividing the audio data to be monitored into a plurality of audio data frames by adopting a preset processing mode;

determining the channel type of each audio data frame;

Printing information of audio data frames, of which the numerical value meets a preset condition, in the audio data frames corresponding to each sound channel into a log of the electronic equipment, wherein the numerical value meets the preset condition and comprises any one of the numerical value being the maximum value, the next-largest value or the average value;

And the audio data to be monitored is divided into a plurality of audio data frames by adopting a preset processing mode based on the first audio format, comprising,

The frame size N bits of the audio data frame corresponding to the first audio format is smaller than the preset data reading amount M bits of the preset processing mode, and N bits of audio data are read from the audio data to be monitored each time to serve as a frame of audio data frame;

And selecting M-bit high-order data from the N-bit data of the audio data to be monitored each time as a frame of audio data frame, wherein the frame size N bits of the audio data frame corresponding to the first audio format is larger than the preset data reading amount M bits of the preset processing mode, and the preset data reading amount M meets the corresponding first precision when the audio data frame is printed.

2. The method of claim 1, wherein dividing the audio data to be monitored into a plurality of audio data frames based on the first audio format using a predetermined processing scheme comprises,

And taking the audio data read by the preset data reading amount each time as a frame of audio data frame from the audio data to be monitored.

3. The method according to any of claims 1-2, wherein the first audio format comprises any of PCM 16bit LE, PCM 24bit LE, PCM 32bit LE; and

The preset data reading quantity comprises any one of 8 bits, 16 bits, 24 bits and 32 bits.

4. The method of claim 1, wherein the channel types to which the frame of audio data belongs include a left channel and a right channel; and

The printing the information of the audio data frames with the digital values meeting the preset conditions in the audio data frames corresponding to each sound channel into the log of the electronic equipment comprises the following steps:

Printing information of audio data frames with digital values meeting preset conditions in the audio data frames belonging to the left channel in the data to be monitored into a log of the electronic equipment, and

And printing information of the audio data frames with the numerical value meeting the preset condition in the audio data frames belonging to the right channel in the data to be monitored into a log of the electronic equipment.

5. The method of claim 1, wherein the channel type to which the frame of audio data belongs is mono; and

The printing the information of the audio data frames with the digital values meeting the preset conditions in the audio data frames corresponding to each sound channel into the log of the electronic equipment comprises the following steps:

and printing information of the audio data frames with the digital values meeting preset conditions in the audio data frames into a log of the electronic equipment.

6. The method according to claim 4 or 5, wherein the information of the audio data frame includes a decibel value corresponding to the audio data frame satisfying the preset condition and/or the audio data frame satisfying the preset condition.

7. The method according to claim 4 or 5, wherein the value satisfying a preset condition comprises: the value in the frame of audio data is a maximum value.

8. The audio data monitoring method is applied to electronic equipment, the electronic equipment comprises an audio stream detection module, and the audio stream detection module performs the following steps:

Acquiring audio data to be monitored, wherein the audio data to be monitored has a first audio format;

Based on the first audio format, dividing the audio data to be monitored into a plurality of audio data frames by adopting a preset processing mode;

determining the channel type of each audio data frame;

Printing information of audio data frames, of which the numerical value meets a preset condition, in the audio data frames corresponding to each sound channel into a log of the electronic equipment, wherein the numerical value meets the preset condition and comprises any one of the numerical value being the maximum value, the next-largest value or the average value;

And the audio data to be monitored is divided into a plurality of audio data frames by adopting a preset processing mode based on the first audio format, comprising,

The frame size N bits of the audio data frame corresponding to the first audio format is smaller than the preset data reading amount M bits of the preset processing mode, and N bits of audio data are read from the audio data to be monitored each time to serve as a frame of audio data frame;

And selecting M-bit high-order data from the N-bit data of the audio data to be monitored each time as a frame of audio data frame, wherein the frame size N bits of the audio data frame corresponding to the first audio format is larger than the preset data reading amount M bits of the preset processing mode, and the preset data reading amount M meets the corresponding first precision when the audio data frame is printed.

9. An electronic device, comprising:

one or more processors;

One or more memories; the one or more memories stores one or more programs that, when executed by the one or more processors, cause the electronic device to perform the method of monitoring audio data of any of claims 1-7.

10. A computer readable storage medium having stored thereon instructions which, when executed on a computer, cause the computer to perform the method of monitoring audio data according to any of claims 1 to 7.

11. A computer program product, characterized in that the computer program product comprises instructions which, when executed, cause a computer to perform the method of monitoring audio data according to any of claims 1 to 7.