CN109669783B - Data processing method and device - Google Patents

Abstract

Embodiments of the present invention provide a data processing method and device. The method includes: receiving multiple channels of image data sent by the hardware abstraction layer; selecting at least one channel of first image data from the multiple channels of image data; and processing the data according to the set method. Process the selected at least one first image data. Implementing the processing of multi-channel image data in the application framework layer reduces the difficulty of application development. Moreover, in the application framework layer, for a certain data processing method, only the best channel is selected from the multi-channel image data. Appropriate image data is processed in this data processing method, rather than all multi-channel image data is processed in this data processing method, thereby reducing data processing costs, that is, reducing consumption of processing resources.

Description

Data processing methods and equipment

Technical field

The present invention relates to the technical field of operating systems, and in particular, to a data processing method and device.

Background technique

Terminal devices such as mobile phones generally provide a camera function, and there are also some applications in the terminal device that use the data collected by the camera, such as video call applications, image processing applications, etc.

Generally speaking, when a camera is used, it will output multiple channels of data at the same time, such as preview, photo, video, callback and other image data. Applications that need to use image data often need to perform various processing on multiple channels of data at the same time: For example, the three channels of image data including preview, photo taking, and video recording are processed such as beautification, filters, and specific object recognition. For example, adding a beauty effect to the preview data, and hoping that the effect will also be synchronized to the photo data.

Currently, the processing of image data in operating systems such as Android generally involves the application layer (Application), application framework layer (Application Framework), hardware abstraction layer (HardwareAbstraction Layer), etc. The processing process can be simply described as: the hardware abstraction layer transmits the multi-channel image data collected by the image to the application framework layer. Therefore, as shown in Figure 1, the application framework layer sends the multi-channel image data separately through independent interfaces. To the application program, the application program repeatedly processes each channel of image data. For example: assuming that the application program needs to perform beautification processing on the received image data, and assuming that two channels of image data are received, the application program needs to process the two channels of image data. Each channel of image data is processed separately for beautification, which will exponentially increase resource consumption. For example, it will exponentially increase CPU operation and bandwidth consumption, and power consumption will also be greatly increased. In addition, in order to realize the application's various processing of multi-channel image data and ensure the synchronization of the processing of multi-channel image data, the application needs to develop complex software solutions to ensure the parallel processing of multi-channel image data, and to ensure the processing of images. Data cannot affect the processing of important threads such as UI, which makes application development more difficult.

Contents of the invention

In view of this, embodiments of the present invention provide a data processing method and device to reduce image data processing costs and application development difficulty.

In a first aspect, embodiments of the present invention provide a data processing method, applied in the application framework layer, including:

Receive multi-channel image data sent by the hardware abstraction layer;

Select at least one channel of first image data from the multiple channels of image data;

The selected at least one channel of first image data is processed according to the set data processing method.

In a second aspect, embodiments of the present invention provide a data processing device applied in an application framework layer, including:

The receiving module is used to receive multi-channel image data sent by the hardware abstraction layer;

A selection module configured to select at least one channel of first image data from the multiple channels of image data;

A processing module, configured to process the selected at least one channel of first image data according to a set data processing method.

In a third aspect, embodiments of the present invention provide an electronic device. The electronic device can be implemented as a user terminal device, including a processor and a memory. The memory is used to store information that supports the user terminal device in executing the data processing method in the first aspect. Program, the processor is configured to execute the program stored in the memory. The electronic device may also include a communication interface for communicating with other devices or communication networks.

In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the electronic device, which includes programs for executing the data processing method in the first aspect.

In the fourth aspect, embodiments of the present invention provide a data processing method applied in an application program, including:

Send the data processing method to the application framework layer, so that the application framework layer processes at least one channel of image data among the multiple channels of image data received from the hardware abstraction layer according to the data processing method;

Receive processed image data sent by the application framework layer.

In a fifth aspect, embodiments of the present invention provide a data processing device applied in an application program, including:

A sending module, configured to send the data processing method to the application framework layer, so that the application framework layer processes at least one of the multiple channels of image data according to the data processing method, and the multiple channels of image data are Sent by the hardware abstraction layer to the application framework layer;

A receiving module, configured to receive processed image data sent by the application framework layer.

In a sixth aspect, embodiments of the present invention provide an electronic device. The electronic device can be implemented as a user terminal device, such as a smartphone, and includes a processor and a memory. The memory is used to store and support the electronic device in executing the fourth aspect. The processor is configured to execute the program stored in the memory. The electronic device may also include a communication interface for communicating with other devices or communication networks.

In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the electronic device, which includes programs for executing the data processing method in the fourth aspect.

In the data processing method and device provided by the embodiments of the present invention, the processing of multi-channel image data is implemented at the application framework layer. Specifically, optionally, the application program located at the application layer can send the data processing method for the multi-channel image data to the application framework layer, so that the application framework layer receives the multi-channel image data sent by the hardware abstraction layer. Finally, at least one channel of first image data is selected from the multiple channels of image data, and then the at least one channel of first image data is processed in the data processing method. Through this solution, the processing of multi-channel image data is realized in the application framework layer, which reduces the difficulty of application development. Moreover, in the application framework layer, for a certain data processing method, not all multi-channel image data can be processed. This data processing method is performed, and for example, only the most suitable image data is selected from multiple channels of image data to be processed in this data processing method, thereby reducing the data processing cost, that is, reducing the consumption of processing resources.

Description of the drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic diagram of the principle of image data processing provided by the prior art;

Figure 2 is a schematic diagram of the principle of a data processing method provided by an embodiment of the present invention;

Figure 3 is a flow chart of a data processing method provided by an embodiment of the present invention;

Figure 4 is a flow chart of another data processing method provided by an embodiment of the present invention;

Figure 5 is an interaction diagram of a data processing method provided by an embodiment of the present invention;

Figure 6 is a schematic structural diagram of a data processing device corresponding to the embodiment shown in Figure 3;

Figure 7 is a schematic structural diagram of an electronic device corresponding to the data processing device shown in Figure 6;

Figure 8 is a schematic structural diagram of a data processing device corresponding to the embodiment shown in Figure 4;

FIG. 9 is a schematic structural diagram of electronic equipment corresponding to the data processing device shown in FIG. 8 .

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present invention.

The terminology used in the embodiments of the present invention is only for the purpose of describing specific embodiments and is not intended to limit the present invention. As used in this embodiment and the appended claims, the singular forms "a," "the" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. Generally, at least two are included, but at least one is not excluded.

It should be understood that the term "and/or" used in this article is only an association relationship describing related objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A alone exists, and A and A exist simultaneously. B, there are three situations of B alone. In addition, the character "/" in this article generally indicates that the related objects are an "or" relationship.

It should be understood that although the terms first, second, third, etc. may be used to describe XXX in the embodiments of the present invention, these XXX should not be limited to these terms. These terms are only used to distinguish XXX. For example, without departing from the scope of the embodiments of the present invention, the first XXX may also be called the second XXX, and similarly, the second XXX may also be called the first XXX.

Depending on the context, the words "if" or "if" as used herein may be interpreted as "when" or "when" or "in response to determination" or "in response to detection." Similarly, depending on the context, the phrase "if determined" or "if (stated condition or event) is detected" may be interpreted as "when determined" or "in response to determining" or "when (stated condition or event) is detected )" or "in response to detecting (a stated condition or event)".

It should also be noted that the terms "includes", "includes" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a good or system including a list of elements includes not only those elements but also those not expressly listed other elements, or elements inherent to the product or system. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of other identical elements in the goods or systems that include the stated element.

In addition, the sequence of steps in the following method embodiments is only an example and is not strictly limited.

FIG. 2 is a schematic diagram of the principle of a data processing method provided by an embodiment of the present invention, which is used to perform the same processing on multiple channels of image data. As shown in Figure 2, the processing of image data in operating systems such as Android generally involves the application layer, application framework layer, hardware abstraction layer, etc. The data processing method provided by the embodiment of the present invention implements the processing of multi-channel image data at the application framework layer to reduce the difficulty of application development and avoid developing relevant processing logic in the application; and, optionally, When it is necessary to perform one or more types of processing on multiple channels of image data, for each type of processing, you only need to select the most appropriate channel of image data from the multiple channels of image data to perform the processing, and the processing results are synchronized to or Multiplexing to other channels of image data completes the parallel processing of multiple channels of image data, thereby overcoming the problem of increasing processing costs due to the need to repeatedly perform the same processing on multiple channels of image data, and reducing CPU computing costs, power consumption, and bandwidth. and other costs.

It is worth noting that the above-mentioned multiple channels of image data can be considered to be generated simultaneously. The difference mainly lies in the image resolution of each channel of image data. For example, the image resolution corresponding to the preview image data is smaller than the image resolution corresponding to the photographed image data. In addition, taking the more common photographing scenes in practical applications as an example, during the actual photographing process, not only the photographed image data is generated, but the preview image data is also generated at the same time. The multi-channel image data generated simultaneously can be generated directly by hardware or by software.

Optionally, the application framework layer can include three components: a controller, a selector, and a synthesizer. These three components work together to process the multi-channel image data received from the hardware abstraction layer, and can combine the multiple channels of image data. The processing results of the image data (for example: video image data, photographed image data, preview image data) are fed back to the application program. Among them, these three components can be understood as three functional modules in the application framework layer, each with different processing logic. The following is a description of these three components in the application framework layer in conjunction with the embodiment shown in Figure 3. How to cooperate with each other to complete the processing of multi-channel image data is explained.

It is worth noting that the above three components are only an example of artificial division for easy understanding, and are not strictly limited to the fact that the above three components must exist in the actual product form.

FIG. 3 is a flow chart of a data processing method provided by an embodiment of the present invention. The data processing method provided by this embodiment is implemented in the application framework layer shown in FIG. 2 . As shown in Figure 3, the method includes the following steps:

101. Receive multi-channel image data sent by the hardware abstraction layer.

102. Select at least one channel of first image data from multiple channels of image data.

103. Process at least one channel of first image data according to the set data processing method.

The above-mentioned multi-channel image data may be collected simultaneously by the image acquisition device in the user terminal device, and transmitted to the application framework layer through the hardware abstraction layer.

In actual applications, user terminal devices are often installed with applications that require the use of cameras, such as some image processing applications, some instant messaging applications, etc. Users can operate the application to start the camera to collect the above-mentioned multi-channel image data.

In addition, optionally, the user can also set shooting parameters in the relevant interface of the application, such as setting which image data the camera needs to collect, such as collecting preview image data and photographed image data; such as collecting preview image data and video recording image data, And so on, to control the camera to collect corresponding multi-channel image data. In addition, the interface can also provide settings for data processing methods for users to select the required data processing methods, such as filtering, beautification, augmented reality (AR), face detection, etc. for multi-channel image data. One or more treatments.

Of course, in addition to the user's settings on the application program interface, the above data processing methods may also optionally be the data processing methods configured by default in the operating system.

Therefore, when the user sets the data processing method in the application interface, the application will send the data processing method set by the user for the currently collected multi-channel image data to the application framework layer, and the application framework layer will The controller receives the data processing method, so that the controller triggers the selector to select at least one channel of first image data from the multiple channels of image data, so as to process the at least one channel of first image data according to the data processing method. Wherein, the number of channels of at least one channel of selected first image data is smaller than the number of channels of multiple channels of image data.

In an optional embodiment, selecting at least one channel of first image data from multiple channels of image data can be implemented by: selecting at least one channel of first image data from multiple channels of image data that corresponds to the set data processing method, At this time, the data processing method not only indicates what kind of data processing needs to be performed on the image data, but can also be used as a basis for image data selection.

At this time, at least one kind of first image data corresponding to any data processing method can be selected according to the preset corresponding relationship between the data processing method and the image data, wherein one data processing method can correspond to one kind of first image data. or several image data.

Since the data processing method set for the multi-channel image data currently collected at the same time may be one or multiple, it is necessary to select at least one first channel from the multi-channel image data for each data processing method. The image data is processed in this data processing method. At least one piece of first image data corresponding to different data processing methods may be different or the same.

In an optional embodiment, after the application framework layer processes the selected at least one channel of first image data according to the set data processing method, the application framework layer can send the processing result of the at least one channel of first image data to the application. program, or the processing result of the at least one channel of first image data and the unprocessed other channels of image data in the multiple channels of image data can also be sent to the application program.

For example, assuming that the current multi-channel image data includes preview image data and video recording image data, the set data processing method is beautification processing, and assuming that at least one first channel of image data selected therefrom is video recording image data, then only The video image data is beautified and the video image data after beautification is fed back to the application program. The preview image data does not need to be processed and is directly fed back to the application program for the user to watch.

In another optional embodiment, after processing the selected at least one channel of first image data according to the set data processing method, multiple channels of image data can also be processed based on the processing results of the at least one channel of first image data. At least one channel of second image data in the at least one channel of second image data is all or part of the remaining image data in the multiple channels of image data except the first image data. Therefore, optionally, after step 103, the following steps may also be included:

104. Process at least one channel of second image data among the multiple channels of image data according to the processing result of at least one channel of first image data.

105. Transmit the processed image data to the application program; or transmit the processed image data and the unprocessed image data in the multi-channel image data to the application program.

At this time, the processed image data includes processed first image data and processed second image data.

It is worth noting that in this embodiment, at least one channel of second image data is processed based on the processing results of at least one channel of first image data, rather than directly using the above-set data processing method to process at least one channel of second image data. Image data is processed. The processing of at least one channel of second image data based on the processing results of at least one channel of first image data can be simply understood as multiplexing or synchronizing the processing results of at least one channel of first image data to the at least one channel of second image data. image data.

With reference to FIG. 2 , at least one channel of first image data can be processed by the controller, and then the processing result can be synchronized by the synthesizer to the at least one channel of second image data.

For example, assume that the currently collected multi-channel image data includes preview image data and video recording image data, and the data processing method includes three processing methods: filter, beautification, and some preset object detection, and then assume that with the filter At least one channel of first image data corresponding to the beautification processing method is preview image data, and at least one channel of first image data corresponding to a certain preset object detection processing method is video recording image data. Then the preview image data can be filtered first. Mirror and beauty processing, perform some kind of preset object detection processing on the video image data, add semantic information corresponding to the preset object in the image frame detected to contain the preset object, and then process the preview image data The results are synchronized to the recorded image data after adding semantics, and the processing results of the recorded image data are synchronized to the preview image data after filtering and beautification processing. Therefore, both the preview image data and the recorded image data have filters, beautification and beautification. Preset object detection processing effect.

Combined with the above example, in an optional embodiment, for each data processing method, only one channel of image data is selected from multiple channels of image data for processing in that data processing method, and the processing result of this channel of image data is complex. It is used for the remaining channels of image data, thereby realizing parallel processing of the multiple channels of image data, so that the multiple channels of image data have the same processing effect.

In addition, combined with the above example, processing at least one channel of second image data among multiple channels of image data based on the processing result of at least one channel of first image data may include the following two methods of image processing: size transformation and adding image semantics.

The size transformation specifically includes: performing size transformation processing on the processing results of at least one channel of first image data according to the image resolution corresponding to at least one channel of first image data and the image resolution corresponding to at least one channel of second image data, so as to achieve the size transformation. The result replaces at least one second image data. Specifically, the size transformation process may be performed on the processing result of at least one channel of first image data according to the ratio between the image resolution corresponding to at least one channel of first image data and the image resolution corresponding to at least one channel of second image data.

Optionally, when the number of at least one channel of first image data is not 1, one first image data may be randomly selected, or the first image data with the highest or lowest resolution may be selected based on the selection. The resolution ratio between the selected first image data and each second image data is determined, and size transformation processing, such as downsampling or interpolation processing, is performed on the processing result of the selected first image data.

The above-mentioned adding image semantics is specifically: assuming that the set data processing method includes object recognition processing, based on the object recognition results in at least one channel of first image data, adding corresponding to the object recognition results in at least one channel of second image data Image semantic information. In actual applications, optionally, users can set objects to be recognized, such as setting face recognition. The image semantic information corresponding to the object recognition result may be relevant description information of the recognized object, or may simply be an annotation box used to annotate the recognized object.

For a simple example, suppose there are two channels of image data, the resolution of which is the first image data is 640*480, and the resolution of the other channel which is the second image data is 320*240, and it is assumed that the data processing method is beauty processing. , then after beautification processing is performed on the first image data, the beautification result can be synchronized to the second image data through downsampling, for example, two adjacent pixels in the first image data after beautification processing are line by line. The pixel values are averaged and synthesized into one pixel, thereby obtaining 320*240 image data with a beauty effect, and this image data replaces the original second image data.

For another example, suppose there are two channels of image data, the resolution of which is the first image data is 640*480, and the resolution of the other channel which is the second image data is 320*240, and it is assumed that the data processing method is preset object detection processing. , then perform preset object detection processing on the first image data to mark the preset object in the image frame containing the preset object, such as marking the detected preset object through a rectangular frame. After that, based on the proportional relationship between the resolution of 320*240 and 640*480, the annotated first image data is first downsampled, and then the rectangular frame is reduced twice, thereby synchronizing the object detection results to the second image. data. Of course, in this example, it is also possible not to perform downsampling processing, but to directly multiplex and synchronize the processing results of the preset objects in the first image data to the second image data, that is, directly use the annotated first image to The data replaces the original second image data.

As mentioned in the aforementioned optional embodiments, at least one first image data can be selected according to a certain data selection basis, and the acquisition method of the data selection basis can be achieved through the following two optional methods:

First, the way users set it up by themselves. At this point, the user can select a data selection basis in the relevant interface of the application. In an optional embodiment, the application program can not only provide a data processing method selection interface, but also provide a data selection basis setting interface. Therefore, the user can select one or several data processing methods in the data processing method selection interface, and when the user selects a certain data processing method, the data selection basis setting interface can pop up, which displays a variety of data The selection basis options are for the user to choose from. When the user sets multiple data processing methods, the user can set the corresponding data selection basis for each set data processing method. Therefore, the data selection basis can be corresponding to each data processing method set by the user, from Select at least one channel of first image data corresponding to each data processing method from the multiple channels of image data.

In this method, optionally, the data selection basis includes one of the following: performance meets the set conditions, quality meets the set conditions, set specific camera data, and the corresponding relationship between the preset data processing method and the image data.

Second, the application framework layer can dynamically determine the basis for data selection based on the resource usage status of the user terminal device. That is, the resource usage status of the terminal device is obtained, the data selection basis is determined according to the resource usage status of the terminal device, and at least one channel of first image data is selected from the multiple channels of image data based on the determined data selection basis.

Among them, the resource usage status of the terminal device is, for example, represented by one or a combination of CPU load level, remaining power, etc. Optionally, at this time, the basis for data selection may be that performance meets set conditions or quality meets set conditions.

Alternatively, for example, the resource usage status of the terminal device can be simply divided into two situations: high resource consumption and low resource consumption based on a certain threshold; or, the resource usage status of the terminal device can be divided according to whether the resource consumption exceeds or falls below The degree of the threshold is divided into multiple intervals in advance. Therefore, when the actual resource consumption of the terminal device is greater than the threshold or is within a certain interval corresponding to high resource consumption, it is determined that the data selection basis at this time is that the performance meets the set conditions. On the contrary, if the resource consumption of the terminal device When it is less than the threshold or within a certain interval corresponding to low resource consumption, it can be determined that the data selection basis at this time is that the quality meets the set conditions.

Optionally, when the resource usage status includes multiple metrics, you can set the contribution weights of different metrics to the resource usage status. For example, when the resource usage status is measured by CPU load and remaining power, you can set the CPU load. The contribution weight of degree is 0.7, and the contribution degree of remaining power is 0.3, so the resource usage status is evaluated by the weighted sum of the two.

Alternatively, in addition to determining the data selection basis based on the resource usage status of the terminal device, it can also be further combined with the preset resource consumption corresponding to each data processing method (which can also be called the algorithm corresponding to each data processing method). Complexity) to determine the basis for data selection. Among them, the resource consumption corresponding to various commonly used data processing methods can be preset. The resource consumption corresponding to each data processing method can be set in advance through testing. Optionally, the resource consumption can be reflected as a specific numerical value; it can also be reflected as a degree, such as high resource consumption and low resource consumption. For example, the data processing method for resource consumption greater than a certain threshold can be set to high resource consumption; Set the data processing method for data whose resource consumption is less than a certain threshold to low resource consumption. Therefore, the data selection basis can be determined based on the resource usage status of the terminal device and the resource consumption corresponding to the preset data processing method.

Optionally, at this time, the determination of the data selection basis can be implemented as follows: the resource usage status of the terminal device can be pre-divided into multiple intervals according to the degree to which the resource consumption of the terminal device exceeds or falls below a certain threshold, such as according to The order of resource consumption is divided into three intervals: A, B, and C. The resource consumption corresponding to the various data processing methods currently set by the user is accumulated, so as to determine the data selection basis corresponding to all the data processing methods currently set by the user based on the accumulated resource consumption and the interval in which the resource usage status of the current terminal device is located. . Specifically, the data selection basis may be determined based on the accumulated resource consumption and whether the interval in which the resource usage status of the current terminal device is located will cause a change in the interval in which the resource usage status of the terminal device is located. For example, assume that the cumulative resource consumption corresponding to all data processing methods set by the current user is On the basis of adding the Based on the quality meeting the set conditions.

That is to say, if the cumulative resource consumption of the data processing method will make the resource consumption of the terminal device very heavy, then the performance meeting the set conditions will be used as the basis for data selection. On the contrary, the quality can be used as the basis for data selection. Based on this, a balance is made between the image quality after image data processing and the resource consumption of the terminal device.

It can be understood that when the resource usage status of the terminal device or a combination of the resource usage status of the terminal device and the preset resource consumption of each data processing method is used as the basis for data selection, the various data processing methods set at this time can be considered Correspond to the same data selection basis.

The following is a detailed description of the specific implementation of selecting at least one first image data based on the several data selection criteria mentioned above:

The performance satisfies the set conditions: image data is selected from multiple channels of image data in order from low to high corresponding image resolution as at least one channel of first image data. A simple situation may be: selecting the image data corresponding to the lowest image resolution from the multiple channels of image data as the first image data.

The quality meets the set conditions: image data is selected from the multiple channels of image data in descending order of corresponding image resolution as at least one channel of first image data. A simple situation may be: selecting the image data corresponding to the highest image resolution from the multiple channels of image data as the first image data.

Set specific image data. The specific image data is selected from multiple channels of image data as at least one channel of first image data.

The correspondence between the preset data processing method and the image data: This correspondence can be customized by the user in the relevant interface of the application. Therefore, at least one channel of first image data can be selected from multiple channels of image data according to the corresponding relationship.

Among them, performance or quality that meets the set conditions can be understood as having the least impact on the processing performance of the terminal device or the best quality of the processed image data.

Regarding the above-mentioned data selection basis, assuming that one channel of first image data is selected, and assuming that at least one channel of second image data is the remaining other channels of image data in the multi-channel image data, only taking size transformation as an example, according to The processing result of the first image data and processing of other image data can be implemented as:

If the data selection basis is that the performance meets the set conditions, after data processing is performed on the first image data corresponding to the lowest image resolution in the multi-channel image data, the corresponding image resolutions of the other channels of image data can be compared with the first image data. The ratio between the resolutions of an image data is to perform interpolation processing on the processing results of the first image data, respectively, and replace the corresponding other channels of image data with the interpolated first image data.

If the data selection basis is that the quality meets the set conditions, after data processing is performed on the first image data corresponding to the highest image resolution in the multi-channel image data, the image resolution corresponding to the other channels of image data can be compared with the first image data. The ratio between the resolutions of the first image data is to perform downsampling processing on the processing results of the first image data, and replace the corresponding other channels of image data with the downsampled first image data.

If the data selection basis is the corresponding relationship between the set specific image data or the preset data processing method and the image data, then after performing data processing on the first image data selected from the multi-channel image data, the data can be processed according to the other channels of image data. The ratio between the respective image resolutions of the data and the resolution of the first image data, respectively perform downsampling or difference processing on the processing results of the first image data, and replace the corresponding results with the downsampling or difference processing results. Other image data.

In summary, in the embodiment of the present invention, various processing of multi-channel image data is implemented at the application framework layer. Specifically, the application program located at the application layer sends the data processing method for the multi-channel image data to the application framework layer, so that after receiving the multi-channel image data from the hardware abstraction layer, the application framework layer processes the multi-channel image data from the multi-channel image data. Select the image data to be processed corresponding to the data processing method from the image data, and then first process the image data to be processed in the data processing method, and then synchronize the processing results of the image data to be processed to other image data, So that other channels of image data also have processing effects corresponding to the data processing method, and finally the processed multi-channel image data is transmitted to the application program. Through this solution, the processing of multi-channel image data is realized in the application framework layer, which reduces the difficulty of application development. Moreover, in the application framework layer, for a certain data processing method, only from the multi-channel image data The most suitable image data is selected for processing in this data processing method, instead of processing multiple image data in this data processing method, thereby reducing the data processing cost, that is, reducing the consumption of processing resources.

Figure 4 is a flow chart of another data processing method provided by an embodiment of the present invention. The data processing method provided by this embodiment can be performed by an application in the application layer shown in Figure 2 that needs to use image data. accomplish. As shown in Figure 4, the method may include the following steps:

201. Send the data processing method to the application framework layer, so that the application framework layer processes at least one channel of image data among the multiple channels of image data received from the hardware abstraction layer according to the data processing method.

202. Receive processed image data sent by the application framework layer.

Optionally, the application may include a data collection setting interface and a data processing method selection interface. The user can set which channels of image data need to be started by starting the camera in the data collection setting interface. For example: if the user sets the camera to photo mode, preview image data and photo image data will be collected; if the user sets the camera to video mode, What is collected is preview image data and video recording image data. After the user completes the image data collection settings, the data processing method selection interface can be displayed. The selection interface contains a variety of data processing methods for the user to choose. After the user selects one or more data processing methods corresponding to the multi-channel image data, the application program sends the one or more data processing methods to the application framework layer, so that the application framework layer implements the The method introduced in the example selects at least one channel of first image data from multiple channels of image data, and processes the at least one channel of first image data according to each set data processing method.

In addition, optionally, as mentioned in the foregoing embodiments, the user can also set the data selection basis corresponding to each data processing method in the application program, so that the application framework layer can select the corresponding image data from the multi-channel image data. The first image data corresponding to each data processing method. Therefore, optionally, when the user selects a data processing method, a data selection basis setting interface may pop up, which contains a variety of data selection basis for the user to select.

Since the processing logic for multi-channel image data is implemented in the application framework layer, the application only needs to input setting information to the application framework layer and receive the processing results of the application framework layer, which reduces the difficulty of application development.

The following describes the data provided by the embodiment of the present invention with reference to the embodiment shown in Figure 5, taking the actual collected multi-channel image data including preview image data and photographed image data, and the processing of the multi-channel image data including filters and beautification processing as an example. Approach.

Figure 5 is an interaction diagram of a data processing method provided by an embodiment of the present invention. As shown in Figure 5, it may include the following steps:

301. The application responds to the user's settings and triggers the camera to open to collect preview image data and photographed image data.

302. The application sends the filters and beauty methods set by the user to the application framework layer.

303. The application framework layer receives preview image data and photographed image data from the hardware abstraction layer.

304. The application framework layer selects the photographed image data for filtering and beautification processing.

305. The application framework layer synchronizes the filters and beauty results of the photographed image data to the preview image data.

For example, based on the resolution ratio of the pre-image data and the photographed image data, interpolation or downsampling is performed on the processed photographed image data, and the original preview image data is replaced with the interpolation or downsampling processing result.

306. The application framework layer sends the processed preview image data and photographed image data to the application.

Users can then see the preview image data after filtering and beautification processing.

The data processing apparatus of one or more embodiments of the present invention will be described in detail below. Those skilled in the art can understand that these data processing devices can be configured using commercially available hardware components and configured through the steps taught in this solution.

Figure 6 is a schematic structural diagram of a data processing device corresponding to the embodiment shown in Figure 3. The data processing device corresponds to the application framework layer. As shown in Figure 6, the device includes: a receiving module 11, a selection module 12, Processing module 13 and sending module 14.

The receiving module 11 is used to receive multi-channel image data sent by the hardware abstraction layer.

The selection module 12 is used to select at least one channel of first image data from the multiple channels of image data.

The processing module 13 is configured to process the selected at least one channel of first image data according to a set data processing method.

Wherein, the number of channels of the at least one channel of first image data is smaller than the number of channels of the multiple channels of image data.

Optionally, the processing module 13 is also configured to process at least one channel of second image data among the multiple channels of image data according to the processing result of the at least one channel of first image data.

The device also includes: a sending module 14 for transmitting the processed image data to the application program; or, transmitting the processed image data and the unprocessed image data in the multi-channel image data to the application program.

Optionally, the processing module 13 is specifically used to:

According to the image resolution corresponding to the at least one channel of first image data and the image resolution corresponding to the at least one channel of second image data, a size transformation process is performed on the processing result of the at least one channel of first image data. After size transformation, The result replaces the at least one second image data.

Optionally, when the set data processing method includes object recognition processing, the processing module 13 is specifically configured to: based on the object recognition results in the at least one channel of first image data, in the at least one channel of second image data Add image semantic information corresponding to the object recognition result.

Optionally, the receiving module 11 is also configured to receive the data selection basis sent by the application program.

Correspondingly, the selection module 12 is specifically configured to select the at least one channel of first image data from the multiple channels of image data according to the data selection basis.

The data selection basis includes the following: performance meets the set conditions, quality meets the set conditions, set specific camera data, and the corresponding relationship between the preset data processing method and the image data.

Optionally, the selection module 12 is also used to:

Obtain the resource usage status of the terminal device; determine the data selection basis according to the resource usage status of the terminal device; select the at least one channel of first image data from the multiple channels of image data based on the data selection basis.

Among them, the selection module 12 is specifically used for:

The data selection basis is determined according to the resource usage status of the terminal device and the resource consumption corresponding to the preset data processing method.

Wherein, the data selection basis is that the performance meets the set conditions or the quality meets the set conditions.

Specifically, the data selection basis is that the performance meets the set conditions, and the selection module 12 is specifically used to:

Select image data from the multiple channels of image data as the at least one channel of first image data in order from low to high corresponding image resolutions.

Specifically, the data selection basis is that the quality meets the set conditions, and the selection module 12 is specifically used to:

Select image data from the multiple channels of image data as the at least one channel of first image data in descending order of corresponding image resolutions.

The device shown in Figure 6 can perform the method in Figure 3. For parts not described in detail in this embodiment, reference can be made to the relevant description of the embodiment shown in Figure 3. For the implementation process and technical effects of this technical solution, please refer to the description in the embodiment shown in Figure 3 and will not be described again here.

In a possible design, the structure of the data processing device shown in Figure 6 can be implemented as an electronic device. The electronic device is a user terminal device, such as a smartphone. As shown in Figure 7, the user terminal device can include: a processor 21 and memory 22. The memory 22 is used to store programs that support the user terminal device to execute the data processing methods provided in the above-described embodiments, and the processor 21 is configured to execute the programs stored in the memory 22 .

The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 21, the following steps can be implemented:

Receive multi-channel image data sent by the hardware abstraction layer;

Select at least one channel of first image data from the multiple channels of image data;

The selected at least one channel of first image data is processed according to the set data processing method.

Optionally, the processor 21 is also configured to execute all or part of the steps of the aforementioned methods.

The structure of the user terminal device may also include a communication interface 23 for the user terminal device to communicate with other devices or communication networks.

In addition, embodiments of the present invention provide a computer storage medium for storing computer software instructions used in user terminal equipment, which includes programs for executing the data processing methods in the above-described method embodiments.

Figure 8 is a schematic structural diagram of a data processing device corresponding to the embodiment shown in Figure 4. As shown in Figure 8, the device includes: a sending module 31 and a receiving module 32.

The sending module 31 is used to send the data processing method to the application framework layer, so that the application framework layer processes at least one channel of image data in the multi-channel image data according to the data processing method. The multi-channel image data Data is sent from the hardware abstraction layer to the application framework layer.

The receiving module 32 is configured to receive the processed image data sent by the application framework layer.

Optionally, the sending module 31 is also used to:

The data selection basis is sent to the application framework layer, so that the application framework layer selects the at least one channel of image data.

The device shown in Figure 8 can perform the method of the embodiment shown in Figure 4. For parts that are not described in detail in this embodiment, reference can be made to the relevant description of the embodiment shown in Figure 4. For the implementation process and technical effects of this technical solution, please refer to the description in the embodiment shown in Figure 4 and will not be described again here.

In a possible design, the structure of the data processing device shown in Figure 8 can be implemented as an electronic device. The electronic device is a user terminal device, such as a smartphone. As shown in Figure 9, the user terminal device can include: a processor 41 and memory 42. The memory 42 is used to store programs that support the user terminal device to execute the data processing methods provided in the above-described embodiments, and the processor 41 is configured to execute the programs stored in the memory 42 .

The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 41, the following steps can be implemented:

Send the data processing method to the application framework layer, so that the application framework layer processes at least one of the multiple channels of image data according to the data processing method, and the multiple channels of image data are sent by the hardware abstraction layer to the application framework layer;

Receive processed image data sent by the application framework layer.

Optionally, the processor 41 is also configured to execute all or part of the steps of the aforementioned methods.

The structure of the user terminal device may also include a communication interface 43 for the user terminal device to communicate with other devices or communication networks.

In addition, embodiments of the present invention provide a computer storage medium for storing computer software instructions used in user terminal equipment, which includes programs for executing the data processing methods in the above-described method embodiments.

The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.

From the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding the necessary general hardware platform, or of course, can also be implemented by combining hardware and software. Based on this understanding, the above technical solution can be embodied in the form of a computer product in nature or in other words, the part that contributes to the existing technology. The present invention can use one or more computer-usable storage devices containing computer-usable program codes. The form of a computer program product implemented on media (including but not limited to disk storage, CD-ROM, optical storage, etc.).

The invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a use A device for realizing the functions specified in one process or multiple processes of the flowchart and/or one block or multiple blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions The device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in computer-readable media, random access memory (RAM), and/or non-volatile memory in the form of read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer-readable media includes both persistent and non-volatile, removable and non-removable media that can be implemented by any method or technology for storage of information. Information may be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), and read-only memory. (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cassettes, tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be used Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent substitutions are made to some of the technical features; however, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (14)

1. A data processing method, characterized in that it is applied to the application framework layer, including: Receive multi-channel image data sent by the hardware abstraction layer; Select at least one channel of first image data from the multiple channels of image data, the multiple channels of image data are generated simultaneously and the image resolution of each channel of image data is different; Process the selected at least one channel of first image data according to the set data processing method; Process at least one channel of second image data in the multiple channels of image data according to the processing result of the at least one channel of first image data; Wherein, processing at least one channel of second image data among the multiple channels of image data based on the processing result of the at least one channel of first image data includes: According to the image resolution corresponding to the at least one channel of first image data and the image resolution corresponding to the at least one channel of second image data, a size transformation process is performed on the processing result of the at least one channel of first image data. After size transformation, The result replaces the at least one second image data; Or, when the set data processing method is object recognition processing, based on the object recognition results in the at least one channel of first image data, add an image corresponding to the object recognition result in the at least one channel of second image data. semantic information. 2. The method according to claim 1, characterized in that the number of channels of the at least one channel of first image data is smaller than the number of channels of the multiple channels of image data. 3. The method according to any one of claims 1 to 2, further comprising; Transfer the processed image data to the application; or The processed image data and the unprocessed image data in the multi-channel image data are transmitted to the application program. 4. The method of claim 1, wherein selecting at least one channel of first image data from the multiple channels of image data includes: Basis for receiving data selection; The at least one channel of first image data is selected from the multiple channels of image data according to the data selection basis. 5. The method according to claim 4, characterized in that the data selection basis includes one of the following: The performance meets the set conditions, the quality meets the set conditions, the set camera data, the preset data processing method and the correspondence between the image data. 6. The method of claim 1, wherein selecting at least one channel of first image data from the multiple channels of image data includes: Obtain the resource usage status of the terminal device; Determine the data selection basis according to the resource usage status of the terminal device; The at least one channel of first image data is selected from the multiple channels of image data according to the data selection basis. 7. The method according to claim 6, wherein the determining the data selection basis according to the resource usage status of the terminal device includes: The data selection basis is determined according to the resource usage status of the terminal device and the resource consumption corresponding to the preset data processing method. 8. The method according to claim 6, characterized in that the data selection basis is that performance meets set conditions or quality meets set conditions. 9. The method according to claim 5 or 8, characterized in that the data selection basis is that performance meets set conditions, and selecting at least one channel of first image data from the multiple channels of image data includes: Select image data from the multiple channels of image data as the at least one channel of first image data in order from low to high corresponding image resolutions. 10. The method according to claim 5 or 8, characterized in that the data selection basis is that the quality meets the set conditions, and the selection of at least one first image data from the multiple channels of image data includes: Select image data from the multiple channels of image data as the at least one channel of first image data in descending order of corresponding image resolutions. 11. An electronic device, characterized by comprising a memory and a processor; wherein, The memory is used to store one or more computer instructions, wherein when the one or more computer instructions are executed by the processor, the data processing method according to any one of claims 1 to 10 is implemented. 12. A data processing method, characterized in that it is applied in an application program, including: The data processing method is sent to the application framework layer as claimed in claim 1, so that the application framework layer processes at least one of the multiple channels of image data according to the data processing method, and the multiple channels of image data are Image data is sent by the hardware abstraction layer to the application framework layer; Receive processed image data sent by the application framework layer. 13. The method according to claim 12, characterized in that the method further comprises: The data selection basis is sent to the application framework layer, so that the application framework layer selects the at least one channel of image data. 14. An electronic device, characterized by comprising a memory and a processor; wherein, The memory is used to store one or more computer instructions, wherein when the one or more computer instructions are executed by the processor, the data processing method as claimed in claim 12 or 13 is implemented.