CN117274783A - Data processing method, device, equipment and storage medium based on motion capture - Google Patents

Abstract

The disclosure provides a data processing method, device, equipment and storage medium based on motion capture, relates to the technical field of image processing, and particularly relates to the field of computer vision and virtual reality. The specific implementation scheme is as follows: after observing the motion capture object to obtain the observed quantity of the motion parameters of the joints of the motion capture object, respectively adopting a first jitter elimination algorithm to process the observed quantity of the motion parameters of each joint to obtain a first reference quantity, and adopting a second jitter elimination algorithm to process the observed quantity of the motion parameters of each joint to obtain a second reference quantity. And selecting a target jitter elimination algorithm matched with the difference to correct the observed quantity of the motion parameter according to the difference between the first reference quantity and the second reference quantity. The motion speed state of the motion capture object is reflected through the difference between the results of two different jitter elimination algorithms, and then the matched target jitter elimination algorithm is selected based on the different motion speed states to correct the observed quantity of the motion parameters, so that the motion restoration degree of the motion capture object is improved.

Description

Data processing methods, devices, equipment and storage media based on motion capture

Technical field

The present disclosure relates to the field of image processing technology, specifically to the field of computer vision and virtual reality technology, which can be applied to animation scenes, and in particular to data processing methods, devices, equipment and storage media based on motion capture.

Background technique

Motion capture technology can capture the body posture characteristics, joint positions and other action states of humans or animals through optical, inertial and visual methods to obtain motion data. Since the original motion data obtained by motion capture commonly suffers from jitter and uneven trajectories, the original motion data must be manually refined by animators or post-processed to restore the motion of the motion capture object to a certain extent.

Contents of the invention

The present disclosure provides a data processing method, device, equipment and storage medium based on motion capture.

According to one aspect of the present disclosure, a data processing method based on motion capture is provided, including:

Observe the motion capture object to obtain motion parameter observations of the joints of the motion capture object;

The motion parameter observations of the joints are processed using a first jitter elimination algorithm to obtain the first reference quantity, and are processed using a second jitter elimination algorithm to obtain the second reference quantity; wherein, the second jitter elimination algorithm The smoothing strength is different from the first jitter elimination algorithm;

According to the difference between the first reference quantity and the second reference quantity, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observation.

According to another aspect of the present disclosure, a data processing device based on motion capture is provided, including:

An observation module, used to observe the motion capture object to obtain motion parameter observations of the joints of the motion capture object;

A processing module configured to process the motion parameter observations of the joint using a first jitter elimination algorithm to obtain a first reference quantity, and to process the motion parameter observations using a second jitter elimination algorithm to obtain a second reference quantity; wherein, the The smoothing strength of the second jitter removal algorithm is different from the first jitter cancellation algorithm;

A correction module, configured to select a target jitter elimination algorithm that matches the difference according to the difference between the first reference quantity and the second reference quantity to correct the motion parameter observation.

According to another aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores information that can be used by the at least one processor. Execution instructions, the instructions are executed by the at least one processor, so that the at least one processor can execute the method described in the embodiment of the first aspect of the present disclosure.

According to another aspect of the disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method described in the embodiment of the first aspect of the disclosure.

According to another aspect of the disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the embodiment of the first aspect of the disclosure.

The data processing method, device, equipment and storage medium based on motion capture provided by the present disclosure observe the motion capture object to obtain the motion parameter observations of the joints of the motion capture object, and then observe the motion parameters of each joint. The quantity is processed by using the first jitter elimination algorithm to obtain the first reference quantity, and is processed by using the second jitter elimination algorithm to obtain the second reference quantity. According to the difference between the first reference quantity and the second reference quantity, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observation. Since the smoothing intensity of the second jitter elimination algorithm is different from the first jitter elimination algorithm, the difference between the results of these two different algorithms can reflect the motion speed state of the motion capture object, and further, based on the different motion speed states Select a matching target jitter elimination algorithm to correct the motion parameter observations, so that the corrected motion parameter observations can smooth the trajectory while maintaining the original rhythm of the action, and improve the action restoration of the motion capture object.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

Description of the drawings

The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. in:

Figure 1 is a schematic flow chart of a data processing method based on motion capture provided by an embodiment of the present disclosure;

Figure 2 is a schematic diagram of the joint;

Figure 3 is a schematic flowchart of another data processing method based on motion capture provided by an embodiment of the present disclosure;

Figure 4 is a schematic diagram of the principle of a data processing method based on motion capture provided by an embodiment of the present disclosure;

Figure 5 is a schematic flowchart of another data processing method based on motion capture provided by an embodiment of the present disclosure;

Figure 6 is a schematic structural diagram of a motion capture-based data processing device 600 provided by an embodiment of the present disclosure;

7 illustrates a schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the present disclosure are included to facilitate understanding and should be considered to be exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

In related virtual reality or animation technology, in order to solve the problems of data jitter and unsmooth trajectory when performing motion capture, filtering is commonly used in post-processing methods in the industry to reduce jitter. This method will lose the degree of restoration of the motion. , increasing the action delay, making the followability of fast actions worse.

In order to ensure a certain degree of followability while being smooth, in the embodiment of the present disclosure, the motion capture object is observed to obtain the motion parameter observations of the joints of the motion capture object, and then the motion parameter observations of each joint are: The first jitter elimination algorithm is used for processing to obtain the first reference quantity, and the second jitter elimination algorithm is used for processing to obtain the second reference quantity. Since the smoothing intensity of the second jitter elimination algorithm is different from the first jitter elimination algorithm, the difference between the results of these two different algorithms can reflect the motion speed state of the motion capture object, and further, based on the different motion speed states Select a matching target jitter elimination algorithm to correct the motion parameter observations, so that the corrected motion parameter observations can smooth the trajectory while maintaining the original rhythm of the action, and improve the action restoration of the motion capture object.

Figure 1 is a schematic flow chart of a data processing method based on motion capture provided by an embodiment of the present disclosure. The method provided by this embodiment can be executed by a motion capture device, as shown in Figure 1, including:

Step 101: Observe the motion capture object to obtain motion parameter observations of the joints of the motion capture object.

Among them, the motion capture object can be a human body or an animal.

Motion capture technology includes optical, inertial and visual methods to obtain movement information of the human body or animals. In this embodiment, what is obtained is the observation of motion parameters of joints related to the action state.

As for the motion parameter observation, Figure 2 is a schematic diagram of a joint. The motion parameter observation can be at least one of the movement speed, posture, position coordinates and angle of the joint shown in Figure 2, which is not limited in this embodiment. As shown in Figure 2, joints connected by lines belong to the same joint motion chain.

As a possible implementation manner, joint positions of the motion capture object are observed to obtain the joint position coordinates of each joint; based on the joint position coordinates of each joint, the joint angle of each joint is determined; the position is Coordinates and/or joint angles serve as the motion parameter observations.

As another possible implementation method, observe the joint angles of the motion capture object to obtain the joint angles of each joint; use a kinematic model to identify the relative positions of each joint point based on the joint angles of each joint, Relative positions and/or joint angles are used as the observed motion parameters.

Step 102: Use the first jitter elimination algorithm to process the joint motion parameter observations to obtain the first reference quantity, and use the second jitter elimination algorithm to process the joint movement parameter observations to obtain the second reference quantity, wherein the second jitter elimination algorithm The smoothing intensity of the algorithm differs from the first jitter removal algorithm.

As a possible implementation manner, the first jitter elimination algorithm and the second jitter elimination algorithm adopt different filtering orders, that is, the order of the filtering formulas is different. For example, the first jitter elimination algorithm may be a jitter elimination algorithm based on first-order filtering, and the second jitter elimination algorithm may be a jitter elimination algorithm based on second-order filtering.

As another possible implementation manner, the first jitter elimination algorithm and the second jitter elimination algorithm use the same filtering order, but the parameters of the filter formula are different.

It should be noted that the first jitter elimination algorithm and the second jitter elimination algorithm are both based on the historical motion parameters observed before the current motion parameter observation is observed, or the historical motion parameters are obtained after anti-shake processing, using The filter function specified by the algorithm is smoothed to achieve an anti-shake effect.

Step 103: According to the difference between the first reference quantity and the second reference quantity, select a target jitter elimination algorithm that matches the difference to correct the motion parameter observations.

In this embodiment, after observing the motion capture object to obtain the motion parameter observations of the joints of the motion capture object, the motion parameter observations of each joint are processed using the first jitter elimination algorithm to obtain the first reference quantity, and using The second jitter elimination algorithm performs processing to obtain the second reference quantity. According to the difference between the first reference quantity and the second reference quantity, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observation. The difference between the results of two different jitter elimination algorithms reflects the motion speed of the motion capture object. Then, based on the different motion speed states, a matching target jitter elimination algorithm is selected to correct the motion parameter observations and improve the accuracy of the motion capture object. Action restoration.

Figure 3 is a schematic flowchart of another data processing method based on motion capture provided by an embodiment of the present disclosure. As shown in Figure 3, it includes:

Step 301: Observe the motion capture object to obtain motion parameter observations of the joints of the motion capture object.

Optionally, observe the joint angles of the motion capture object to obtain the joint angle J as the original motion capture data. Based on the pre-established kinematic model K, the three-dimensional coordinate position P of the joint can be determined based on the joint angle J.

Among them, the kinematic model K is a model used to determine the three-dimensional coordinate position of the joint based on the joint angle.

Step 302: Use the first jitter elimination algorithm to process the observed motion parameters of the joints to obtain the first reference quantity, and use the second jitter elimination algorithm to process the motion parameter observations to obtain the second reference quantity.

Wherein, the smoothing strength of the second jitter elimination algorithm is different from the first jitter elimination algorithm.

Optionally, perform first-order filtering on joint angle J. First-order filtering can alleviate the obvious jitter in the data, but keep fast movements unaffected, and obtain joint angle J1. And, perform a second-order filter on the joint angle J. The second-order filter can eliminate all jitter. At the same time, fast movements/large movements will also be significantly weakened or the range of motion disappears, and the joint angle J2 is obtained.

Using the kinematic model K, the 3D position P1 of the joint point is calculated for J1. Similarly, the 3D position P2 of the joint point is calculated for J2 using the kinematic model K.

In order to eliminate data jitter without affecting the motion effect of fast movements, a method that combines first-order filtering and second-order filtering is proposed.

It should be noted that the aforementioned J, J1, J2, P, P1 and P2 may be vectors representing angles or coordinates in a three-dimensional space. Comprehensively consider multiple motion parameters to improve recognition accuracy.

For details of step 301 and step 302, reference may be made to step 101 and step 102 in the previous embodiment. This will not be described again in this embodiment.

Step 303: When there are multiple joints from which the motion parameter observation is observed, generate a first matrix and a second matrix based on at least two target joints belonging to the same joint kinematic chain among the multiple joints, and generate a third matrix. Three matrices.

Each row vector in the first matrix corresponds to the target joint and is used to indicate the first reference quantity of the corresponding target joint in the three-dimensional space.

Each row vector in the second matrix corresponds to the target joint and is used to indicate the second reference quantity of the corresponding target joint in the three-dimensional space.

Each row vector in the third matrix corresponds to the target joint and is used to indicate the observed motion parameter of the corresponding target joint in the three-dimensional space.

Optionally, a joint kinematic chain is set in advance to represent that the movement of a parent node usually brings about the movement of a child node, for example, all joints on an arm form a joint kinematic chain. Joint motions on the same joint kinematic chain will affect each other, while motions on different kinematic chains will have little influence on each other. In this embodiment, the joints on the same joint kinematic chain are comprehensively considered to identify the movement speed, which helps to improve the identification accuracy.

For example, based on J, J1, J2, P, P1 and P2 of each joint on joint kinematic chain C, the first matrix [J1] and [P1], the second matrix [J2] and [P2], the third matrix [ J] and [P]. Among them, each row of [J1] corresponds to J1 of each joint, and J1 is a row vector representing the angle. [P1], [J2], [P2], [J] and [P] are generated in a similar way to [J1]. The row vectors correspond to each joint. The only difference is that the row vectors are P1, J2, P2, J and P, I won’t go into details here.

Further, at least two target joints belonging to the same joint kinematic chain are determined based on the parent-child relationship between the multiple joints. As shown in Figure 2, joints connected by the same line are target joints belonging to the same joint kinematic chain. During the movement, these joints will have some linkage relationships, that is, the angles or positions between the joints will affect each other. Therefore, based on the joint identification of the target joints on the same joint kinematic chain whether they are in a fast or slow state, a more accurate result will be obtained. Recognition results and improve recognition accuracy.

Step 304: Perform a weighted summation of the distance between the first matrix and the second matrix and the distance between the third matrix and the second matrix to obtain a distance value.

Substitute the first matrix, the second matrix and the third matrix into the judgment conditions to judge whether the kinematic chain is in a fast or slow state. Specific judgment conditions can be:

H＝R1×F1([J1]–[J2])+R2×F2([P1]–[P2])+R3×F1([J]–[J2])+R4×F2([P]–[ P2]).

Among them, F1([J1]–[J2]) represents the spherical distance between [J1] and [J2], and F2([P1]–[P2]) represents the Euclidean distance between [P1] and [P2]. Similarly, F1([J]–[J2]) represents the spherical distance between [J] and [J2], and F2([P]–[P2]) represents the Euclidean distance between [P] and [P2] distance.

R1 to R4 are the weights of the corresponding sub-items.

Substituting the first matrix, the second matrix and the third matrix into the judgment condition, the distance value H is obtained. In this embodiment, not only the distance value H used for fast and slow state identification is obtained based on the difference in results between two different jitter elimination algorithms, but also the original motion parameter observations are introduced to compare the smoothness of the two jitter elimination algorithms. The difference in the results of a stronger jitter elimination algorithm makes the correlation between the distance value H and the speed state stronger.

Step 305: According to the distance value, select a matching target jitter elimination algorithm to correct the motion parameter observations.

Optionally, based on the value interval to which the distance value belongs, a target jitter elimination algorithm corresponding to the value interval is selected to correct the motion parameter observation.

For example, when H is greater than the threshold h, it is determined to be in a fast motion state, and first-order filtering is applied; otherwise, it is determined to be second-order filtering.

It should be noted that those skilled in the art can understand that the value range of H can also be divided into multiple stages, for example, further divided into corresponding low and low speeds, medium and low speeds, high and low speeds, medium speeds, high and low speeds, medium and high speeds, and high and high speeds. There are multiple more detailed value intervals, and each value interval is configured with a corresponding target jitter elimination algorithm to correct the motion parameter observations, so as to make the jitter elimination algorithm more accurate.

In a possible implementation, the observed motion parameter is a joint angle. Figure 4 is a schematic diagram of the principle of a data processing method based on motion capture provided by an embodiment of the present disclosure. As shown in Figure 4, the observed joint angle is input Go to the kinematic model and obtain the position coordinates of the joints. For target joints belonging to the same joint kinematic chain, first-order filtering and second-order filtering are used to filter the position coordinates of the target joint respectively. Based on the difference between the first-order filter and the second-order filter of each target joint in the same joint kinematic chain, it is judged whether the motion state of the motion capture object is fast or slow.

In a fast motion state, a jitter elimination algorithm that has a slight jitter elimination effect but does not affect fast movements is needed. Therefore, as shown in Figure 4, a first-order filtering algorithm is selected to filter the position coordinates of the target joint in a fast motion state. This eliminates the effects of jitter without affecting fast movements.

In the slow motion state, a jitter elimination algorithm with obvious jitter elimination effect but affecting fast movements is needed. Therefore, as shown in Figure 4, a second-order filtering algorithm is selected to filter the position coordinates of the target joint in the slow motion state. , to eliminate the effects of jitter without worrying about its impact on fast movements.

In this embodiment, after observing the motion capture object to obtain the motion parameter observations of the joints of the motion capture object, the motion parameter observations of each joint are processed using the first jitter elimination algorithm to obtain the first reference quantity, and using The second jitter elimination algorithm performs processing to obtain the second reference quantity. According to the difference between the first reference quantity and the second reference quantity, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observation. The difference between the results of two different jitter elimination algorithms reflects the motion speed of the motion capture object. Then, based on the different motion speed states, a matching target jitter elimination algorithm is selected to correct the motion parameter observations and improve the accuracy of the motion capture object. Action restoration.

Figure 5 is a schematic flowchart of another data processing method based on motion capture provided by an embodiment of the present disclosure. As shown in Figure 5, it includes:

Step 501: Observe the motion capture object to obtain motion parameter observations of the joints of the motion capture object.

Step 502: Use the first jitter elimination algorithm to process the observed motion parameters of the joints to obtain the first reference quantity, and use the second jitter elimination algorithm to process the motion parameter observations to obtain the second reference quantity.

Wherein, the smoothing strength of the second jitter elimination algorithm is higher than that of the first jitter elimination algorithm.

For details of step 501 and step 502, reference may be made to step 101 and step 102 in the previous embodiment. This will not be described again in this embodiment.

Step 503: When there are multiple joints for which the motion parameter observation is observed, compare the difference with a threshold for at least two target joints belonging to the same joint kinematic chain among the multiple joints.

Further, based on the corresponding relationship between the joint kinematic chain and the threshold value, the threshold value corresponding to the joint kinematic chain to which each of the target joints belongs is determined.

Further, at least two target joints belonging to the same joint kinematic chain are determined based on the parent-child relationship between the multiple joints. As shown in Figure 2, joints connected by the same line are target joints belonging to the same joint kinematic chain. During the movement, these joints will have some linkage relationships, that is, the angles or positions between the joints will affect each other. Therefore, based on the joint identification of the target joints on the same joint kinematic chain whether they are in a fast or slow state, a more accurate result will be obtained. Recognition results and improve recognition accuracy.

Step 504: If at least a set number of target joints among at least two target joints satisfy that the difference is greater than the threshold, select the first jitter elimination algorithm as the target jitter elimination algorithm to use the joints. The first reference quantity of each target joint in the kinematic chain corrects the observed motion parameter.

Step 505: If the target joints of at least two target joints that are less than the set number satisfy that the difference is greater than the threshold, select the second jitter elimination algorithm as the target jitter elimination algorithm to use The second reference quantity of each target joint in the joint kinematic chain corrects the observed motion parameter.

According to the relationship between the difference and the threshold, it is determined to use the second reference quantity or the first reference quantity of each target joint in the joint kinematic chain to correct the motion parameter observation, while improving the motion restoration degree of the motion capture object, The amount of calculation is also small. You only need to calculate two jitter elimination algorithms and compare the differences, and then you can select the appropriate algorithm among the two for jitter elimination.

In this embodiment, after observing the motion capture object to obtain the motion parameter observations of the joints of the motion capture object, the motion parameter observations of each joint are processed using the first jitter elimination algorithm to obtain the first reference quantity, and using The second jitter elimination algorithm performs processing to obtain the second reference quantity. According to the difference between the first reference quantity and the second reference quantity, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observation. The difference between the results of two different jitter elimination algorithms reflects the motion speed of the motion capture object. Then, based on the different motion speed states, a matching target jitter elimination algorithm is selected to correct the motion parameter observations and improve the accuracy of the motion capture object. Action restoration.

Figure 6 is a schematic structural diagram of a data processing device 600 based on motion capture provided by an embodiment of the present disclosure. As shown in Figure 6, it includes: an observation module 601, a processing module 602, and a correction module 603.

The observation module 601 is used to observe the motion capture object to obtain motion parameter observations of the joints of the motion capture object.

The processing module 602 is used to process the motion parameter observations of the joints using a first jitter elimination algorithm to obtain a first reference quantity, and use a second jitter elimination algorithm to process the motion parameter observations to obtain a second reference quantity; wherein, The smoothing intensity of the second jitter elimination algorithm is different from that of the first jitter cancellation algorithm.

The correction module 603 is configured to select a target jitter elimination algorithm that matches the difference according to the difference between the first reference quantity and the second reference quantity to correct the motion parameter observation.

As a possible implementation, the correction module 603 includes:

A generation unit configured to generate a first matrix and a second matrix based on at least two target joints belonging to the same joint kinematic chain among the plurality of joints when the motion parameter observation value is observed to be multiple joints. , wherein each row vector in the first matrix corresponds to the target joint and is used to indicate the first reference quantity of the corresponding target joint in the three-dimensional space, and each row vector in the second matrix corresponds to the target joint, using To indicate the second reference quantity corresponding to the target joint in the three-dimensional space.

A selection unit configured to select the target jitter elimination algorithm to correct the motion parameter observation according to the distance between the first matrix and the second matrix.

Optionally, a unit is selected to: generate a third matrix based on the at least two target joints, wherein each row vector in the third matrix corresponds to the target joint and is used to indicate the position of the corresponding target joint in the three-dimensional space. The motion parameter observations; perform a weighted summation of the distance between the first matrix and the second matrix, and the distance between the third matrix and the second matrix to obtain the distance value; according to the distance value, select a matching target jitter elimination algorithm to correct the motion parameter observation.

A selection unit configured to: based on the value interval to which the distance value belongs, select a target jitter elimination algorithm corresponding to the value interval to correct the motion parameter observation.

As another possible implementation, the smoothing intensity of the second jitter elimination algorithm is higher than that of the first jitter elimination algorithm. Based on this, the correction module 603 is used to:

When there are multiple joints from which the motion parameter observation is obtained, compare the difference with a threshold for at least two target joints belonging to the same joint kinematic chain among the multiple joints;

When at least a set number of target joints among at least two target joints satisfy that the difference is greater than the threshold, the first jitter elimination algorithm is selected as the target jitter elimination algorithm to use the joint kinematic chain. The first reference quantity of each target joint corrects the observed motion parameter;

If less than the set number of target joints among at least two target joints satisfy that the difference is greater than the threshold, the second jitter elimination algorithm is selected as the target jitter elimination algorithm to use the joints The second reference quantity of each target joint in the kinematic chain corrects the observed motion parameter.

Optionally, the correction module is further configured to: determine the threshold corresponding to the joint kinematic chain to which each target joint belongs based on the correspondence between the joint kinematic chain and the threshold.

Optionally, the motion capture-based data processing device 600 further includes a determination module configured to determine at least two target joints belonging to the same joint kinematic chain based on the parent-child relationship between the multiple joints.

In some possible embodiments, the observation module 601 is used to:

Observe the joint angles of the motion capture object to obtain the joint angles of each joint;

A kinematics model is used to identify the relative position of each joint point based on the joint angle of each joint, so that the relative position and the joint angle are used as the motion parameter observations.

Or, in other possible embodiments, the observation module 601 is used to:

Observe the joint positions of the motion capture object to obtain the joint position coordinates of each joint;

Determine the joint angle of each joint based on the joint position coordinates of each joint;

The position coordinates and the joint angles are used as the motion parameter observations.

In this embodiment, after observing the motion capture object to obtain the motion parameter observations of the joints of the motion capture object, the motion parameter observations of each joint are processed using the first jitter elimination algorithm to obtain the first reference quantity, and using The second jitter elimination algorithm performs processing to obtain the second reference quantity. According to the difference between the first reference quantity and the second reference quantity, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observation. The difference between the results of two different jitter elimination algorithms reflects the motion speed of the motion capture object. Then, based on the different motion speed states, a matching target jitter elimination algorithm is selected to correct the motion parameter observations and improve the accuracy of the motion capture object. Action restoration.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

7 illustrates a schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to refer to various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in Figure 7, the device 700 includes a computing unit 701, which can be loaded into a RAM (Random Access Memory) according to a computer program stored in a ROM (Read-Only Memory) 702 or from a storage unit 708. The computer program in the memory) 703 to perform various appropriate actions and processes. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. Computing unit 701, ROM 702 and RAM 703 are connected to each other via bus 704. I/O (Input/Output, input/output) interface 705 is also connected to bus 704.

Multiple components in the device 700 are connected to the I/O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, optical disk, etc. ; and communication unit 709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices through computer networks such as the Internet and/or various telecommunications networks.

Computing unit 701 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence, artificial intelligence) computing chips, various running The computing unit of the machine learning model algorithm, DSP (Digital Signal Processor, digital signal processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the motion capture-based data processing method may be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 700 via ROM 702 and/or communication unit 709 . When the computer program is loaded into RAM 703 and executed by computing unit 701, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the motion capture-based data processing method in any other suitable manner (eg, by means of firmware).

Various implementations of the systems and technologies described above in this article can be implemented in digital electronic circuit systems, integrated circuit systems, FPGA (Field Programmable Gate Array, field programmable gate array), ASIC (Application-Specific Integrated Circuit, application-specific integrated circuit) , ASSP (ApplicationSpecific Standard Product, dedicated standard product), SOC (System On Chip, system on chip), CPLD (Complex Programmable Logic Device, complex programmable logic device), computer hardware, firmware, software, and/or they realized in a combination. These various embodiments may include implementation in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor The processor, which may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device. An output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions specified in the flowcharts and/or block diagrams/ The operation is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include electrical connections based on one or more wires, laptop disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory, erasable programmable read-only memory) Or flash memory, optical fiber, CD-ROM (CompactDisc Read-Only Memory, portable compact disk read-only memory), optical storage device, magnetic storage device, or any suitable combination of the above.

In order to provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, CRT (Cathode-Ray Tube, cathode ray tube)) or LCD ( Liquid Crystal Display (liquid crystal display) monitor); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be provided in any form, including Acoustic input, voice input or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., A user's computer having a graphical user interface or web browser through which the user can interact with implementations of the systems and technologies described herein), or including such backend components, middleware components, or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communications network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

Computer systems may include clients and servers. Clients and servers are generally remote from each other and typically interact over a communications network. A client and server relationship is created by computer programs running on corresponding computers and having a client-server relationship with each other. The server can be a cloud server, also known as cloud computing server or cloud host. It is a host product in the cloud computing service system to solve the problem of traditional physical host and VPS service ("Virtual Private Server", or "VPS" for short) Among them, there are defects such as difficult management and weak business scalability. The server can also be a distributed system server or a server combined with a blockchain.

Among them, it should be noted that artificial intelligence is the study of using computers to simulate certain human thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It has both hardware-level technology and software-level technology. Artificial intelligence hardware technology generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, etc.; artificial intelligence software technology mainly includes computer vision technology, speech recognition technology, natural language processing technology, and machine learning/depth Learning, big data processing technology, knowledge graph technology and other major directions.

It should be understood that various forms of the process shown above may be used, with steps reordered, added or deleted. For example, each step described in the present disclosure can be executed in parallel, sequentially, or in a different order. As long as the desired results of the technical solution disclosed in the present disclosure can be achieved, there is no limitation here.

The above-mentioned specific embodiments do not constitute a limitation on the scope of the present disclosure. It will be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions are possible depending on design requirements and other factors. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this disclosure shall be included in the protection scope of this disclosure.

Claims (21)

1. A data processing method based on motion capture, including: Observe the motion capture object to obtain motion parameter observations of the joints of the motion capture object; The motion parameter observations of the joints are processed using a first jitter elimination algorithm to obtain the first reference quantity, and are processed using a second jitter elimination algorithm to obtain the second reference quantity; wherein, the second jitter elimination algorithm The smoothing strength is different from the first jitter elimination algorithm; According to the difference between the first reference quantity and the second reference quantity, a target jitter elimination algorithm matching the difference is selected to correct the motion parameter observation. 2. The method according to claim 1, wherein, according to the difference between the first reference quantity and the second reference quantity, a target jitter elimination algorithm matching the difference is selected to calculate the motion parameter Observations are corrected, including: When there are multiple joints from which the motion parameter observation is obtained, a first matrix and a second matrix are generated based on at least two target joints belonging to the same joint kinematic chain among the plurality of joints, wherein: Each row vector in the first matrix corresponds to the target joint and is used to indicate the first reference quantity of the corresponding target joint in the three-dimensional space. Each row vector in the second matrix corresponds to the target joint and is used to indicate the corresponding target joint. the second reference quantity in the three-dimensional space; According to the distance between the first matrix and the second matrix, the target jitter elimination algorithm is selected to correct the motion parameter observations. 3. The method according to claim 2, wherein the smoothing strength of the second jitter elimination algorithm is higher than that of the first jitter elimination algorithm; distance, select the target jitter elimination algorithm to correct the motion parameter observations, including: Based on the at least two target joints, generate a third matrix, wherein each row vector in the third matrix corresponds to the target joint and is used to indicate the observed motion parameter of the corresponding target joint in the three-dimensional space; Perform a weighted summation of the distance between the first matrix and the second matrix and the distance between the third matrix and the second matrix to obtain a distance value; According to the distance value, a matching target jitter elimination algorithm is selected to correct the motion parameter observations. 4. The method according to claim 3, wherein selecting a matching target jitter elimination algorithm to correct the motion parameter observations according to the distance value includes: Based on the value interval to which the distance value belongs, a target jitter elimination algorithm corresponding to the value interval is selected to correct the motion parameter observation. 5. The method according to claim 1, wherein the smoothing strength of the second jitter elimination algorithm is higher than that of the first jitter elimination algorithm, and the smoothing strength is higher than that of the first jitter elimination algorithm according to the first reference quantity and the second reference quantity. The difference between the differences is selected, and the target jitter elimination algorithm matching the difference is selected to correct the motion parameter observations, including: When there are multiple joints from which the motion parameter observation is obtained, compare the difference with a threshold for at least two target joints belonging to the same joint kinematic chain among the multiple joints; When at least a set number of target joints among at least two target joints satisfy that the difference is greater than the threshold, the first jitter elimination algorithm is selected as the target jitter elimination algorithm to use the joint kinematic chain. The first reference quantity of each target joint corrects the observed motion parameter; If less than the set number of target joints among at least two target joints satisfy that the difference is greater than the threshold, the second jitter elimination algorithm is selected as the target jitter elimination algorithm to use the joints The second reference quantity of each target joint in the kinematic chain corrects the observed motion parameter. 6. The method of claim 5, wherein the method further comprises: Based on the correspondence between the joint kinematic chain and the threshold, the threshold corresponding to the joint kinematic chain to which each target joint belongs is determined. 7. The method according to claim 2 or 5, wherein the method further includes: According to the parent-child relationship between the multiple joints, at least two target joints belonging to the same joint kinematic chain are determined. 8. The method according to any one of claims 1 to 6, wherein observing the motion capture object to obtain motion parameter observations of the joints of the motion capture object includes: Observe the joint angles of the motion capture object to obtain the joint angles of each joint; A kinematics model is used to identify the relative position of each joint point based on the joint angle of each joint, so that the relative position and the joint angle are used as the motion parameter observations. 9. The method according to any one of claims 1 to 6, wherein observing the motion capture object to obtain motion parameter observations of the joints of the motion capture object includes: Observe the joint positions of the motion capture object to obtain the joint position coordinates of each joint; Determine the joint angle of each joint based on the joint position coordinates of each joint; The position coordinates and the joint angles are used as the motion parameter observations. 10. A data processing device based on motion capture, including: An observation module, used to observe the motion capture object to obtain motion parameter observations of the joints of the motion capture object; A processing module configured to process the motion parameter observations of the joint using a first jitter elimination algorithm to obtain a first reference quantity, and to process the motion parameter observations using a second jitter elimination algorithm to obtain a second reference quantity; wherein, the The smoothing strength of the second jitter removal algorithm is different from the first jitter cancellation algorithm; A correction module, configured to select a target jitter elimination algorithm that matches the difference according to the difference between the first reference quantity and the second reference quantity to correct the motion parameter observation. 11. The device according to claim 10, wherein the correction module includes: A generation unit configured to generate a first matrix and a second matrix based on at least two target joints belonging to the same joint kinematic chain among the plurality of joints when the motion parameter observation value is observed to be multiple joints. , wherein each row vector in the first matrix corresponds to the target joint and is used to indicate the first reference quantity of the corresponding target joint in the three-dimensional space, and each row vector in the second matrix corresponds to the target joint, using To indicate the second reference quantity corresponding to the target joint in the three-dimensional space; A selection unit configured to select the target jitter elimination algorithm to correct the motion parameter observation according to the distance between the first matrix and the second matrix. 12. The device according to claim 11, wherein the selecting unit is used for: Based on the at least two target joints, generate a third matrix, wherein each row vector in the third matrix corresponds to the target joint and is used to indicate the observed motion parameter of the corresponding target joint in the three-dimensional space; Perform a weighted summation of the distance between the first matrix and the second matrix and the distance between the third matrix and the second matrix to obtain a distance value; According to the distance value, a matching target jitter elimination algorithm is selected to correct the motion parameter observations. 13. The device according to claim 12, wherein the selecting unit is used for: Based on the value interval to which the distance value belongs, a target jitter elimination algorithm corresponding to the value interval is selected to correct the motion parameter observation. 14. The device according to claim 10, wherein the smoothing strength of the second jitter elimination algorithm is higher than that of the first jitter elimination algorithm; the correction module is configured to: When there are multiple joints from which the motion parameter observation is obtained, compare the difference with a threshold for at least two target joints belonging to the same joint kinematic chain among the multiple joints; When at least a set number of target joints among at least two target joints satisfy that the difference is greater than the threshold, the first jitter elimination algorithm is selected as the target jitter elimination algorithm to use the joint kinematic chain. The first reference quantity of each target joint corrects the observed motion parameter; If less than the set number of target joints among at least two target joints satisfy that the difference is greater than the threshold, the second jitter elimination algorithm is selected as the target jitter elimination algorithm to use the joints The second reference quantity of each target joint in the kinematic chain corrects the observed motion parameter. 15. The device according to claim 14, wherein the correction module is also used for: Based on the correspondence between the joint kinematic chain and the threshold, the threshold corresponding to the joint kinematic chain to which each target joint belongs is determined. 16. The device according to claim 11 or 14, wherein the device further includes: A determining module, configured to determine at least two target joints belonging to the same joint kinematic chain based on the parent-child relationship between the multiple joints. 17. The device according to any one of claims 10-15, wherein the observation module is used for: Observe the joint angles of the motion capture object to obtain the joint angles of each joint; A kinematics model is used to identify the relative position of each joint point based on the joint angle of each joint, so that the relative position and the joint angle are used as the motion parameter observations. 18. The device according to any one of claims 10-15, wherein the observation module is used for: Observe the joint positions of the motion capture object to obtain the joint position coordinates of each joint; Determine the joint angle of each joint based on the joint position coordinates of each joint; The position coordinates and the joint angles are used as the motion parameter observations. 19. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can perform any one of claims 1-9. Methods. 20. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method according to any one of claims 1-9. 21. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-9.