CN102156751A - Method and device for extracting video fingerprint - Google Patents

Abstract

The invention discloses a method for extracting a video fingerprint, which is used for carrying out the purpose of extracting a video fingerprint, so that the video can be identified in a relatively accurate way, the similarity comparison of videos and warehousing management and the like can be favorable to be realized by the fingerprint. The method provided by the invention comprises the following steps of: partitioning a gray level image of one frame in a video file; obtaining average gray values for the partitions of the gray level image of one frame; obtaining a fingerprint of the gray level image of the frame according to the average gray values of each partition; merging the fingerprints of gray level images of a plurality of frames; and taking the merged fingerprint as the fingerprint of the video file. The invention also discloses a device for realizing the method.

Description

A method and device for extracting video fingerprints

technical field

The invention relates to the fields of computers and images, in particular to a method and device for extracting video fingerprints.

Background technique

With the development of network technology and multimedia services, video services have penetrated into people's lives. In the process of video uploading or video downloading, it is inseparable for service providers to query and retrieve videos. However, the size of a video file is usually relatively large, basically in several megabytes (M) or G. Therefore, how to quickly retrieve such large files has become the focus of attention in the industry.

At present, content identification (CID) code technology is proposed. The CID code is a string composed of several bytes that can uniquely identify a video file. The CID code can be obtained after calculating the video data in the video file according to the set algorithm. The set algorithm may be any algorithm that obtains different processing results for different data processing, such as a hash (hash) algorithm. Specifically, the hash algorithm can adopt Message-Digest Algorithm 5 (MD5), MD4, Secure Hash Algorithm (Secure Hash Algorithm, SHA), etc.

It can be seen that the CID code is a technology for identifying a complete video file. If two video files are exactly the same, their CID codes are the same, that is to say, they can be compared or retrieved through the CID codes. However in real life more video files are not exactly the same. For example, one is a complete video file, and the other is a video file without opening song. For the user, the content of these two video files is the same, and they are all the targets of search. These two video files cannot be searched at the same time. Therefore, it is urgent to obtain a new fingerprint extraction technology, so as to compare the similarity of videos through the fingerprint.

Contents of the invention

Embodiments of the present invention provide a method and device for extracting video fingerprints, which are used to extract video fingerprints, identify videos more accurately, and facilitate video similarity comparison and storage management through the fingerprints.

A method for extracting video fingerprints, comprising the following steps:

Block a frame of grayscale image in the video file;

Obtain the average gray value of each block in a frame of gray image;

Obtain the fingerprint of the grayscale image of the frame according to the correlation value of the average grayscale value of each block;

The fingerprints of multiple grayscale images are merged, and the merged fingerprints are used as the fingerprints of the video file.

A method for video similarity comparison, comprising the following steps:

Obtain the m-frame fingerprints of the two video files respectively, and each frame fingerprint is: divide a frame of grayscale image into blocks; obtain the average gray value of each block in a frame of grayscale image; according to the average gray value of each block Obtain the fingerprint of the grayscale image of the frame through the correlation value of the intensity value;

According to the m frame fingerprints of the two video files, obtain the similarity value of the m frames of the two video files;

Determine whether the obtained similarity value is greater than a preset similarity threshold, if greater, then determine that the two video files are not similar with respect to the m frames, otherwise determine that the two video files are similar with respect to the m frames.

A device for extracting video fingerprints, comprising:

The block module is used to block a frame of grayscale image in the video file;

Calculation module, used to obtain the average gray value of each block in a frame of gray image;

The fingerprint module is used to obtain the fingerprint of the gray-scale image of the frame according to the correlation value of the average gray-scale value of each block, and merge the fingerprints of the gray-scale images of multiple frames, and use the merged fingerprint as the fingerprint of the video file.

A device for video similarity comparison, comprising:

The acquisition module is used to obtain the fingerprints of each m frames of two video files respectively, and each frame of fingerprints is: a frame of grayscale image is divided into blocks; the average gray value of each block in the frame of grayscale image is obtained; according to each The correlation value of the average gray value of the block obtains the fingerprint of the gray image of the frame;

The similarity calculation module is used to obtain the similarity value of the m frames of the two video files according to the m frame fingerprints of the two video files;

The comparison module is used to judge whether the similarity value obtained is greater than a preset similarity threshold, if greater, then determine that the two video files are not similar about the m frame, otherwise determine that the two video files are similar about the m frame.

The embodiment of the present invention obtains the fingerprint of the frame image through the gray value in the frame image, realizes the identification of the image content of a frame, facilitates the comparison of image content frame by frame, and then realizes the similarity comparison between video files.

Description of drawings

Fig. 1 is the flow chart of the method for extracting video fingerprint in the embodiment of the present invention;

Fig. 2 is a schematic diagram of the sequence numbers corresponding to the block gray value in the embodiment of the present invention;

FIG. 3 is a schematic diagram of an interference zone in an embodiment of the present invention;

4 is a flowchart of a method for video similarity comparison in an embodiment of the present invention;

FIG. 5A is a flowchart of a method for updating a basic fingerprint library in an embodiment of the present invention;

5B is a schematic diagram of the length relationship between the shortest media segment, the segment to be queried, and the slice in the embodiment of the present invention;

FIG. 6 is a main structural diagram of a device 600 in an embodiment of the present invention;

FIG. 7 is a detailed structural diagram of a device 600 in an embodiment of the present invention;

FIG. 8 is a main structural diagram of a device 800 in an embodiment of the present invention;

FIG. 9 is a detailed structural diagram of a device 800 in an embodiment of the present invention.

Detailed ways

The embodiment of the present invention obtains the fingerprint of the frame image through the gray value in the frame image, realizes the identification of the image content of a frame, facilitates the comparison of image content frame by frame, and then realizes the similarity comparison between video files.

Referring to Fig. 1, the method flow process of extracting video fingerprint in the present embodiment is as follows:

Step 101: Divide a frame of grayscale image in a video file into blocks. The obtained blocks are also called macro blocks, and the size of each block may be 4×4 or 8×8.

Step 102: Obtain the average gray value of each block in a frame of gray image.

Step 103: Obtain the fingerprint of the grayscale image of the frame according to the correlation value of the average grayscale value of each block.

Step 104: Merge the fingerprints of the multiple frames of grayscale images, and use the merged fingerprint as the fingerprint of the video file.

There are multiple specific implementation methods in step 103. For example, in the first method, the average gray value of each block in a frame of gray image is sorted, and the serial number corresponding to each block is obtained, and the gray value of each block is obtained according to the number of blocks in the gray image. The order in is to combine the sequence numbers corresponding to each block in order, and use the combined string of symbols as the fingerprint of the grayscale image of the frame. As in the second method, compare the average gray value of each block in a frame of gray image, combine the comparison results represented by 0 or 1 in the order of comparison, and use the combined string of symbols as the frame Fingerprints of grayscale images. As in the third method, according to the obtained average gray value, the overall gradient direction value of each block is obtained, and the overall gradient direction value corresponding to each block is sequentially combined according to the order of the block in the gray image, and the combined The last string of symbols is used as the fingerprint of the grayscale image of the frame.

Specifically, for the first method, taking 12 macroblocks per frame as an example, the average gray value of each block is 80, 50, 110, 30, 60, 90, 180, 160, 70, 120, 20 , 40, and then sort the average gray value of each block in a frame of gray image, and obtain the sequence number corresponding to each block. Referring to the frame shown in Figure 2, it provides the serial numbers corresponding to the brightness values of each macroblock, these serial numbers sequentially form the frame fingerprint, and the frame fingerprint can be stored in the form of an array, then the frame fingerprint is in the form of: seq[12 ]={7, 4, 9, 2, 5, 8, 12, 11, 6, 10, 1, 3}, for example, the gray value of the first macroblock ranks 7th among all the gray values of the macroblocks, Then the value of the first element in the array gray[12] is 7.

For the second method, take each frame including 12 macroblocks as an example, compare the average gray value of each block in a frame of gray image, or compare the sequence numbers of the average gray value after sorting Two-by-two comparison, if gray[i]<gray[j], the result is 1, otherwise it is 0; or, if gray[i]<gray[j], the result is 0, otherwise it is 1; gray[] means Array, i represents the i-th sub-block, j represents the j-th sub-block, i=0,..., P, j=0,..., P, P is a frame of grayscale image that participates in the synthesis of fingerprints The total number of chunks. The value of each element in the array is the average gray value of the block or the serial number of the sorting, etc. In short, any value that can reflect the relationship between the gray scale is acceptable.

For the third method, quantize each frame into 8 directions, or take 12 macroblocks as an example, each frame has 12 gradient direction values, and the gradient direction values are data related to the gray value, also That is to say, the sequence of these 12 gradient direction values constitutes the frame fingerprint of the frame.

In step 104, the fingerprints of the multi-frame grayscale images can be merged in a variety of ways, for example, the fingerprints of the grayscale images of all frames of the video are connected, or the grayscales of the frames of the front, middle and rear three segments of the video are taken The fingerprints of the images are concatenated and so on.

In addition, each frame does not necessarily have the entire picture as an image. Usually, there are several lines of black borders at the top and bottom of a frame. There will be subtitles in the middle of the picture, and icons in the upper left corner of the picture. These parts will affect the frame content. The judgment of similarity will affect the accuracy of fingerprints and the effect of similarity comparison. Therefore, this embodiment filters video files before obtaining fingerprints. The filtering operation includes at least one of the following operations: removing black borders and removing interference regions.

To black edge, as the name suggests, is to remove the black edge in each frame. Specifically, from one side of the image to the middle position, the pixel mean value (or average gray value) of each line is counted line by line, and the first line whose pixel mean value is greater than the pixel threshold is determined, and from this line to the side The area is bordered in black, which does not include the row. However, sometimes the content of the picture is relatively dark, and it is easy to determine part of the content of the picture as black borders, so it needs to be detected. Specifically, determine the black border area in a frame of grayscale image; judge whether the size of the area other than the black border in a frame of grayscale image is not less than the preset size threshold, if so, remove the black border area, otherwise Do not remove black borders for the frame, or perform black border removal operations on the frame according to the black border areas of other frames that meet the judgment conditions. There are many detection methods, and this embodiment only provides a feasible method for reference, and other detection methods are applicable to this embodiment. In order to improve the accuracy of detection, it can also be judged whether the gray average value of the area except the black border in a frame of gray image is greater than the preset gray threshold; the gray value of the area except the black border in a frame of gray image When the average value is greater than the preset grayscale threshold, remove the black border area, otherwise, do not remove black borders for the frame, or perform black border removal operations on the frame according to the black border regions of other frames that meet the judgment conditions.

Removing the interference area is to delete the data in the preset interference area. It is not necessary to obtain the data related to the gray value of this part of the area, so it does not participate in the acquisition of the frame fingerprint. In this embodiment, the interference area is the shaded area as shown in Figure 3, including the two macroblocks below, usually the subtitle area, and the macroblocks including the upper left and upper right corners, usually the icon area, such as the station logo of a TV station or advertising etc. It is also possible to set other macroblocks as the interference area according to the actual situation. The characteristic of the interference area is that the content of the interference area is basically inconvenient when the pictures of different frames change, especially the brightness of the interference area is basically inconvenient.

After the frame fingerprint is obtained, the two video files can be compared frame by frame according to the frame fingerprint to determine whether there are frames with similar (including the same) content between the two video files, and then determine the similarity between the two video files. Spend. The following describes the video similarity comparison process.

Referring to Fig. 4, the method flow of video similarity comparison in the present embodiment is as follows:

Step 401: Obtain m-frame fingerprints of two video files respectively. The fingerprint of each frame is: block a frame of grayscale image; obtain the average gray value of each block in a frame of gray image; obtain the gray value of the frame of gray image according to the correlation value of the average gray value of each block fingerprint.

Step 402: According to the m-frame fingerprints of the two video files, obtain the similarity value of the m-frames of the two video files.

Step 403: Determine whether the obtained similarity value is greater than a preset similarity threshold, if so, proceed to step 404, otherwise proceed to step 405.

Step 404: Determine that the two video files are not similar with respect to the m frames.

Step 405: Determine that the two video files are similar with respect to the m frames.

The specific implementation in step 402 is as follows: calculating the Manhattan (Manhattan) distance value between m-frame fingerprints of two video files. For example, the m-frame fingerprint of a video file is 3 5 6 7 8 9...11 10, and the m-frame fingerprint of another video file is 4 5 6 7 8 9...11 10, then the distance between two sets of m frames is |3-4|+|5-5|+|6-6|+|7-7|+|8-8|+|9-9|+...+|11-11|+|10- 10|. The distance value is the similarity value. The smaller the distance is, the greater the similarity is. If the distance is 0, the fingerprints of the two sets of m frames are the same, and it can be determined that the video content of the two sets of m frames is the same.

In this embodiment, grayscale values are used as fingerprints of video frames. In very special cases, some frame content of two video files may be different but grayscale is the same. Therefore, before similarity comparison, the preferred The method is that the selected m frames in each video file are the m frames after removing the consecutive multiple frames with drastic changes, that is, the consecutive multiple frames with drastic changes in the video file are removed. The specific implementation is such as obtaining the Hamming distance between every adjacent two frames in the video file; determining whether the Hamming distance between the frames exceeding the preset ratio in the continuous multiple frames is greater than the preset distance threshold, and if so, Then it is determined that the continuous multi-frame is a continuous multi-frame with drastic change and removed, otherwise it is kept.

For continuous multiple frames with drastic changes, it will affect the recognition effect and need to be removed. There are many ways to judge whether it is a drastic change. In this embodiment, the Hamming distance (hamming distance) between two adjacent frames is used to judge. For example, the fingerprint of frame f(i) is 0 1 2 5 6 3 4 8 9 7 10 11, and the fingerprint of frame f(i+1) is 0 3 1 5 6 2 4 89 7 10 11, then the Hamming of the adjacent frame The distance is H(i+1)=|0-0|+|1-3|+|2-1|+...+|10-10|+|11-11|=4. Determine whether the Hamming distance between frames exceeding a preset ratio (such as 2/3) is greater than a preset distance threshold (such as 72) in the continuous multi-frames, and if so, determine that the continuous multi-frames are continuous with drastic changes Multiple frames and remove. For example, if the Hamming distance between more than 2/3 of a continuous multi-frame and its adjacent frame is greater than 72, it is determined that the continuous multi-frame changes drastically and needs to be removed.

After comparing the similarity of the two video files, the basic fingerprint library of the video files can be updated according to the comparison results. For the specific implementation process, see the following process.

One of the two video files is a basic video file in the basic fingerprint library, and the other video file is a video file to be queried;

Referring to FIG. 5A, the flow of the method for updating the basic fingerprint library in this embodiment is as follows:

Step 501: Compare the fingerprints of multiple m-frames of the video file to be queried with the fingerprints of multiple m-frames of the basic video file for similarity comparison.

Step 502: Determine the content inclusion relationship between the video file to be queried and the basic video file according to the comparison result.

Step 503: Update the basic fingerprint library according to the obtained content inclusion relationship.

In this embodiment, the multiple groups of m frames of the video file to be queried are part of equal-length slices in the video file to be queried, and the length sum of the multiple groups of m frames (that is, the segment to be queried) and the slice does not exceed the length of the basic video file. The length of the shortest media segment composed of m frames. The length relationship among the shortest media segment, the segment to be queried and the slice is shown in FIG. 5B . The length relationship can ensure that the shortest media segment contains at least one complete segment to be queried in terms of length, so as to compare the similarity of the segments.

The content inclusion relationship includes: no overlap at all, no overlap at all, the base video file is included in the video file to be queried, and the base video file contains the video file to be queried.

If multiple groups of m-frame fingerprints in the video file to be queried all correspond to m-frame fingerprints with the same comparison result in the basic video file, it is determined that the basic video file includes the video file to be queried. If multiple groups of m-frame fingerprints in the basic video file all correspond to m-frame fingerprints with the same comparison result in the video file to be queried, then it is determined that the basic video file is included in the video file to be queried. If the comparison results of the fingerprints of some frames in the plurality of m frame fingerprints of the video file to be queried are the same, it is determined that the video file to be queried is not completely overlapped with the basic video file. If none of the sets of m-frame fingerprints in the video file to be queried has m-frame fingerprints with the same comparison result in the basic video file, it is determined that there is no overlap between the video file to be queried and the basic video file.

When the content inclusion relationship is completely non-overlapping, add the index of the video file to be queried in the basic fingerprint database, and classify the video file to be queried into the basic fingerprint database.

When the content inclusion relationship is incomplete overlap, add the index of the video file to be queried in the basic fingerprint database, classify the video file to be queried into the basic fingerprint database, and record the difference between the video file to be queried and the basic video file in the basic fingerprint database. overlapping position.

The content inclusion relationship is that when the base video file is included in the video file to be queried, the index of the video file to be queried is used to replace the index of the base video file in the base fingerprint database, and the base video file is replaced with the video file to be queried in the base fingerprint database .

The content inclusion relationship is that when the basic video file contains the video file to be queried, the index of the video file to be queried is added to the basic fingerprint database.

For service providers, the basic fingerprint library is usually updated in batches. In order to improve update efficiency, it is necessary to determine the storage location of each video file to be queried in the basic fingerprint database when updating the basic fingerprint library according to multiple multimedia files to be queried, and sort the video files to be queried according to the storage location, according to the sorted Sequentially update the video files to be queried to the basic fingerprint database.

When the similarity and relative positional relationship between a large number of video files are indexed in the database, fast video retrieval can be realized. Here, the unique identifier of each video is the hash value of the binary file of the video file.

Extract the relevant information of its client, such as client resolution, supported video formats, etc. Inquire about relevant information. Then according to the specified query information (file hash, resolution, format, etc.), retrieve the video with the same content as the video to be queried from the database, and the acquired video to be recommended contains the file in content and meets the resolution of the mobile phone and format information. According to the selection result, the obtained link address of the video file satisfying the above conditions is submitted to the client. The client then requests other links with the same binary file as the link according to the link, so as to realize fast download and playback of the video.

The process of fingerprint extraction, similarity comparison and update of the basic fingerprint database has been described above, and the internal structure and functions of the device for realizing the above process will be introduced below.

Referring to FIG. 6 , an apparatus 600 for extracting video fingerprints in this embodiment includes: a block module 601 , a calculation module 602 and a fingerprint module 603 .

Blocking module 601 is used to block a frame of grayscale image in the video file.

The calculation module 602 is used to obtain the average gray value of each block in a frame of gray image.

The fingerprint module 603 is used to obtain the fingerprint of the frame grayscale image according to the correlation value of the average grayscale value of each block, and merge the fingerprints of multiple frames of grayscale images, and use the merged fingerprint as the fingerprint of the video file. Specifically, the fingerprint module 603 sorts the average gray value of each block in a frame of gray image, and obtains the sequence numbers corresponding to each block, and sorts the sequence numbers corresponding to each block according to the order of the blocks in the gray image. Combining in order, the combined string of symbols is used as the fingerprint of the grayscale image of the frame; or, the average grayscale value of each block in a frame of grayscale image is compared in pairs, and the comparison result represented by 0 or 1 is according to The order of comparison is combined, and the combined string of symbols is used as the fingerprint of the grayscale image of the frame; or, according to the obtained average grayscale value, the overall gradient direction value of each block is obtained, according to the block in the grayscale image Combine the overall gradient direction values corresponding to each block sequentially, and use the combined string of symbols as the fingerprint of the grayscale image of the frame.

When comparing the average gray value of each block in a frame of gray image, the fingerprint module 602 compares gray[i] with gray[j]. If gray[i]<gray[j], the result is 1, otherwise it is 0; or, if gray[i]<gray[j], the result is 0, otherwise it is 1; gray[] represents an array, i represents the i-th block, j represents the j-th block, i=0, . . . , P, j=0, .

The device 600 also includes: a filtering module 604, as shown in FIG. 7, the filtering module 604 is used to perform a black border removal operation on a frame of grayscale image.

Specifically, the filter module 604 determines the black border area in a frame of grayscale image, and judges whether the size of the area other than the black border in a frame of grayscale image is not less than a preset size threshold, and if so, removes the black border area.

The device 600 further includes: a detection module 605 for judging whether the average gray value of a region other than the black border in a frame of gray image is greater than a preset gray threshold. The filtering module 604 removes the black border region when the gray average value of the region except the black border in a frame of grayscale image is greater than a preset gray threshold.

Referring to FIG. 8 , an apparatus 800 for video similarity comparison in this embodiment includes: an acquisition module 801 , a similarity calculation module 802 and a comparison module 803 .

The obtaining module 801 is used to obtain m-frame fingerprints of two video files respectively. The fingerprint of each frame is: block a frame of grayscale image; obtain the average gray value of each block in a frame of gray image; obtain the gray value of the frame of gray image according to the correlation value of the average gray value of each block fingerprint. That is, the acquiring module 801 can acquire the fingerprint from the device 600 . Apparatus 600 may exist as a module in apparatus 800 .

The similarity calculation module 802 is used to obtain the similarity value of the m frames of the two video files according to the m frame fingerprints of the two video files. The similarity calculation module 802 calculates the Manhattan distance value between m-frame fingerprints of two video files.

The comparison module 803 is used to determine whether the obtained similarity value is greater than a preset similarity threshold, if greater, then determine that the two video files are not similar with respect to the m frame, otherwise determine that the two video files are similar with respect to the m frame.

The m frames in each video file are the m frames after removing the consecutive multiple frames with drastic changes.

The device 800 also includes: a filtering module 804, as shown in FIG. 9, the filtering module 804 is used to obtain the Hamming distance between every two adjacent frames in the video file; determine whether there are frames exceeding the preset ratio in the continuous multiple frames The Hamming distance between them is greater than a preset distance threshold, and if so, the continuous multi-frames are determined to be continuous multi-frames with drastic changes and removed. If device 600 is located in device 800, filter module 604 and filter module 804 may be the same module.

The device 800 can also be used to update the basic fingerprint database, one of the two video files is the basic video file in the basic fingerprint database, and the other video file is the video file to be queried.

The comparison module 803 is also used to compare the similarities between the fingerprints of multiple groups of m frames of the video file to be queried and the fingerprints of multiple groups of m frames of the basic video file;

The apparatus 800 further includes: a relationship module 805 and an update module 806 .

The relationship module 805 is configured to determine the content inclusion relationship between the video file to be queried and the basic video file according to the comparison result.

An update module 806, configured to update the basic fingerprint library according to the obtained content inclusion relationship.

In this embodiment, the multiple groups of m frames of the video file to be queried are part of equal-length slices in the video file to be queried, and the length sum of the multiple groups of m frames and the slice does not exceed the length of the multiple groups of m frames in the basic video file. The length of the shortest media segment.

The content inclusion relationship includes: no overlap at all, no overlap at all, the base video file is included in the video file to be queried, and the base video file contains the video file to be queried.

Relational module 805 if multiple groups of m frame fingerprints in the video file to be queried all correspond to m frame fingerprints with the same comparison result in the basic video file, then the relational module 805 determines that the basic video file includes the video file to be queried; Multiple groups of m frame fingerprints in the video file to be queried correspond to m frame fingerprints with the same comparison result, then the relationship module 805 determines that the basic video file is included in the video file to be queried; if multiple groups of m frame fingerprints of the video file to be queried Part of the frame fingerprint comparison results are the same in the middle, then the relationship module 805 determines that there is no complete overlap between the video file to be queried and the basic video file; the same m-frame fingerprints, the relationship module 805 determines that there is no overlap between the video file to be queried and the basic video file.

When the content inclusion relationship is completely non-overlapping, the update module 806 increases the index of the video file to be queried in the basic fingerprint database, and the video file to be queried is classified into the basic fingerprint database;

When the content inclusion relationship is incomplete overlap, the update module 806 adds the index of the video file to be queried in the basic fingerprint database, puts the video file to be queried into the basic fingerprint database, and records the video file to be queried and the basic video file in the basic fingerprint database. overlapping positions between files;

When the content inclusion relationship is that the basic video file is included in the video file to be queried, the update module 806 replaces the index of the basic video file with the index of the video file to be queried in the basic fingerprint database, and replaces the index of the video file with the video file to be queried in the basic fingerprint database. base video file;

When the content inclusion relationship is that the basic video file contains the video file to be queried, the update module 806 adds an index of the video file to be queried in the basic fingerprint database.

Preferably, when the update module 806 needs to update the basic fingerprint library according to a plurality of video files to be queried, determine the storage location of each video file to be queried in the basic fingerprint library, sort the video files to be queried according to the storage location, and sort according to In the following sequence, the video files to be queried are updated to the basic fingerprint library.

The embodiment of the present invention obtains the fingerprint of the frame image through the gray value in the frame image, realizes the identification of the image content of a frame, facilitates the comparison of image content frame by frame, and then realizes the similarity comparison between video files. In the embodiment of the present invention, on the basis of obtaining the fingerprint of the frame image, the content similarity comparison between video files is realized through the fingerprint. The comparison result can be used for video retrieval, download and upload, and the basic fingerprint library and storage of video Video database establishment and update, etc.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims (32)

1. a method of extracting video finger print is characterized in that, may further comprise the steps:

Frame gray level image in the video file is carried out piecemeal;

Obtain the average gray value of each piecemeal in the frame gray level image;

Obtain the fingerprint of this frame gray level image according to the correlation of the average gray value of each piecemeal;

The fingerprint of multiframe gray level image is merged, and the fingerprint after will merging is as the fingerprint of video file.

2. the method for claim 1 is characterized in that, the step that obtains the fingerprint of this frame gray level image according to the correlation of the gray-scale value of each piecemeal comprises:

Average gray value to each piecemeal in the frame gray level image sorts, and obtain each piecemeal corresponding sequence number, according to the order of piecemeal in gray level image with each piecemeal corresponding sequence number sequential combination, with the string symbol after the combination as the fingerprint of this frame gray level image; Perhaps

Average gray value to each piecemeal in the frame gray level image compares in twos, will be by 0 or 1 comparative result of representing according to sequential combination relatively, with the fingerprint of the string symbol after the combination as this frame gray level image; Perhaps

According to acquired average gray value, obtain the overall gradient direction value of each piecemeal, according to the overall gradient direction value sequential combination of the order of piecemeal in gray level image, with the fingerprint of the string symbol after the combination as this frame gray level image with each piecemeal correspondence.

3. method as claimed in claim 2 is characterized in that, the step that the average gray value of each piecemeal in the frame gray level image is compared in twos comprises:

If gray[i]＜gray[j], then the result is 1, otherwise is 0; Perhaps, if gray[i]＜gray[j], then the result is 0, otherwise is 1; Gray[] the expression array, i represents i piecemeal, j represents j piecemeal, i=0 ..., P, j=0 ..., P, P are the sum that participates in the piecemeal of synthetic fingerprint in the frame gray level image.

4. the method for claim 1 is characterized in that, a frame gray level image is carried out also comprising step before the piecemeal: a frame gray level image is gone the black surround operation.

5. method as claimed in claim 4 is characterized in that, goes the step of black surround operation to comprise to a frame gray level image:

Determine the black surround zone in the frame gray level image;

Whether the size of judging the zone except that black surround in the frame gray level image is not less than default dimension threshold, if then remove described black surround zone.

6. method as claimed in claim 5 is characterized in that, removes before the described black surround zone, also comprises step: whether the gray average of judging the zone except that black surround in the frame gray level image is greater than default gray threshold;

The step of removing described black surround zone comprises: the gray average in the zone in a frame gray level image except that black surround is removed described black surround zone during greater than default gray threshold.

7. video similarity method relatively is characterized in that, may further comprise the steps:

Obtain each m frame fingerprint of two video files respectively, every frame fingerprint is: a frame gray level image is carried out piecemeal; Obtain the average gray value of each piecemeal in the frame gray level image; Obtain the fingerprint of this frame gray level image according to the correlation of the average gray value of each piecemeal;

According to the m frame fingerprint of two video files, obtain the similarity value of two video files about this m frame;

Whether judge the similarity value that obtains greater than default similarity threshold, if greater than, then determine two video files about this m frame dissmilarity, otherwise determine that two video files are similar about this m frame.

8. method as claimed in claim 7, it is characterized in that, according to the m frame fingerprint of two video files, obtain two video files and comprise: calculate the Manhattan Manhattan distance value between the m frame fingerprint of two video files about the step of the similarity value of this m frame.

9. method as claimed in claim 7 is characterized in that, the m frame of the m frame in each video file after for the continuous multiple frames of removing acute variation.

10. method as claimed in claim 9 is characterized in that, the step of removing the continuous multiple frames of acute variation comprises:

Obtain the Hamming distance between every adjacent two frames in the video file;

Determine whether to have in the continuous multiple frames above the Hamming distance between the frame of preset ratio quantity greater than default distance threshold, if determine that then this continuous multiple frames is the continuous multiple frames of acute variation and removes.

11. method as claimed in claim 7 is characterized in that, a video file in described two video files is the basic video file in the basic fingerprint base, and another video file is a video file to be checked;

Described method also comprises step:

Many groups m frame fingerprint of video file to be checked is carried out similarity relatively with many groups m frame fingerprint of basic video file respectively;

Determine content relation of inclusion between video file to be checked and the basic video file according to comparative result;

Upgrade basic fingerprint base according to the content relation of inclusion that obtains.

12. method as claimed in claim 11, it is characterized in that, many groups m frame of video file to be checked is the part of section isometric in the video file to be checked, and should many group m frames with the length of section with the length of the shortest media fragment that is no more than that the m of group frames are constituted more in the basic video file.

13. method as claimed in claim 11 is characterized in that, the content relation of inclusion comprises: fully zero lap, not exclusively overlapping, basic video file is contained in video file to be checked and basic video file comprises video file to be checked.

14. method as claimed in claim 13 is characterized in that, determines that according to comparative result the step of the content relation of inclusion between video file to be checked and the basic video file comprises:

If the many groups m frame fingerprint in the video file to be checked all in basic video file to the identical m frame fingerprint of comparative result should be arranged, determine that then basic video file comprises video file to be checked;

If the many groups m frame fingerprint in the basic video file all in video file to be checked to the identical m frame fingerprint of comparative result should be arranged, determine that then basic video file is contained in video file to be checked;

If the partial frame fingerprint comparison results is identical in many groups m frame fingerprint of video file to be checked, then determine between video file to be checked and the basic video file not exclusively overlapping;

If the many groups m frame fingerprint in the video file to be checked all not in basic video file to the identical m frame fingerprint of comparative result should be arranged, then determine complete zero lap between video file to be checked and the basic video file.

15. method as claimed in claim 13 is characterized in that, the step of upgrading basic fingerprint base according to the content relation of inclusion that obtains comprises:

When the content relation of inclusion is complete zero lap, video file to be checked is included into basic fingerprint base;

The content relation of inclusion is incomplete when overlapping, and video file to be checked is included into basic fingerprint base;

The content relation of inclusion is a basic video file when being contained in video file to be checked, replaces basic video file with video file to be checked in basic fingerprint base.

16. method as claimed in claim 11, it is characterized in that, in the time of need upgrading basic fingerprint base according to a plurality of video files to be checked, determine each video file to be checked memory location in basic fingerprint base, according to this memory location video file to be checked is sorted, video file to be checked is updated to basic fingerprint base according to the order after the ordering.

17. a device that is used to extract video finger print is characterized in that, comprising:

The piecemeal module is used for a frame gray level image of video file is carried out piecemeal;

Computing module is used for obtaining the average gray value of each piecemeal of frame gray level image;

Fingerprint module is used for obtaining according to the correlation of the average gray value of each piecemeal the fingerprint of this frame gray level image, and the fingerprint of multiframe gray level image is merged, and the fingerprint after will merging is as the fingerprint of video file.

18. device as claimed in claim 17, it is characterized in that, fingerprint module sorts to the average gray value of each piecemeal in the frame gray level image, and obtain each piecemeal corresponding sequence number, according to the order of piecemeal in gray level image with each piecemeal corresponding sequence number sequential combination, with the string symbol after the combination as the fingerprint of this frame gray level image; Perhaps, the average gray value of each piecemeal in the frame gray level image is compared in twos, will be by the comparative result of 0 or 1 expression according to sequential combination relatively, with the fingerprint of the string symbol after the combination as this frame gray level image; Perhaps, according to acquired average gray value, obtain the overall gradient direction value of each piecemeal, according to the overall gradient direction value sequential combination of the order of piecemeal in gray level image, with the fingerprint of the string symbol after the combination as this frame gray level image with each piecemeal correspondence.

19. device as claimed in claim 18 is characterized in that fingerprint module is with gray[i] and gray[j] relatively, if gray[i]＜gray[j], then the result is 1, otherwise is 0; Perhaps, if gray[i]＜gray[j], then the result is 0, otherwise is 1; Gray[] the expression array, i represents i piecemeal, j represents j piecemeal, i=0 ..., P, j=0 ..., P, P are the sum that participates in the piecemeal of synthetic fingerprint in the frame gray level image.

20. device as claimed in claim 17 is characterized in that, also comprises: filtering module is used for a frame gray level image is gone the black surround operation.

21. device as claimed in claim 20, it is characterized in that filtering module is determined the black surround zone in the frame gray level image, judge whether the size in the zone except that black surround in the frame gray level image is not less than default dimension threshold, if then remove described black surround zone.

22. device as claimed in claim 21 is characterized in that, also comprises: detection module is used for judging that whether the gray average in the zone of frame gray level image except that black surround is greater than default gray threshold;

The gray average in filtering module zone except that black surround in a frame gray level image is removed described black surround zone during greater than default gray threshold.

23. one kind is used for video similarity device relatively, it is characterized in that, comprising:

Acquisition module is used for obtaining respectively each m frame fingerprints of two video files, and every frame fingerprint is: a frame gray level image is carried out piecemeal; Obtain the average gray value of each piecemeal in the frame gray level image; Obtain the fingerprint of this frame gray level image according to the correlation of the average gray value of each piecemeal;

Similarity calculation module is used for the m frame fingerprint according to two video files, obtains the similarity value of two video files about this m frame;

Whether comparison module, the similarity value that is used to judge acquisition greater than default similarity threshold, if greater than, then determine two video files about this m frame dissmilarity, otherwise determine that two video files are similar about this m frame.

24. device as claimed in claim 23 is characterized in that, the Manhattan Manhattan distance value between the m frame fingerprint of two video files of similarity calculation module calculating.

25. device as claimed in claim 23 is characterized in that, the m frame of the m frame in each video file after for the continuous multiple frames of removing acute variation.

26. device as claimed in claim 25 is characterized in that, also comprises: filtering module is used for obtaining the video file Hamming distance between adjacent two frames whenever; Determine whether to have in the continuous multiple frames above the Hamming distance between the frame of preset ratio quantity greater than default distance threshold, if determine that then this continuous multiple frames is the continuous multiple frames of acute variation and removes.

27. device as claimed in claim 23 is characterized in that, a video file in described two video files is the basic video file in the basic fingerprint base, and another video file is a video file to be checked;

Comparison module also is used for many groups m frame fingerprint of video file to be checked is carried out similarity relatively with many groups m frame fingerprint of basic video file respectively;

Described device also comprises: relationship module is used for determining content relation of inclusion between video file to be checked and the basic video file according to comparative result;

Described device also comprises: update module is used for upgrading basic fingerprint base according to the content relation of inclusion that obtains.

28. device as claimed in claim 27, it is characterized in that, many groups m frame of video file to be checked is the part of section isometric in the video file to be checked, and should many group m frames with the length of section with the length of the shortest media fragment that is no more than that the m of group frames are constituted more in the basic video file.

29. device as claimed in claim 27 is characterized in that, the content relation of inclusion comprises: fully zero lap, not exclusively overlapping, basic video file is contained in video file to be checked and basic video file comprises video file to be checked.

30. device as claimed in claim 29, it is characterized in that, relationship module if the many groups m frame fingerprint in the video file to be checked all in basic video file to the identical m frame fingerprint of comparative result should be arranged, determine that then basic video file comprises video file to be checked; If the many groups m frame fingerprint in the basic video file all in video file to be checked to the identical m frame fingerprint of comparative result should be arranged, determine that then basic video file is contained in video file to be checked; If the partial frame fingerprint comparison results is identical in many groups m frame fingerprint of video file to be checked, then determine between video file to be checked and the basic video file not exclusively overlapping; If the many groups m frame fingerprint in the video file to be checked all not in basic video file to the identical m frame fingerprint of comparative result should be arranged, then determine complete zero lap between video file to be checked and the basic video file.

31. device as claimed in claim 29 is characterized in that, when the content relation of inclusion was complete zero lap, update module was included into basic fingerprint base with video file to be checked;

The content relation of inclusion is incomplete when overlapping, and update module is included into basic fingerprint base with video file to be checked;

The content relation of inclusion is a basic video file when being contained in video file to be checked, and update module is replaced basic video file with video file to be checked in basic fingerprint base.

32. device as claimed in claim 27, it is characterized in that, when update module need be upgraded basic fingerprint base according to a plurality of video files to be checked, determine each video file to be checked memory location in basic fingerprint base, according to this memory location video file to be checked is sorted, video file to be checked is updated to basic fingerprint base according to the order after the ordering.

Images