CN102693522A - Method for detecting region duplication and forgery of color image - Google Patents
Method for detecting region duplication and forgery of color image Download PDFInfo
- Publication number
- CN102693522A CN102693522A CN2012101297779A CN201210129777A CN102693522A CN 102693522 A CN102693522 A CN 102693522A CN 2012101297779 A CN2012101297779 A CN 2012101297779A CN 201210129777 A CN201210129777 A CN 201210129777A CN 102693522 A CN102693522 A CN 102693522A
- Authority
- CN
- China
- Prior art keywords
- image
- image block
- color image
- block
- tampered
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
本发明提出一种彩色图像区域复制篡改检测方法,首先对被检测的彩色图像去除噪声;将噪声处理后的图像进行分块,相邻的图像块之间仅有一行或一列的像素不同;分别对每个图像块的红色、绿色和蓝色三个通道的像素量化,并将每个通道依据灰度分成M段,统计出每段的像素数目作为该图像块的特征值,把获得的三个通道的所有特征值组合起来作为该图像块的特征向量D;利用每个图像块的特征向量,一图像块与其余图像块分别进行相似性匹配,获得正确匹配对图像块;建立一个与被检测图像大小相同且各个像素点的灰度级全为零的二值图,把获得的正确匹配对图像块的位置标记到二值图中,在获得的由检测结果而生成的二值图中检测出彩色图像区域复制篡改。
The present invention proposes a color image region copy tampering detection method, first removes noise from the detected color image; divides the noise-processed image into blocks, and only one row or one column of pixels is different between adjacent image blocks; Quantize the pixels of the red, green and blue channels of each image block, divide each channel into M segments according to the gray level, count the number of pixels in each segment as the feature value of the image block, and divide the obtained three All the eigenvalues of each channel are combined as the eigenvector D of the image block; using the eigenvectors of each image block, one image block is similarly matched with the rest of the image blocks to obtain the correct matching pair of image blocks; Detect the binary image with the same image size and the gray level of each pixel is all zero, mark the position of the obtained correct matching image block into the binary image, and in the binary image generated by the obtained detection result Detects color image region copy tampering.
Description
技术领域 technical field
本发明属于信息安全技术领域,特别涉及一种检测彩色图像的区域是否经过复制篡改及对复制区域和篡改区域进行定位的方法,该方法能对存在噪声、压缩、模糊等常规变换和存在旋转、缩放、翻转等几何变换情况下的区域复制篡改进行有效的检测,可用于网络环境下数字图像的真实性鉴别和取证。The invention belongs to the technical field of information security, and particularly relates to a method for detecting whether a color image area has been copied and tampered and locating the copied area and the tampered area. It can effectively detect the copying and tampering of regions under geometric transformations such as zooming and flipping, and can be used for authenticity identification and forensics of digital images in network environments.
背景技术 Background technique
通信技术、计算机和互联网的飞速发展使得人们可以实现资源的共享,并且可以很轻易地获得网上的图像、音频和视频。但是由于各种图像处理软件的普及与应用,一般用户就可以对数字图像进行随意的篡改且不会留下用人眼能看出的处理痕迹。这就使得眼见并不一定为实,如果这些虚假图片在互联网上广泛地传播或者一些伪造的图像被应用于司法取证、新闻报道、摄影比赛等场合,其所导致的误判、错误报道等情况将会造成难以估量的损失。因此,对数字图像的真实性和完整性的鉴别具有十分重要的意义。The rapid development of communication technology, computers and the Internet has enabled people to share resources and easily obtain images, audio and video on the Internet. However, due to the popularity and application of various image processing software, ordinary users can tamper with digital images at will without leaving traces of processing that can be seen by human eyes. This makes seeing not necessarily believing. If these false pictures are widely disseminated on the Internet or some forged images are used in judicial evidence collection, news reports, photography competitions and other occasions, it will lead to misjudgments, wrong reports, etc. Incalculable losses will be caused. Therefore, it is of great significance to identify the authenticity and integrity of digital images.
目前,数字图像认证方法主要分为主动认证和被动认证两大类。主动认证分为两类:一是基于脆弱数字水印的图像认证。在图像中预先嵌入脆弱数字水印,假如图像遭受了篡改,数字水印将会受损且受损部分将会暴露篡改行为。二是基于数字签名的图像认证,这种方法利用图像内容生成认证码或数字签名。基于数字水印的方法需要在图像中嵌入水印,对图像的感知性能会产生一定的影响,且水印容易受到攻击,对没有嵌入水印的图像起不到保护作用;基于数字签名的方法虽然不会对图像的感知性能造成影响,但是需要预先产生辅助信息且签名容易丢失。互联网中的图像大部分没有嵌入数字水印或者预先生成辅助信息。由于主动认证的局限性,图像被动认证的方法越来越受到人们的关注,这种方法既不需要提前对图像嵌入数字水印,也不需要事先生成辅助信息,仅仅根据图像本身信息就可以判断出是否经过篡改、合成等伪造处理。数字图像的被动认证已经成为图像处理领域的前沿研究课题,对于多媒体的信息安全、甄别虚假新闻、司法取证等方面更具有重要意义。At present, digital image authentication methods are mainly divided into two categories: active authentication and passive authentication. Active authentication is divided into two categories: one is image authentication based on fragile digital watermarks. A fragile digital watermark is pre-embedded in the image. If the image is tampered, the digital watermark will be damaged and the damaged part will expose the tampering behavior. The second is image authentication based on digital signature, which uses image content to generate authentication code or digital signature. The method based on digital watermark needs to embed the watermark in the image, which will have a certain impact on the perceptual performance of the image, and the watermark is vulnerable to attack, and cannot protect the image without the embedded watermark; although the method based on digital signature will not The perceptual performance of the image is affected, but auxiliary information needs to be generated in advance and the signature is easily lost. Most images on the Internet do not embed digital watermarks or generate auxiliary information in advance. Due to the limitations of active authentication, the method of passive image authentication has attracted more and more attention. This method does not need to embed digital watermarks in the image in advance, nor does it need to generate auxiliary information in advance. It can be judged only based on the information of the image itself. Whether it has been falsified by tampering or synthesis. Passive authentication of digital images has become a cutting-edge research topic in the field of image processing, and it is of great significance for multimedia information security, screening of fake news, and judicial evidence collection.
图像的区域复制篡改就是将一幅图像中的一个区域进行拷贝,并粘贴到该图像中的另一个区域,以达到掩盖特定目标或突出某一场景的目的。由于复制和粘贴操作都是在同一幅图像中进行的,篡改区域与整幅图像具有相同的纹理、颜色、噪声等特性,所以肉眼很难辨别图像是否经过了篡改。文献:Fridrich J,Soukal D,Lukas J.Detection of Copy-Move Forgery inDigital Images.Proceedings of Digital Forensic Research Workshop,Cleveland,2003.提出了离散余弦变换(DCT)量化系数模糊匹配的方法,将图像分块后,计算每个图像块的DCT量化系数,并对量化后DCT系数进行字典排序,以检测出图像的篡改区域。文献:Popescu A,Farid H.Exposing Digital Forgeries by Detecting Duplicated Image Regions.Dartmouth College,USA,TR2004-515,2004.提出了基于主成分分析(PCA)的检测方法,该方法利用PCA对分块后的图像特征向量进行降维,可以有效提高检测率。在发明专利:骆伟祺,黄继武.一种鲁棒的图像区域复制篡改检测方法(专利号:ZL200610036600.9)中,骆伟祺等提出了一种基于图像块相似性比较的鲁棒区域复制篡改检测方法。Image area copying and tampering is to copy an area in an image and paste it to another area in the image to achieve the purpose of covering up a specific target or highlighting a certain scene. Since the copy and paste operations are performed in the same image, the tampered area has the same texture, color, noise and other characteristics as the entire image, so it is difficult for the naked eye to distinguish whether the image has been tampered with. Literature: Fridrich J, Soukal D, Lukas J. Detection of Copy-Move Forgery in Digital Images. Proceedings of Digital Forensic Research Workshop, Cleveland, 2003. A fuzzy matching method for discrete cosine transform (DCT) quantization coefficients is proposed, and the image is divided into blocks Finally, the DCT quantization coefficients of each image block are calculated, and the quantized DCT coefficients are lexicographically sorted to detect tampered areas of the image. Literature: Popescu A, Farid H. Exposing Digital Forgeries by Detecting Duplicated Image Regions. Dartmouth College, USA, TR2004-515, 2004. A detection method based on principal component analysis (PCA) was proposed, which uses PCA to analyze the The dimensionality reduction of the image feature vector can effectively improve the detection rate. In the invention patent: Luo Weiqi, Huang Jiwu. A Robust Detection Method for Copying and Tampering of Image Areas (Patent No.: ZL200610036600.9), Luo Weiqi et al. proposed a robust area copying and tampering detection method based on image block similarity comparison .
上述方法只能对篡改区域没有经过几何变换的图像进行检测,然而篡改者为了掩盖篡改的痕迹,可能会对被复制的目标进行一定的几何变形,如对区域进行旋转、缩放、翻转等变换,使上述方法对此类区域复制篡改无能为力。The above method can only detect the image that has not undergone geometric transformation in the tampered area. However, in order to cover up the traces of tampering, the tamperer may perform certain geometric deformations on the copied target, such as rotating, scaling, flipping and other transformations on the area. Make the above method impotent to such zone copy tampering.
发明内容 Contents of the invention
为克服上述缺陷,本发明提供了一种可以检测被复制区域经过旋转、缩放、翻转、噪声或模糊等多种处理后的图像篡改检测方法,可以有效地检测互联网环境中存在区域复制篡改的图像真伪。本方法能够对疑似篡改的图像进行检测并且判断是否经过篡改,如果篡改过则标记出篡改区域和复制区域,从而确定图像的真伪。In order to overcome the above-mentioned defects, the present invention provides an image tampering detection method that can detect the copied area after being processed by rotation, scaling, flipping, noise or blurring, etc., and can effectively detect images that have been tampered with by copying areas in the Internet environment authenticity. The method can detect the suspected tampered image and judge whether it has been tampered with, and if it has been tampered with, mark the tampered area and the duplicated area, so as to determine the authenticity of the image.
为实现上述发明目的,提出一种彩色图像区域复制篡改检测方法,其特征在于,包括以下步骤:In order to achieve the above-mentioned purpose of the invention, a method for detecting color image region duplication and tampering is proposed, which is characterized in that it comprises the following steps:
步骤1):对被检测的彩色图像去除噪声;Step 1): removing noise from the detected color image;
步骤2):将所述步骤1)处理后的图像进行分块,图像块之间相互重叠,并且相邻的图像块之间仅有一行或一列的像素不同;Step 2): The image processed in step 1) is divided into blocks, the image blocks overlap each other, and only one row or one column of pixels is different between adjacent image blocks;
步骤3):分别对所述步骤2)获得的每个图像块的红色(R)、绿色(G)和蓝色(B)三个通道中灰度处在[0T]之间的像素分成M段,统计出每段的像素数值作为该图像块的特征值,把获得的三个通道的所有特征值组合起来作为该图像块的特征向量D;其中,M为正整数,每个图像块共获得3*M个特征值;Step 3): the red (R), green (G) and blue (B) three channels of the red (R), green (G) and blue (B) of each image block obtained in the step 2) are respectively divided into M segment, the pixel value of each segment is counted as the feature value of the image block, and all the feature values of the obtained three channels are combined as the feature vector D of the image block; where M is a positive integer, and each image block has a total of Obtain 3*M eigenvalues;
步骤4):利用所述步骤3)获得的每个图像块的特征向量,将每一个图像块与其余图像块分别进行相似性匹配,获得正确匹配对图像块;Step 4): using the eigenvectors of each image block obtained in the step 3), each image block is similarly matched with the remaining image blocks to obtain a correct pair of image blocks;
步骤5):建立一个与被检测图像大小相同且各个像素点的灰度值全为零的二值图,把所述步骤4)中获得的正确匹配对图像块的位置标记到二值图中,在获得的由检测结果而生成的二值图中标记出彩色图像区域复制篡改。Step 5): Set up a binary image with the same size as the detected image and the gray value of each pixel is all zero, and mark the position of the correct matching image block obtained in step 4) into the binary image , in the obtained binary image generated from the detection results, the copy tampering of the color image area is marked.
更优选地,还包括步骤6);所述步骤6):对所述步骤5)获得的由检测结果而生成的二值图通过滤波去除错误匹配,并结合形态学处理得到最终检测结果。More preferably, step 6) is also included; said step 6): the binary image generated from the detection result obtained in step 5) is filtered to remove false matches, and combined with morphological processing to obtain the final detection result.
更优选地,所述步骤1)采用高斯低通滤波法对图像进行滤波,去除噪声。More preferably, the step 1) adopts a Gaussian low-pass filtering method to filter the image to remove noise.
更优选地,所述步骤2)中将处理后的图像分解为半径为b的圆形图像块。More preferably, in the step 2), the processed image is decomposed into circular image blocks with a radius b.
更优选地,所述步骤3)的具体工作步骤为:More preferably, the specific working steps of said step 3) are:
步骤31):将每个图像块的红色(R)、绿色(G)和蓝色(B)三个通道中灰度级处在[0T]之间的像素分成M段,这M个灰度区间分别为
步骤32):统计所述步骤31)中每个通道灰度区间的像素数值,该像素数值为图像块对应通道的特征值;Step 32): Counting the pixel value of each channel grayscale interval in the step 31), the pixel value is the feature value of the corresponding channel of the image block;
步骤33):利用所述步骤32)获得的三个通道的所有特征值组合起来作为对应图像块的特征向量D,把整幅图像所有图像块中获得的特征向量组成向量组S。Step 33): Use all the eigenvalues of the three channels obtained in the step 32) to combine as the eigenvector D of the corresponding image block, and form the eigenvectors obtained in all image blocks of the entire image into a vector group S.
更优选地,所述步骤5)中对二值图进行标记时,只标记图像块中心位置及其周围上下左右的点。More preferably, when marking the binary image in step 5), only the center position of the image block and the points around it are marked up, down, left, and right.
更优选地,所述步骤6)中采用滤波方法对错误匹配的图像块进行去除,所述滤波过程包括:More preferably, in the step 6), a filtering method is used to remove the wrongly matched image blocks, and the filtering process includes:
首先,生成一个h×h的方形窗口;First, generate a square window of h×h;
然后,对所述步骤5)获得的由检测结果而生成的二值图进行滤波,对图像开始检测;Then, the binary image generated by the detection result obtained in the step 5) is filtered, and the image is detected;
当检测到窗口中白点的个数大于或等于设定的阈值时,则不做任何处理;当检测到窗口中白点的个数小于阈值时,则把窗口中的像素点的灰度级全部置零;其中,窗口移动的步长为h。When it is detected that the number of white points in the window is greater than or equal to the set threshold, no processing is done; when the number of white points in the detected window is less than the threshold, the gray level of the pixels in the window is All are set to zero; among them, the step size of window movement is h.
本发明的优点在于,能够对疑似篡改过的图像进行检测,并判断出是否经过了区域复制篡改操作,若经过了篡改则定位出篡改的位置和复制的位置,从而判定数字图像的真伪性。本方法与现有的一些方法相比,提出基于彩色图像的量化直方图的区域复制篡改检测方法,不仅能检测出直接平移的区域篡改问题,还能检测出区域经过旋转、缩放、翻转等线性几何变换的区域复制篡改问题,此外还能够处理区域经斜切、区域局部扭曲、投射变换等非线性仿射变形情况下的区域复制篡改。该方法计算简单,仅需在空域上对图像块进行匹配,提高了计算效率。The advantage of the present invention is that it can detect the suspected tampered image, and judge whether it has passed through the area copy tampering operation, and if it has been tampered with, locate the tampered position and the copied position, thereby judging the authenticity of the digital image . Compared with some existing methods, this method proposes a region copy tampering detection method based on the quantized histogram of the color image, which can not only detect the region tampering problem of direct translation, but also detect the linearity of the region after rotation, scaling, flipping, etc. The area copy tampering problem of geometric transformation can also deal with area copy tampering in the case of non-linear affine deformation such as oblique cutting, local distortion of the area, and projective transformation. The method is simple in calculation, and only needs to match the image blocks in the air domain, which improves the calculation efficiency.
附图说明 Description of drawings
图1是本发明提出的一种彩色图像区域复制篡改检测方法流程图;Fig. 1 is a flow chart of a method for detecting color image region duplication and tampering proposed by the present invention;
图2是篡改后的图像及其红、绿、蓝三色的直方图;其中,(a)是篡改图,(b)是红色分量直方图,(c)是绿色分量直方图,(d)是蓝色分量直方图;Figure 2 is the tampered image and its red, green, and blue histograms; among them, (a) is the tampered image, (b) is the histogram of the red component, (c) is the histogram of the green component, (d) is the blue component histogram;
图3是原始图像和篡改图像及检测结果图;其中,(a)是原始图像,(b)是篡改图像,(c)是滤波图,(d)是初始检测结果图,(e)是滤波后的结果图,(f)是最终结果图;Figure 3 is the original image, tampered image and detection result diagram; among them, (a) is the original image, (b) is the tampered image, (c) is the filter diagram, (d) is the initial detection result diagram, (e) is the filter After the result figure, (f) is the final result figure;
图4是篡改区域旋转20度的篡改图和检测结果图;其中,(a)是篡改区域旋转20度的篡改图,(b)是检测结果图;Fig. 4 is a tamper diagram and a detection result diagram of a tampered area rotated 20 degrees; wherein, (a) is a tampered diagram rotated 20 degrees of a tampered area, and (b) is a detection result diagram;
图5是篡改区域旋转90度的篡改图和检测结果图;其中,(a)是篡改区域旋转90度的篡改图,(b)是检测结果图;Fig. 5 is a tamper diagram and a detection result diagram of a tampered area rotated 90 degrees; wherein, (a) is a tampered diagram rotated 90 degrees of a tampered area, and (b) is a detection result diagram;
图6是篡改区域水平翻转的篡改图和检测结果图;其中,(a)是篡改区域水平翻转的篡改图,(b)是检测结果图;Fig. 6 is a tampering diagram and a detection result diagram of a horizontal flip of a tampered area; wherein, (a) is a tampered diagram of a horizontal flip of a tampered area, and (b) is a diagram of a detection result;
图7是篡改区域垂直翻转的篡改图和检测结果图;其中,(a)是篡改区域垂直翻转的篡改图,(b)是检测结果图;Fig. 7 is a tampering diagram and a detection result diagram of a vertical flip of a tampered region; wherein, (a) is a tampered diagram of a vertical flip of a tampered region, and (b) is a diagram of a detection result;
图8是篡改区域缩小为原来大小0.8倍的篡改图和检测结果图;其中,(a)是篡改区域缩小为原来大小0.8倍的篡改图,(b)是检测结果图;Fig. 8 is a tampering diagram and a detection result diagram in which the tampering area is reduced to 0.8 times the original size; wherein (a) is a tampering diagram in which the tampering area is reduced to 0.8 times the original size, and (b) is a detection result diagram;
图9是篡改区域放大为原来大小1.2的篡改图和检测结果图;其中,(a)是篡改区域放大为原来大小1.2倍的篡改图,(b)是检测结果图;Fig. 9 is a tampering diagram and a detection result diagram in which the tampering area is enlarged to an original size of 1.2; where (a) is a tampering diagram in which the tampering area is enlarged to 1.2 times the original size, and (b) is a detection result diagram;
图10是篡改图像和JPEG压缩、添加高斯白噪声和高斯模糊的篡改图;其中,(a)是篡改图像,图(b)、(c)、(d)分别是JPEG压缩的品质因子分别为70、80、90的检测结果图,图(e)和(f)是添加高斯白噪声SNR分别为15、25的检测结果图,图(g)和(h)是窗口为3×3,标准方差分别为3和5高斯模糊的检测图。Figure 10 is a tampered image with JPEG compression, Gaussian white noise and Gaussian blur added; where (a) is a tampered image, and pictures (b), (c), and (d) are the quality factors of JPEG compression respectively 70, 80, and 90 detection results. Figures (e) and (f) are the detection results of adding Gaussian white noise with SNR of 15 and 25 respectively. Figures (g) and (h) are the window size 3×3, standard Detection maps with Gaussian blur with variances of 3 and 5, respectively.
具体实施方式 Detailed ways
下面结合附图,对本发明的技术方案进行进一步详细的说明。The technical solution of the present invention will be described in further detail below in conjunction with the accompanying drawings.
本发明技术方案包括以下步骤:(1)将待检测的彩色图像进行高斯低通滤波;(2)将滤波后的图像分成固定大小的圆形图像块,且相邻的图像块之间只有一行或一列不同;(3)计算圆形图像块红色、绿色和蓝色(RGB)三个分量的量化直方图,并构成组合直方图,以此作为图像块的特征;(4)对图像块的特征进行匹配,找出最相似的图像块并将它们标记到图像上;(5)通过滤波和形态学处理去掉错误匹配,整个检测过程如流程图1所示。The technical solution of the present invention comprises the following steps: (1) performing Gaussian low-pass filtering on the color image to be detected; (2) dividing the filtered image into circular image blocks of a fixed size, and there is only one line between adjacent image blocks Or a column is different; (3) Calculate the quantized histogram of the three components of the red, green and blue (RGB) components of the circular image block, and form a combined histogram, which is used as the feature of the image block; (4) the image block Match the features, find the most similar image blocks and mark them on the image; (5) remove the wrong match through filtering and morphological processing, the whole detection process is shown in flow chart 1.
(1)对彩色图像进行滤波(1) Filter the color image
设疑似进行过区域复制篡改过的彩色图像为f,把疑似篡改过的图像f进行高斯低通滤波,提取图像的低频部分,对图像进行滤波时使用模板为3×3的窗口,标准方差取值为3,篡改图像如图3(b),滤波后的图像如图3(c);通过滤波可以有效减少干扰,为图像的后续操作做准备。Let f be the color image that is suspected to have been tampered with by region copying, and perform Gaussian low-pass filtering on the suspected tampered image f to extract the low-frequency part of the image. When filtering the image, use a window with a template of 3×3, and the standard deviation is taken as The value is 3, the tampered image is shown in Figure 3(b), and the filtered image is shown in Figure 3(c); the interference can be effectively reduced by filtering, and it is prepared for the subsequent operation of the image.
(2)对彩色图像分块(2) Block the color image
设疑似篡改过的彩色图像大小f为W×H,首先将图像f分解成半径为b=8的圆形图像块,并保证相邻的图像块之间只有一行或者一列的像素不同,由此可知图像f共可获得图像块数为B=(W-2b+1)×(H-2b+1);圆形图像快与别的形状的图像块相比有一些优势:当篡改区域发生了旋转操作时对匹配造成的影响比较小。Assuming that the size f of the suspected tampered color image is W×H, first decompose the image f into circular image blocks with a radius of b=8, and ensure that only one row or column of pixels between adjacent image blocks is different, thus It can be seen that the total number of image blocks available for image f is B=(W-2b+1)×(H-2b+1); the circular image has some advantages compared with image blocks of other shapes: when the tampered area occurs The effect on matching is relatively small when the rotation operation is performed.
(3)提取图像块的特征(3) Extract the features of the image block
首先把获得的图像块的红色R、绿色G、蓝色B三个通道的灰度级在[0 199]之间的像素分别量化,使每个通道分成5段灰度等级。图2(b)、(c)、(d)分别是图2(a)的红色、绿色、蓝色分量的直方图,图像块的直方图彩色图像的颜色范围过于庞大,在RGB颜色空间中,区域颜色直方图包含256×256×256=224个颜色。图像块R、G、B三种颜色的像素的灰度级主要分布区间为[0 199],而灰度级大于199的像素比较少。考虑到算法效率和统计的特点,将每个颜色通道量化到5个灰度等级[0 39]、[40 79]、[80 119]、[120 159]、[159 199],统计出各个区间的像素数值,并把统计出的每个区间像素数值作为图像块的特征值,把获得的三个通道的所有特征值组合起来作为图像块的特征向量。每个图像块共获得15个特征值,设整幅特征向量组为S,整幅图像总共可以获得(W-2b+1)×(H-2b+1)×15个特征值。Firstly, the pixels whose gray levels of the red R, green G, and blue B channels of the obtained image block are between [0 and 199] are respectively quantized, so that each channel is divided into 5 gray levels. Figure 2 (b), (c), and (d) are the histograms of the red, green, and blue components of Figure 2 (a), respectively. The color range of the histogram color image of the image block is too large, and in the RGB color space , the region color histogram contains 256×256×256=2 24 colors. The main distribution interval of the gray levels of the pixels of the three colors of R, G, and B in the image block is [0 199], and the pixels with gray levels greater than 199 are relatively few. Considering the algorithm efficiency and statistical characteristics, quantize each color channel to 5 gray levels [0 39], [40 79], [80 119], [120 159], [159 199], and count each interval The pixel value of each interval is counted as the feature value of the image block, and all the feature values of the obtained three channels are combined as the feature vector of the image block. A total of 15 eigenvalues are obtained for each image block, and assuming that the entire eigenvector group is S, a total of (W-2b+1)×(H-2b+1)×15 eigenvalues can be obtained for the whole image.
(4)特征匹配并定位匹配块(4) Feature matching and positioning of matching blocks
首先对特征向量组S进行降维处理,由于S是一个三维向量,不容易进行字典排序,调整特征向量组的维数使之变成二维向量并生成一个行数(W-2b+1)×(H-2b+1)列数为15的数组A。然后对数组A的(W-2b+1)×(H-2b+1)特征向量作字典排序,使数组中的特征向量从上到下依次从小到大排列并记录每行特征向量对应的图像块在原始图像中的位置。First, dimensionality reduction is performed on the feature vector group S. Since S is a three-dimensional vector, it is not easy to perform dictionary sorting. Adjust the dimension of the feature vector group to make it a two-dimensional vector and generate a number of rows (W-2b+1) ×(H-2b+1) array A with 15 columns. Then sort the (W-2b+1)×(H-2b+1) eigenvectors of array A lexicographically, so that the eigenvectors in the array are arranged from small to large in order from top to bottom and record the image corresponding to each row of eigenvectors The position of the block in the original image.
在图像块的特征向量进行相似性匹配时需要预先设定三个阈值分别是:相似度阈值Ts、距离阈值Td和区域面积阈值Ta。When performing similarity matching on the feature vectors of image blocks, three thresholds need to be preset: similarity threshold T s , distance threshold T d and area area threshold T a .
相似度阈值Ts:相似度阈值是衡量两个图像块的特征向量的相似性的参数,为了衡量特征向量Ai,s、Aj,s的相似度本发明采用欧氏距离,定义为Similarity threshold T s : the similarity threshold is a parameter to measure the similarity of the feature vectors of two image blocks. In order to measure the similarity of feature vectors A i, s , A j, s, the present invention adopts Euclidean distance, which is defined as
首先设定特征比较范围,由于对所有的特征向量进行了字典排序,使得相似图像块的特征向量在数组A中的位置离得比较近,设定特征比较范围为20行。从第一行数据开始分别与其下面20行特征向量进行比较并计算其欧氏距离;图像块越相似,计算出的欧氏距离越小。在进行试验过程中,我们随机选取了100幅图像做区域复制篡改,篡改区域大小为80×80,然后对篡改图像做了不同的后处理操作,包括添加高斯白噪声、高斯模糊、旋转、缩放和JPEG压缩。对于每幅经过篡改后的图像,当相似度阈值Ts取5.5时检测效果比较好并且能有效够检测出篡改区域和复制区域。Firstly, the feature comparison range is set. Since all feature vectors are sorted lexicographically, the position of feature vectors of similar image blocks in array A is relatively close, and the feature comparison range is set to 20 rows. From the first row of data, compare it with the 20 rows of feature vectors below and calculate its Euclidean distance; the more similar the image blocks, the smaller the calculated Euclidean distance. During the experiment, we randomly selected 100 images for area copy and tampering, the size of the tampered area was 80×80, and then performed different post-processing operations on the tampered images, including adding Gaussian white noise, Gaussian blur, rotation, and scaling and JPEG compression. For each tampered image, when the similarity threshold T s is 5.5, the detection effect is better and the tampered area and the duplicated area can be detected effectively.
距离阈值Td:篡改图像的图像块与其相邻的图像块只有一行或者一列的像素不同,在图像块与图像块的特征向量进行相似匹配过程中,如果所比较的两个图像块位置很近,即使不是正确的匹配块,也会得到较高的相似度,导致错误匹配。为此,提出了距离阈值Td。在实验过程中选取的匹配块的半径为8,为了减少相邻的匹配块造成的影响,当匹配块对的中心坐标之间的距离大于或者等于距离阈值时进行比较,在本方法中我们取距离阈值Td为16。Distance threshold T d : The image block of the tampered image has only one row or one column of pixels different from its adjacent image block. , even if it is not the correct matching block, it will get a high similarity, resulting in a false match. To this end, a distance threshold T d is proposed. The radius of the matching block selected during the experiment is 8. In order to reduce the influence caused by adjacent matching blocks, when the distance between the center coordinates of the matching block pair is greater than or equal to the distance threshold, the comparison is made. In this method, we take The distance threshold T d is 16.
区域面积阈值Ta:在自然图像中,除了大片平坦区域(如蓝天、草地、公路、白云等),存在相似(包括颜色、形状、纹理等)大面积区域的可能性不是很大。如果检测到一幅彩色图像中存在大面积的相似区域,那么很可能此图像是被区域复制篡改过的。在文献:骆伟祺,黄继武,丘国平.鲁棒的区域复制图像篡改检测技术.计算机学报,2007,30(11):1998-2007.中作者指出大面积区域不小于原始图像尺寸的0.85%,在检测过程中,检测到的图像块与原始的篡改块在形状上往往存在一定的差异,因此可定义区域面积阈值Ta为:Area area threshold T a : In natural images, except for large flat areas (such as blue sky, grassland, road, white clouds, etc.), the possibility of similar large areas (including color, shape, texture, etc.) is not very high. If large similar regions are detected in a color image, it is likely that the image has been tampered with by region replication. In the literature: Luo Weiqi, Huang Jiwu, Qiu Guoping. Robust region copy image tampering detection technology. Journal of Computer Science, 2007, 30(11): 1998-2007. In the author pointed out that the large area area is not less than 0.85% of the original image size, In the detection process, the detected image block and the original tampered block often have certain differences in shape, so the area threshold T a can be defined as:
Ta≥W×H×0.85%×η (3)T a ≥ W×H×0.85%×η (3)
其中,η为受损系数。Among them, η is the damage coefficient.
当检测出的匹配块满足设定的阈值Ts和Td时,则认为是正确匹配的一对图像块。在进行比较的20对匹配块中有可能存在多个满足条件的匹配块,假如在比较范围内有多对满足条件的匹配块只选获得的相似度阈值最小的那个图像块作为正确匹配块对并对其定位,如果没有满足条件的则不做任何处理。When the detected matching block satisfies the set thresholds T s and T d , it is considered to be a pair of correctly matched image blocks. In the 20 pairs of matching blocks that are compared, there may be multiple matching blocks that meet the conditions. If there are multiple pairs of matching blocks that meet the conditions within the comparison range, only the image block with the smallest similarity threshold is selected as the correct matching block pair. And locate it, if there is no one that meets the conditions, no processing will be done.
(5)对图像进行滤波及形态学处理(5) Filter and morphologically process the image
建立一个与原始图像大小相同且各个像素点的灰度级全为零的二值图,把步骤(4)中获得的正确匹配对的图像块的位置标记到二值图中,标记时只标记图像块的中心位置及其上下左右共计5个位置上的像素,直接标记的结果如图3(d)。在获得的由检测结果而生成的二值图中有许多孤立的白色像素点,这些点实际上是错误匹配的图像块,因此应当尽可能地去掉这些点。本发明中我们建立一个8×8的滤波窗口并采用此窗口对整幅图像进行滤波。首先从图像的左上角开始检测,当检测到窗口中白点的个数大于或等于设定的阈值20时不做任何处理,若小于阈值20时则把窗口中的像素点的灰度值全部置零。通过此方法不但可以有效滤除那些孤立的错误匹配块,而且可以保护那些正确匹配的图像块不受干扰,提高图像区域复制篡改检测的效果。Create a binary image with the same size as the original image and the gray level of each pixel is all zero, mark the position of the correct matching pair of image blocks obtained in step (4) into the binary image, and only mark The center position of the image block and the pixels at five positions including top, bottom, left and right, the result of direct marking is shown in Figure 3(d). There are many isolated white pixels in the binary image generated by the detection results, these points are actually wrongly matched image blocks, so these points should be removed as much as possible. In the present invention, we establish an 8×8 filtering window and use this window to filter the entire image. First, start detection from the upper left corner of the image. When the number of white points detected in the window is greater than or equal to the set threshold of 20, no processing will be done. If it is less than the threshold of 20, the gray value of the pixels in the window will be all Zero. This method can not only effectively filter out those isolated wrong matching blocks, but also protect those correctly matched image blocks from interference, and improve the effect of image region copy tampering detection.
二值图3(e)经过滤波后如果还有一些较大的错误匹配区域,可以对图像进行数学形态学的开操作去除这些为匹配区域;如果没有这种错误匹配区域这可以省略此步骤。因为对图像进行标记时只是标记了图像块的中心位置及其上下左右共计5个位置上的像素,使得检测出的篡改区域和复制区域比原始的篡改区域和复制区域小,所以对图3(e)进行数学形态学上的闭操作处理。If the binary image 3(e) still has some larger mismatched regions after filtering, the mathematical morphology opening operation can be performed on the image to remove these matched regions; if there are no such mismatched regions, this step can be omitted. Because when the image is marked, only the center position of the image block and the pixels on the five positions in total, up, down, left, and right are marked, so that the detected tampered area and copied area are smaller than the original tampered area and copied area, so for Figure 3 ( e) Perform mathematical morphology closing operation processing.
为了展示本发明的效果,我们给出了一些篡改图像和相应的检测结果。To demonstrate the effectiveness of our invention, we present some tampered images and corresponding detection results.
如图3所示,(a)是原始图像,(b)是篡改后的图像,(c)是篡改图像滤波后的图像,(d)是初始检测结果,图(e)是利用本发明中提出的滤波器进行滤波的结果,(f)是对图(e)进行闭操作后得到的最终检测结果。由图(f)可知,在没有任何几何变换和攻击的情况下,检测效果很好。图4、图5、图6、图7是篡改区域经过旋转20度、90度和水平翻转、垂直翻转的篡改图像和检测结果。由检测结果可知篡改区域即使经过了这些几何变换之后检测效果仍然非常好。图8(a)和图9(a)是篡改区域分别经过缩小为原来大小的0.8倍和放大为原来大小1.2倍后的图像,由其对应的检测结果图(b)可知图像即使经过了这种缩放处理检测效果仍然很好。图10是篡改图像经过有损JPEG压缩、添加高斯白噪声和高斯模糊后的图像相应的检测结果。图(b)、(c)、(d)是经过质量因子分别为70、80、90的有损JPEG压缩后的检测结果,图(e)和(f)是篡改图像添加SNR为15db和25db高斯白噪声处理后的结果,图(g)和(h)是篡改图像经过模板为3×3,标准方差分别为3和5的高斯模糊处理后的检测结果。由以上检测结果可知,本发明对常规的各种信号处理攻击和区域的几何变形都具有很好的鲁棒性。As shown in Figure 3, (a) is the original image, (b) is the tampered image, (c) is the filtered image of the tampered image, (d) is the initial detection result, and figure (e) is the The result of filtering by the proposed filter, (f) is the final detection result obtained after closing the graph (e). As can be seen from Figure (f), the detection works well without any geometric transformation and attack. Figure 4, Figure 5, Figure 6, and Figure 7 are the tampered images and detection results after the tampered area has been rotated by 20 degrees, 90 degrees, horizontally flipped, and vertically flipped. It can be seen from the detection results that the detection effect of the tampered area is still very good even after these geometric transformations. Figure 8(a) and Figure 9(a) are images after the tampered area has been reduced to 0.8 times the original size and enlarged to 1.2 times the original size, respectively. This zoom processing detection effect is still very good. Figure 10 shows the corresponding detection results of tampered images after lossy JPEG compression, adding Gaussian white noise and Gaussian blur. Figures (b), (c), and (d) are the detection results after lossy JPEG compression with quality factors of 70, 80, and 90 respectively. Figures (e) and (f) are tampered images with SNRs of 15db and 25db The results after Gaussian white noise processing, Figures (g) and (h) are the detection results of tampered images after Gaussian blur processing with a template of 3×3 and standard deviations of 3 and 5 respectively. From the above test results, it can be seen that the present invention has good robustness against various conventional signal processing attacks and geometric deformation of the region.
性能测试和实验分析Performance testing and experimental analysis
为了更有效的验证本发明的鲁棒性,我们将本发明的方法在两个数据库中进行测试,这里定义了正确匹配率Fr和错误匹配率Fw,其公式如下,In order to verify the robustness of the present invention more effectively, we test the method of the present invention in two databases, where the correct matching rate F r and the wrong matching rate F w are defined, and the formula is as follows,
其中,C1和C2分别为经过篡改的图像的复制区域大小和检测出的复制区域大小,M1和M2分别为经过篡改的图像篡改区域大小和检测出的篡改区域的大小。Among them, C 1 and C 2 are the size of the copied area of the tampered image and the size of the detected copy area, respectively, M 1 and M 2 are the size of the tampered area of the tampered image and the size of the detected tampered area, respectively.
为了评估检测效果,我们对在两个数据库中的100幅图像进行了测试,这两个数据库分别是G.Schaefer and M.Stich,UCID-An Uncompressed Colour Image Database,Technical Report,School of Computing and Mathematics,Nottingham Trent University,U.K.,2003.数据库和我们自建的一个数据库,由于数据库UCID比较大,我们只随机选取了其中的100图像。在测试试验中,我们选取大小为80×80的图像块从一块区域粘贴到同一图像的另一区域,对篡改后的图像进行高斯噪声、有损JPEG压缩、高斯模糊攻击。由表1和表2获得的实验数据可知,在进行测试的这些攻击的情况下,正确检测率都在0.9以上,并且错误率都比较低。In order to evaluate the detection effect, we tested 100 images in two databases: G.Schaefer and M.Stich, UCID-An Uncompressed Color Image Database, Technical Report, School of Computing and Mathematics , Nottingham Trent University, U.K., 2003. The database and a database we built ourselves, because the database UCID is relatively large, we only randomly selected 100 images. In the test experiment, we selected an image block with a size of 80×80 to paste from one area to another area of the same image, and performed Gaussian noise, lossy JPEG compression, and Gaussian blur attacks on the tampered image. From the experimental data obtained in Table 1 and Table 2, it can be seen that in the case of these attacks tested, the correct detection rate is above 0.9, and the error rate is relatively low.
表1 UCID数据库检测结果Table 1 UCID database detection results
表2 自建数据库检测结果Table 2 Test results of self-built database
Claims (7)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012101297779A CN102693522A (en) | 2012-04-28 | 2012-04-28 | Method for detecting region duplication and forgery of color image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012101297779A CN102693522A (en) | 2012-04-28 | 2012-04-28 | Method for detecting region duplication and forgery of color image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102693522A true CN102693522A (en) | 2012-09-26 |
Family
ID=46858929
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012101297779A Pending CN102693522A (en) | 2012-04-28 | 2012-04-28 | Method for detecting region duplication and forgery of color image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102693522A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103034995A (en) * | 2012-11-18 | 2013-04-10 | 百年金海安防科技有限公司 | Splicing image detecting algorithm based on statistical properties of Benford rule |
| CN105118034A (en) * | 2015-08-19 | 2015-12-02 | 西安电子科技大学 | Color image morphological filtering method based on full map space |
| CN105825504A (en) * | 2016-03-11 | 2016-08-03 | 中山大学 | MSCR-region-feature-based image duplicating detection method |
| CN106056122A (en) * | 2016-05-26 | 2016-10-26 | 中山大学 | KAZE feature point-based image region copying and pasting tampering detection method |
| CN108881663A (en) * | 2018-06-20 | 2018-11-23 | 暨南大学 | A kind of image zone duplicating detection method for supporting privacy protection function |
| CN111008955A (en) * | 2019-11-06 | 2020-04-14 | 重庆邮电大学 | Multi-scale image block matching rapid copying pasting tampering detection method |
| CN111415450A (en) * | 2019-01-08 | 2020-07-14 | 深圳怡化电脑股份有限公司 | Bill altering detection method and detection device and computer storage medium |
| CN111768368A (en) * | 2020-05-26 | 2020-10-13 | 西安理工大学 | An Image Region Copy Tampering Detection Method Based on Maximum Stable Extremum Region |
| CN112990231A (en) * | 2021-03-12 | 2021-06-18 | 河北经贸大学 | Method, device and terminal for removing image mismatching pairs |
| CN113721859A (en) * | 2021-09-08 | 2021-11-30 | 郑州轻工业大学 | Image repeating data deleting method based on artificial intelligence |
| CN116523852A (en) * | 2023-04-13 | 2023-08-01 | 成都飞机工业(集团)有限责任公司 | Foreign matter detection method of carbon fiber composite material based on feature matching |
| CN117576425A (en) * | 2024-01-17 | 2024-02-20 | 南京掌控网络科技有限公司 | Method and system for detecting scurrying image |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1703722A (en) * | 2002-10-09 | 2005-11-30 | 皇家飞利浦电子股份有限公司 | Localisation of image tampering |
| CN1900970A (en) * | 2006-07-20 | 2007-01-24 | 中山大学 | Image zone duplicating and altering detecting method of robust |
| CN101853071A (en) * | 2010-05-13 | 2010-10-06 | 重庆大学 | Vision-based gesture recognition method and system |
-
2012
- 2012-04-28 CN CN2012101297779A patent/CN102693522A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1703722A (en) * | 2002-10-09 | 2005-11-30 | 皇家飞利浦电子股份有限公司 | Localisation of image tampering |
| CN1900970A (en) * | 2006-07-20 | 2007-01-24 | 中山大学 | Image zone duplicating and altering detecting method of robust |
| CN101853071A (en) * | 2010-05-13 | 2010-10-06 | 重庆大学 | Vision-based gesture recognition method and system |
Non-Patent Citations (4)
| Title |
|---|
| XUNYU PAN,SIWEI LYU: "Region Duplication Detection Using Image Feature Matching", 《IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY》 * |
| 王俊文: "数字图像内容篡改盲取证研究", 《中国博士学位论文全文数据库》 * |
| 马晓普,刘妍: "基于统计特征的鲁棒图像拷贝检测技术", 《计算机应用》 * |
| 骆伟祺 等: "鲁棒的区域复制图像篡改检测技术", 《计算机学报》 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103034995A (en) * | 2012-11-18 | 2013-04-10 | 百年金海安防科技有限公司 | Splicing image detecting algorithm based on statistical properties of Benford rule |
| CN105118034A (en) * | 2015-08-19 | 2015-12-02 | 西安电子科技大学 | Color image morphological filtering method based on full map space |
| CN105118034B (en) * | 2015-08-19 | 2017-10-24 | 西安电子科技大学 | Coloured image morphologic filtering method based on full figure space |
| CN105825504A (en) * | 2016-03-11 | 2016-08-03 | 中山大学 | MSCR-region-feature-based image duplicating detection method |
| CN105825504B (en) * | 2016-03-11 | 2018-06-26 | 中山大学 | A kind of image copy detection method based on MSCR provincial characteristics |
| CN106056122A (en) * | 2016-05-26 | 2016-10-26 | 中山大学 | KAZE feature point-based image region copying and pasting tampering detection method |
| CN106056122B (en) * | 2016-05-26 | 2019-05-17 | 中山大学 | A kind of image zone duplicating stickup altering detecting method based on KAZE characteristic point |
| CN108881663A (en) * | 2018-06-20 | 2018-11-23 | 暨南大学 | A kind of image zone duplicating detection method for supporting privacy protection function |
| CN108881663B (en) * | 2018-06-20 | 2019-12-24 | 暨南大学 | A method for image region duplication detection supporting privacy protection |
| CN111415450A (en) * | 2019-01-08 | 2020-07-14 | 深圳怡化电脑股份有限公司 | Bill altering detection method and detection device and computer storage medium |
| CN111008955A (en) * | 2019-11-06 | 2020-04-14 | 重庆邮电大学 | Multi-scale image block matching rapid copying pasting tampering detection method |
| CN111768368A (en) * | 2020-05-26 | 2020-10-13 | 西安理工大学 | An Image Region Copy Tampering Detection Method Based on Maximum Stable Extremum Region |
| CN111768368B (en) * | 2020-05-26 | 2022-05-03 | 西安理工大学 | Image area copying and tampering detection method based on maximum stable extremal area |
| CN112990231A (en) * | 2021-03-12 | 2021-06-18 | 河北经贸大学 | Method, device and terminal for removing image mismatching pairs |
| CN112990231B (en) * | 2021-03-12 | 2022-08-26 | 河北经贸大学 | Method, device and terminal for removing image mismatching pairs |
| CN113721859A (en) * | 2021-09-08 | 2021-11-30 | 郑州轻工业大学 | Image repeating data deleting method based on artificial intelligence |
| CN113721859B (en) * | 2021-09-08 | 2023-07-21 | 欧亚高科数字技术有限公司 | An artificial intelligence-based image deduplication method |
| CN116523852A (en) * | 2023-04-13 | 2023-08-01 | 成都飞机工业(集团)有限责任公司 | Foreign matter detection method of carbon fiber composite material based on feature matching |
| CN116523852B (en) * | 2023-04-13 | 2024-12-13 | 成都飞机工业(集团)有限责任公司 | A foreign body detection method for carbon fiber composite materials based on feature matching |
| CN117576425A (en) * | 2024-01-17 | 2024-02-20 | 南京掌控网络科技有限公司 | Method and system for detecting scurrying image |
| CN117576425B (en) * | 2024-01-17 | 2024-04-16 | 南京掌控网络科技有限公司 | Method and system for detecting scurrying image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102693522A (en) | Method for detecting region duplication and forgery of color image | |
| Li et al. | Localization of diffusion-based inpainting in digital images | |
| Kang et al. | Robust median filtering forensics using an autoregressive model | |
| Popescu et al. | Exposing digital forgeries by detecting duplicated image regions | |
| Cao et al. | Contrast enhancement-based forensics in digital images | |
| CN100440255C (en) | A Robust Method for Image Region Duplication and Tampering Detection | |
| Niu et al. | Robust median filtering detection based on local difference descriptor | |
| Singh et al. | Fast and efficient region duplication detection in digital images using sub-blocking method | |
| CN104504669A (en) | Median filtering detection method based on local binary pattern | |
| CN103839083B (en) | The detection method that a kind of JPEG picture quantization table is distorted | |
| CN102957915B (en) | Double JPEG (Joint Photographic Experts Group) compressed image-targeted tamper detection and tamper locating method | |
| CN102892048B (en) | Video watermark anti-counterfeiting method capable of resisting geometric attacks | |
| Ulutas et al. | A new copy move forgery detection method resistant to object removal with uniform background forgery | |
| CN101916442A (en) | A robust localization method for tampered images using GLCM features | |
| Kang et al. | Color Image Steganalysis Based on Residuals of Channel Differences. | |
| CN102968803A (en) | Tamper detection and tamper positioning method directing at CFA (Color Filter Array) interpolation image | |
| CN119379524B (en) | Image counterfeiting active defense method based on multiple watermark fusion and cross-domain learning | |
| CN109523452B (en) | Color image steganography detection method based on channel differential residual error | |
| CN102521821A (en) | Automatic identification and tampered area positioning method in digital image | |
| Dixit et al. | Copy-move forgery detection exploiting statistical image features | |
| CN107346528A (en) | A kind of distorted image detection method based on the reversible fragile watermark of digraph | |
| CN102592151B (en) | Blind detection method for median filter in digital image | |
| CN117633725A (en) | Image digital watermark copyright protection system based on characteristics unchanged | |
| CN103956173B (en) | A kind of digital watermark information coding and discrimination method based on mathematical morphology and hologram | |
| Li et al. | Detectability of the image operation order: upsampling and mean filtering |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20120926 |