WO2001011864A2

WO2001011864A2 - System and method for processing optically scanned documents

Info

Publication number: WO2001011864A2
Application number: PCT/US2000/022036
Authority: WO
Inventors: Sergio A. Fernandez; Marcelo Funes-Gallanzi
Original assignee: Asci, Inc.
Priority date: 1999-08-11
Filing date: 2000-08-11
Publication date: 2001-02-15
Also published as: AU6901900A; WO2001011864A3

Abstract

A scanning apparatus and method for reducing the spatial distortion, compression, lack of illumination, excessive glare, and out of focus caused when scanning or copying an image of a bound document that is not placed completely flat on the scanner platen. The distortion coefficients of the bound document image are found by analyzing the colors and illumination of each pixel of the image. By applying these distortion coefficients to several different formulas, it is possible to find the coefficient values representative of the spatial distortion correction, compression correction, illumination correction, glare correction and out of focus correction, respectively. These coefficient values are all at once applied to another formula which transforms the distorted document image, giving as a result an image that has substantially less spatial distortion, is less compressed, has a more even illumination, less glare and is more focused than an image that is not placed completely flat on the scanner platen and has been scanned or photocopied without applying these correction factors. In one embodiment, once the correction factors have been applied to the image, it is coupled to an output device for printing or saving.

Description

SYSTEM AND METHOD FOR PROCESSING OPTICALLY SCANNED DOCUMENTS

Background of the Invention Field of the Invention

This invention relates to the field of document scanning and/or reproduction of documents with the aid of electro-optical devices. More specifically, the present invention addresses the problems of scanning an image of a document into a photocopier, a mopier, a digital copier, a line studio camera or scanner when the document cannot be laid completely flat. Description of the Related Technology

The copying or scanning of non-planar originals, such as bound books can be problematic because it is difficult to adequately copy or scan the information adjacent to the binding area. The difficulties of reproduction of the binding area result from the portion of the book adjacent to the binding area being lifted away from the original image plane, which is usually defined by a planar transparent plate. The problem is actually related to a series of factors that cause the quality of reproduction to decrease in relation to increasing distance of the original object from the image plane. This lack of clarity and/or distortion in the copy results from several factors, including: a) the spatial distortion, which is most evident where the paper image consists of straight lines of text where these will appear to rise near the bottom of the page and drop near the bottom of the page; b) the compression of the image along the scan length; c) illumination becomes less than optimal in the spine area; d) in some cases a glare appears on the right side page where the page starts to lift away from the scanner platen; and e) the projected image becomes increasingly out of focus as the original is scanned in the central binding area or spine.

Various approaches have been taken to minimize or eliminate these factors. A common "brute force" approach is to press the book against the surface of the platen to lessen or eliminate displacement from the platen surface of the binding. However, portions of the pages adjacent the binding area still remain a short distance above the platen surface. Consequently, information contained in these portions is out of focus on the output sheet or data structure. Moreover, attempts to press the binding area closer to the platen can result in breaking the binding of the book, especially older books, which have brittle bindings. Furthermore, in the case of some manuscripts that cannot be opened enough to be photocopied, line studio cameras are required. Line studio cameras do not require the document to be on a flat glass surface and are used when the documents may be damaged by forcing these into a flat bed scanner. When using line studio cameras, the book is disposed facing upwards and a camera digitizes the page using an overhead scanning system. Line studio cameras, are cameras similar to photographic cameras which are well known in the art, and they move a CCD in the plane where regular cameras form the image exposed to the film. The paper image is illuminated and the CCD senses the reflected light. Digital cameras may also be used to capture images. Digital cameras use a CCD to capture all the light from the paper image at once. As in scanners, sensing of the reflected light can be captured in a snapshot sensing all the light from the paper at once as with a digital camera, or sensing the light from a point or a line at the time as with a line studio camera An example of this type of system is disclosed in a U S Patent No. 5,835,241 to Saund. As it can be seen, images of bound documents captured in line studio cameras or digital cameras, present the same problems, as do images of bound documents captured with flat bed scanners, since in both of these apparatus, the focusing point is placed on a flat platen and not on the document image itself, which is bent.

One solution that has been tried has been the use of photocopiers where the platen is located near the edge of the photocopier and thereby avoiding spatial distortion, compression, illumination, glare and out of focus problems. These photocopiers allow a book page to be placed flat on the platen while the facing pages of the book drape down the side of the photocopier While such photocopiers have limited applications, they suffer from several drawbacks. One drawback is that facing pages cannot be simultaneously sensed Therefore, two facing pages cannot be copied without a user moving the book and since the documents need to be arranged facing the same way, handling these may become a problem Keeping the book open is problematic and time consuming, and it is hard to make illumination even in all of the documents Another solution is to modify the platen to have a sloping edge portion so that the bound part of the book is placed in a corner portion so that the entire surface of the page being copied is in intimate contact with the platen surface. An example of such a system is disclosed in U.S. Patent No. 3,775,008. This system has several disadvantages. The magnification range is limited because of the restriction on scanner movement in the sloping corner edge. Also, copying capabilities are limited by the inability to employ a "split scan" scanning system, which allows both pages of a book to be placed on a platen and scanned without repositioning

This problem has been partially addressed by height measuring hardware combined with adjustable optics. Such a system is shown in U.S. Patent No. 4,980,720. In the system disclosed therein, a height sensor is used to measure the distance (the "z" direction) from a book page to a copy machine platen as a function of the distance from the book's spine (the "x" direction), the axis parallel to the spine being known as the "y" direction. This height function is then used to adjust mirrors, lens, and light sources so that the image reflects a constant illumination, stays in focus, and is scanned at a constant speed relative to the page While such a system may correct for the spatial distortion of a page being copied, such a system cannot correct for the spatial distortion of the page once an image of the page has been taken. Instead, the correction must be performed as the image is captured. Since the system assumes height is not a function of the "y" direction (i.e., height is constant along a line parallel to the spine from the top to the bottom of the book) the system cannot properly correct for the spatial distortion of a page in a book when the top and bottom of the book are not at the same height This is often the case when the user closes a copy machine cover onto the book, pressing down more heavily on the end of the book closest to the hinge of the copy machine cover.

Another development is a system employing a height sensor on the scanner for determining the height of the original object plane above the platen The height information is used to control the position of a scanning mirror to adjust visual image position, to adjust variable focusing, to control scan speed, and to control illumination. Although this system overcomes many of the shortcomings mentioned above, it requires the need for precise positioning of mirrors and precise controllability of focusing lenses and carriage speeds These factors add to complexity of the equipment and increase its cost. A more refined version of this method is disclosed in U.S. Patent No. 5,276,530. The disadvantage of this system is that it requires the use of a scanning system which includes a detector for sensing the distance between image plane and object surface.

Document restoration methods have been extensively applied to documents which are known to have been corrupted or have low resolution such as aerial photographs, old maps, archaeological images, et cetera. A method of non linear equalization of degraded document images, due to repeated reproduction or fax transmission, is described in U.S. Patent No. 5,745,597 to Agazzi, et al. to improve the performance of optical character recognition (OCR) systems. Methods for distortion correction of scanned images based on analyzing text lines have been reported but these do not generalize to other objects such as lines, image edges and the like, nor consider applying these methods to the process of document reproduction. The main disadvantage of these methods is that they only correct lines of text, and they do not consider the distortion caused by changing object illumination or focus distortion. An example of these methods is disclosed in U.S Patent No. 5,497,236 to Wolff, et al. In Wolff, a document distorted by the curvature of a page of text away from a platen is converted to a digital image One problem with this system is that only spatial distortion is corrected and, although both horizontal and vertical lines are used to calculate distortion separately, line correction data is not combined to derive a whole surface distortion estimate.

Another scanning system, disclosed in U.S. Patent No. 5,835,241, projects a light strip in the bound document to measure the non-planar deformation of each page While this system fixes spatial distortion or warping of the page, it requires adding a light beam to the scanner, scanning the document twice which makes this system slower, and it only corrects spatial distortion

Existing document restoration methods suffer from several major disadvantages First, these methods have not been applied to restoring image quality in real time as documents are being copied or as a processing stage in optical scanners. Second, in the main, they require hardware changes to be applicable to the scanning process and be able to achieve a speed of at least a few sheets per minute Third, they do not fully restore document image quality based on the document image itself.

As it is seen from the previous examples, there is a need for an apparatus and method for addressing the above problems that occur when scanning or copying an image of a document into a photocopier, a mopier, a digital copier, a line studio camera or a scanner when the document cannot be laid completely flat Summary of the Invention

A method and apparatus is provided for reducing various problems that are caused when scanning a bound document that is not placed completely flat on a scanner platen.

One aspect of the invention includes a method of correcting for distortion in a captured image of an object, the method comprising identifying one or more background sections of a captured image, generating a distortion coefficient based upon the background sections, and removing distortion from the captured image by use of the distortion coefficient

Yet another aspect of the invention includes a method of removing distortion from a captured image of an object, the method comprising capturing an image of an object, and automatically removing, in response to the capturing, compression distortion from the captured image

Yet another aspect of the invention includes a method of removing distortion from a captured image of a an object, the method comprising capturing an image of an object and automatically removing, in response to the capturing, illumination distortion from the at least a portion of the captured image

Yet another aspect of the invention includes a method of removing distortion from a captured image, the method comprising capturing an image of an object, and automatically removing, in response to the capturing, glare from at least a portion of the captured image.

Yet another aspect of the invention includes a method of removing distortion from a captured image of an object, the method comprising capturing an image of an object, and automatically removing, in response to the capturing, focus distortion from at least a portion of the captured image Yet another aspect of the invention includes a method of removing distortion from a captured image, the method comprising capturing an image of an object, and automatically removing, in response to the capturing, spatial distortion from the captured image.

Yet another aspect of the invention includes a method of removing distortion from a captured image, the method comprising capturing an image of an object, automatically removing, in response to the capturing, compression distortion from at least a portion of the captured image, automatically removing, in response to the capturing, illumination distortion from at least a portion of captured image, automatically removing, in response to the capturing, glare from at least a portion of captured image, automatically removing, in response to the capturing, focus distortion from at least a portion of the captured image, and automatically removing, in response to the capturing, spatial distortion from at least a portion of the captured image Yet another aspect of the invention includes a distortion correction apparatus, comprising a pixel color and illumination analyzer adapted to analyze the color and illumination of each pixel of a distorted image, a distortion coefficient generator, in data communication with the pixel color and illumination analyzer, adapted to generate values representative of a distortion coefficient of the distorted image, and a transformer, in data communication with the distortion coefficient generator, adapted to transform the distorted image by use of the distortion coefficient of the distorted image.

Brief Description of the Drawings Figure 1 is a block diagram of a system for scanning bound documents. Figure 2 is a flowchart illustrating a process of converting a captured image into a corrected image. Figure 3 is a flowchart illustrating in further detail the process of Figure 2 for removing distortion from a captured image. Figure 4 is an illustration of an exemplary 3 D histogram of an image that is captured by the system of Figure 1.

Figure 5 illustrates a document image of with an enlargement of a pixel and the color numbers of the pixel that is part of a captured image Figure 6A, 6B, and 6C respectively illustrate a grey pixel, a pixel that approximates grey, and a non grey pixel.

Figure 7 is an illustration of a captured image wherein a selected pixel in each column of the captured image has been identified as being proximate to a spine of a book.

Figure 8 is an illustration of an exemplary image that has been captured by the system of Figure 1 , wherein background areas and foreground areas are identified.

Figure 9 is a representational block diagram illustrating a background line and a foreground line of a captured image.

Figure 10 is a histogram illustrating the gray intensities for the background, foreground separately, as well as for the background and foreground combined Figure 1 1 is a diagram illustrating a numeric circle that is used for determining a break point in an image that is captured by the system of Figure 1.

Figure 12 illustrates a graphical user interface for viewing an image that is captured by the system of Figure 1.

Figure 13 illustrates an image having spatial distortion. Figure 14 illustrates the image of Figure 13 subsequent to the removal of the spatial distortion.

Figure 15 is a partial side elevational view of the book of Figure 1.

Figures 16A illustrates an image that has compression distortion.

Figure 16B illustrates the image of Figure 16A without compression distortion.

Figure 17A illustrates an image that has illumination distortion Figure 17B illustrates the image of Figure 17A after illumination distortion has been removed.

Figure 18A illustrates an image that has glare

Figure 18B illustrates the image of Figure 18A wherein the glare has been removed.

Figure 19 is a partial side elevational diagram of the system shown in Figure 1 , wherein the cause of glare is shown. Figure 20A illustrates an image that has focus distortion.

Figure 20B illustrates the image of Figure 20A wherein the focus distortion has been removed.

Figures 21 A and 21 B are diagrams illustrating how out of focus distortion occurs.

Figure 22A illustrates a pixel that is focused

Figure 22B illustrates a pixel that is out of focus Figure 23 illustrates a pivoting plane that is used to remove focus distortion, such as is show in Figure 20A and 22B.

Figure 24 illustrates the characteristics of the pixels population being analyzed by the pivoting area. Detailed Description of the Certain Embodiments Figure 1 illustrates an embodiment of a scanning system 1 , which is adapted to operate in a conventional document scan mode as well as a book copying mode The system 1 is placed beneath a transparent image platen 2, which is usually of glass or other rigid transparent material. A document 3 is shown placed on the platen 2, folded at the spine 4. The left break point 5 and the right break point 6 represents the points where the document 3 starts to move away from the platen 2. The scanning system 1 includes a radiation source such as a lamp 7, mounted within a reflector 8. Radiant energy or light from the lamp 7 is reflected upwardly through the platen 2 and irradiates the document 3 that is placed on the platen 2. The scanning system 1 also includes a scanning mirror 9 positioned to receive radiation reflected from the document 3. The lamp 7, the reflector 8, and mirror 9 may be mounted on a common housing 10.

The reflected optical image data received on scanning mirror 9 is directed via a suitable mirror assembly 1 1 to an optical LRNS focusing assembly 12 that preferably is of a simple, fixed focus type. The optical focusing assembly 12 focuses the optical image information into a beam 13, which is directed onto a suitable photo sensor 14, which converts the optical image data of the beam 13 into electrical image data. Known types of photo-sensors can be used, such as a Charge-Coupled-Device (CCD). The electrical image data signal from CCD 14 can be converted by an analog to digital converter, if required, though many CCD systems now possess a digital output, before being sent to a computer for processing.

The lamp 7, the reflector 8, the scanning mirror 9, the mirror assembly 1 1, the focusing assembly 12, and the CCD 14 can be commonly mounted in a scanning carriage 15. The carriage 15 is adapted to move from left to right beneath the platen 2 at pre-set or controllable scanning speeds. The scanning carriage 15 undergoes a pre-scan excursion to the left prior to initiation of the scan exposure cycle with initial acceleration (and vibration damping) taking place in the pre scan zone shown as PS The start of scan position is identified as point S„. The end-of-scan position is identified as S,

Figure 2 is a flowchart showing the steps for acquiring a less distorted image data with scanning system 1. Depending on the embodiment, certain steps may be removed and others may be added. Before starting, the scanning system 1 is set up as shown in Figure 1. Starting at a state 100, an object, such as a book, is selected for copying using the system 1. Moving to a state 104, the image is captured using the scanning system 1. At the state 104, upon a copy or scan function being initiated, the scanning carriage 15 and the mirror assembly 1 1 are moved through from a start of scan position So to an end of scan position S1 The document 3 is thus illuminated by lamp 7 and the reflected image is received by the mirror 9 that reflects a latent image of the document surface to the mirror assembly 1 1 and then to the lens assembly 12 The lens assembly 12 focuses the latent image data into beam 13 that is directed onto a suitable photo-sensor 14 Continuing to a state 108, the scanning system 1 converts the optical image data into electronic image data. Moving to a state 1 12, once the electronic image data is in the photo sensor 14, distortion in the image data is corrected. In one embodiment of the invention, the distortion in the captured image is removed by a computer 17 that is connected to the system 100 In another embodiment of the invention, the system 100 incorporates in software or hardware the processes that are used to correct the image distortion The computer 17 and the system 1 may be connected to a display device that is used to verify the existence and type of distortion that is identified by the computer 17 and/or the system 1 The process of correcting image distortion is described below in further detail with respect to Figure 3 Next, at a state 1 16, the corrected image may be then printed saved, or sent to a computer for further processing. Figure 3 is a flowchart illustrating a process of correcting certain distortion in a captured image. Depending on the embodiment, certain steps may be removed and others may be added.

Starting at a state 204, a 3 D histogram is created by taking the 2 D image electrical data and analyzing the luminescence of each pixel 20 The 3 D histogram is a graphic representation of background areas Values of a pixel are in the range of 255 (bright) to 0 (dark) The 3 D histogram uses only the pixels in range 255 to 1 15 because pixels in letters are typically in the range 115 0 and cause noise As is seen from a visual inspection of Figure 4, those pixels that are lightest (in gray) are aligned with the platen 2, and those pixels that are darker and are further from the platen 2. Figure 5 illustrates the pixel values for an exemplary pixel With this information, it is possible to create a 2D histogram to find the distortion coefficient of the document (described below).

Continuing to a state 204, the spine of the book is identified. At state 204, the darkest color pixel 27 (Figure 7) in each row is identified In one embodiment, the darkest pixel is that pixel having lowest color average number. Once the darkest pixel is identified, its location is recorded This process is repeated for each line 26, until the darkest color pixel 27 has been located and the column 28 that corresponds to where the darkest color pixel 27 is located has been detected The darkest pixels tend to fall within a selected column which is assumed to identify the spine of the book. This process can also be carried out by using the horizontal lines of the image data when the image is captured of a book that is placed vertically or by a combination of all of these methods At state 107, once the spine of the book 4 is located, each one of the pages is independently analyzed with respect to the remainder of the steps. Figure 7 illustrates the darkest pixel 27 for each column and the tendency of the darkest pixel to fall on or near a selected column.

Moving to a state 208, candidate backgrounds are identified The background area 24 is identified as the area where there is a lack of an important density of ink The foreground area 25 is identified as the area of the document 3 where most objects such as photographs, letters, lines etc are found Figures 8 and 9 each illustrate how background areas and foreground areas are differentiated. In one embodiment, the lines that are candidates to be background areas are identified by analyzing each of the pixel colors of the captured image. Background lines 29 are those lines where the sum of gray pixels 21 and almost gray pixels 22 is larger than the sum of the non gray pixels 23. Foreground lines 30 are those where the sum of the non gray pixels is larger than the sum of gray pixels 21 and almost gray pixels 22. A gray pixel 21 is a pixel that has the same amount of blue, green and red. An almost grey pixel 22 is a pixel whose color values of blue, green and red, do not vary much one from the other. A non-gray pixel 23 is a pixel whose color values of blue, green and red vary greatly one from the other. Figures 6A, 6B, and 6C illustrate examples of the different types of pixels. For example, if a line that crosses a picture is analyzed, it has more non-gray pixels 23 than the sum of gray pixels 21 and almost gray pixels 22. In this case, the line is classified as foreground line 30. If a line that passes between two lines of text is analyzed, the sum of gray pixels 21 and almost gray pixels 22 is typically greater than the sum of the non-gray pixels. In this case, the line would be classified as a background line 29.

Continuing to a state 212, a column histogram illustrating the grey average of each of the background lines is prepared. Figure 10 illustrates an exemplary histogram. A line 31 represents the points of the gray average of each column in the image of the document 3, including the background area 24 and the foreground area 25. The line 31 is usually very distorted because it contains a lot of noise. A line 32 represents the points of the gray average of each column of the background areas 24. As it is seen, the line 32 does not contain as much noise as line 31 and it provides a more uniform line that contains less variances. Next, at a state 216, the line 32 is used for finding the break point 35 (Figure 11 ). In one embodiment, the break point 35 is located by constructing a numeric circle 38. Taking the first N data of the straight line 32 , an average W is calculated. The number W is used to construct an interval (W-M, W + M) that determines the width (2^*M) and the center (W) of the circle. In one embodiment of the invention, the value of M is 7. Once limits of the numeric circle 38 are established, the next data in straight line (N + 1 ) is analyzed. If coordinates of subsequent data fall within the limits, the point is designated as accepted data 39 and its recorded as being the last point accepted. The data that does not fall within the circle is designated as rejected data 40. Every time the numeric circle 38 accepts a new data, it is taken as one of the last N data. At the moment in which the histogram starts to curve itself, there is a point where the data will no longer be accepted into the numeric circle 38 and it is considered as the break point 35. The break point 35 is detected as being the point where the histogram begins to curve, and it is presumably where the document 3 starts to separate from the platen 2.

Once the break point 35 is identified, an approximation by minimum squares is calculated for analyzing the behavior of the gray average of each column before and after the break point 35. Line 33 illustrates the approximation by minimum squares of the points that represent the gray average of each column of the background areas 24. Proceeding to a state 220, the approximation of the gray levels after the break point 35 yields a curve coefficient. Next, at a state 224, the curve coefficient is used to determine a distortion coefficient for the captured image. Once the curve coefficient has been identified, at state 1 12 it is possible to determine the distortion coefficient of the image of the document 3. The coefficient of the document image distortion can be determined by applying Equation 1, which is set forth below. (1 ) Distortion coefficient = Coefficient ^* arc tan (F ^* Coefficient 2), wherein-

Coefficient = 7 1 for left pages or 7.5 right pages; Coeffιcιent2 = 180/ ; and F = the coefficient of the curved line

Continuing to a state 228, a graphical interface (Figure 12) may be displayed to the user to allow the user to correct the distortion coefficient for the capturing image. The graphical interface comprises several handles and guide lines that allow the user to correct the calculated distortion values of the captured image. The graphical interface comprises: (i) at least two handles 43 for identifying the straight portion of the document 3; (n) at least two handles 45 and 46 for each page of the book for designating an arc that is representative of the warping in the captured image due to the binding; and (m) handles 47 and 48 for designating the spine of the book Furthermore, handles 49 may be used to designate a section of a page where glare is appearing The user has the option of manipulating the handles to correct the displayed data It should be noted that the distortion correction techniques described more fully below may be performed automatically or in conjunction with changes indicated by the use via use of the handles 42-49. Continuing to a state 232, spatial distortion in the captured image is corrected Referring to Figure 13, spatial distortion 51 is caused by the binding 4 of the book, which makes the upper areas 50 of the document 3 tend to bend towards the lower part 53 of the document 3 and it makes the lower part of the document 53 tend to bend upwards to the towards the upper areas 50 of the document 3. The upper area 50 lines of the document have a more marked bend than the horizontal lines that are closer to the middle 52 of the image of the document 3. Equations 2-5 may be used to determine the correct line of pixel that falls within a row having deformation.

(2) A0 = Coefficients ^* Column¹

(3) A1 - Coeffιcιent4 * Column²

(4) A2= Coefficients ^* Column³

(5) Correct line of pixel = [Line (A0 + A1 + A2 * Slope)]. { 1 - [{A0+ A1 + A2 ^* Slope)/ Ydim ^* Coefficient 6]}, wherein

Coefficients = .01; Coeffιcιent4 =.000025; Coefficients = .000005; Col = column or vertical line where the pixel is being relocated;

Slope = Distortion coefficient of the document (calculated by Equation 1 ),

Ydim = total height of the document; and

Line - line of the pixel that is being straightened.

Figure 13 illustrates an image having spatial distortion 51 and Figure 14 illustrates an image after the spatial distortion 51 has been corrected When Equation 5 is applied, it indicates a new column for the selected pixel filling within the deformation area It should be understood by a skilled technologist that, depending on the embodiment, other mathematical equations or other coefficients may be used Continuing to a state 236, the compression distortion is removed from the captured image. For example,

Figure 15 illustrates a side view of a document image when it is laying flat on the platen 2. The compression of the image is caused due to the fact that the document 3 separates from the platen 2, it registers an image of a smaller longitudinal length 55 than it would if it were laying flat This causes the image of the document 3 to appear compressed as the curvature of the document 3 increases, whereas the flat length 56 of the document 3 is longer.

Using the distortion coefficient of the document, it is possible to approximate the pendant of the curve of the document 3 and with this information it is possible to determine the extent of the compression of the document 3.

When the document 3 is on the platen 2, the pendant of the document is zero, but as its starts to separate from the platen 2, the pendant begins to increase. This increase is interrelated with the compression of the document by using Equation 6.

(6) Pendant of the column = Coefficient? * arc tan [(A0+ Coefficients ^* A1 +

Coeffιcιent9 ^* A2 ^* Slope)/ Div] wherein:

Coefficient? = 2.3, Coefficients = 2;

Coeffιcιent9 = 2,

Slope = the distortion coefficient (calculated by Equation 1 ); and Div = Col - StartL (left page) or StartR - Col (right page).

The compression can then be fixed using Equation 7: (7) Compressιon(Col) = [Compressιon(Col - 1 ) - (ιnt)Compressιon(Col -

1 )] + (abs) [ 1 / cos(Pendant of the column)], wherein, Compression (0) = 0. The formula calculates the number of pixel units that the pixel 20 should be moved to correct the compression distortion; whether it is some spaces to the left or some spaces to the right. Figure 16A illustrates a document that scanned using known copiers. Figure 16B illustrates the document after it has been decompressed by the foregoing process. It is noted that changes are more notorious in columns closer to the binding of the book.

Moving to a state 240, illumination distortion is removed from the captured image. Figure 17A illustrates an image without any illumination correction and Figure 17B illustrates an image after the illumination factors have been applied. The illumination distortion removal process accounts for the fact that illumination is lost in a linear and hyperbolic manner; that is, in the most upper area 50 and lower area 53 of the document 3, illumination is the worst, and in the middle part 52, it is not lost as much The process to increase illumination has two factors: the linear increase and the parabolic increase The linear increase is the same for the entire column that is being fixed. That is, the amount of increase is the same for a pixel 20 that is on the upper part 50 of the document, than it is for the pixel 20 that is in the middle part 52 or at the lower part 53 of the document. The second increase is made to fix the change of illumination in the page. This is due to the fact that the illumination in an image tends to be greater in the center part 52 of the document than on the upper part 50 and lower part 53 of the document 3. This can be corrected by using a parabolic method to manipulate the increase of illumination in a manner that it will be null in the middle 52 of the document and greatest in the extreme upper part 50 and lower part 53 Equations 8 and 9, set forth below, is used to correct the illumination

(8) Alt - [{2 " line/ Ydim) -1], wherein: Ydim = total height of the document.

(9) Final color of the pixel = (ιnt)(Color of the pixel + [Pendant of the column ^*

^* Pendant of the column ^* (Alt² + Alt⁸) - Constant3] / Constant 4), wherein. Color of the pixel = red, green or blue value of pixel,

Coefficientl O = 26 (a left page) and 10 (a right page) Constant3 = 2; and Constant4 = 100.

For a selected color value of a pixel, Equations 8 and 9 determine the parabolic and the linear increase that should be added to the pixel Equations 8 and 0 are applied separately for each of the three colors of a selected pixel whose illumination has been distorted.

Next, at a state 244, any glare 62 of the captured image is corrected. Glare may be produced due to the bouncing of light when the page is being scanned. It is noted that inner part 63 of the area with glare is brighter than lateral areas 64 of the glare. Figure 18 illustrates an image with a glare and an image without a glare.

Figure 19 illustrates how the light is emitted form lamp 7, reflected by reflector 8 and directed towards the left page 15 and reflected on to the right page 16. Since the image on the right page 16 is receiving light from both the lamp 7 and the left page 15, the right page 16 receives an excessive amount of illumination that causes many of the pixels 20 of this area to lose their color and give an effect of a glare 62 One way of reducing this excessive amount of light or glare 62 in the image of the document 3 is to reconstruct the color of the original pixel 20. Equation 10, set forth below, may be used to eliminate glare.

(10) Final color of the pixel = Color of the pixel ^* {1 Darkness ^* sin²

[(Col - Left handle) ^* Coefficient - 1 / (Right handle - Left handle)]}, wherein:

Left handle = column where the glare begins; Right handle = the column where glare ends; Darkness =

1 if 220 < color of the pixel < = 255, .2 if 190 < color of the pixel < = 220,

.3 if 150 < color of the pixel < = 190; and 4 if color of the pixel < = 150; and Coefficients 3.14.

By applying Equation 10, the area of the light glare 62 is darkened and the pixels acquire a color that is very similar to the original color. Since the colors that are darker lose more color, Equation 10 adds more color to these pixels. Continuing to a state 248, focus distortion is removed from the captured image. Focus distortion is caused when the document 3 moves away from the platen 2, as it goes towards the binding 4 of the book. Figure 20A illustrates an image that is out of focus in the area where the document is not aligned to the platen 2 and Figure 20B illustrates an image to which the out of focus correction factors have been applied Figures 21A and 21 B illustrates how the focusing assembly 12 is focussed on the platen 2 with a specific distance between the platen 2 and the document 3. If document image is not flat, as is the case with most bound documents, distance between the platen and the document 3 varies and produces a defocused image A method for correcting this out of focus is described below.

Figure 22B illustrates how focus distortion can distort a pixel, such as is shown in Figure 22A The focus distortion causes a mixture of information amongst pixels and visual clearness to be lost. One manner of fixing this problem is to detect the pixels that have lost their color and to replace these, and to subtract color from those pixels that have gained color.

Figure 23 illustrates a pivot plane 64 that analyzes and fixes the out of focus portions of the captured image. The out of focus portion of the image typically begins where the value of the pendant begins to increase, i.e., the document image is separating from the platen 2.

To fix this focus distortion, a pivot plane 64 is created that is composed of three specific areas: the pivot area 63, and sweeping areas 66 and 67 The pivot pixel 65 is the pixel 20 that is in the center and which has become out of focus. The pivot area 63 is the area surrounding the pivot pixel 65 In one embodiment, the pivot area 63 is a 5

^* 5 group of pixels having the pivot pixel 65 centrally located within the pivot area 63 It is to be appreciated that other pixel areas may be used. In one embodiment, the sweeping areas 66 and 67 respectively include two areas of 3

^* 3 pixels each, which are each located on the sides of the pivot plane 64, and are each coterminous with three of the pixels of the pivot area 63. The function of these sweeping areas 66 and 67 is to pick up noise that may affect the pivot pixel 65. The sweeping areas 66 and 67 are used to find the contrast between the pixels of the sweeping area and the central pixel of the sweeping area If there is a contrast between them, the central pixel of the sweeping area takes the average of the pixels of the sweeping area being analyzed The left sweeping area 66 is used when the left page of the document is analyzed, and the right sweeping area 67 is used when the right page is analyzed.

When pivot pixel 65 is being analyzed, it is determined whether the pivot pixel 65 is classified as being either important, irrelevant, or as borrowing color from a neighbor. This is done by the analyzing the contrast between pixels of the pivot area 63 and the pivot pixel 65. If a pixel is part of a letter, i.e., an important pixel, focus distortion can cause the surrounding pixels to take the color of it If there is an important contrast between the pivot pixel and its neighbors, the pivot pixel 65 is designated as an important pixel

In one embodiment, to identify the contrast between the pivot pixel 65 and its surrounding pixels, the value of each of the pixels in the pivot area 63 excluding the pivot pixel is added. This sum is then subtracted from the pivot pixel multiplied by 24 (5 ^* 5 1 ) to thereby provide a contrast value For a pivot pixel to be considered important- (i) each of the pixels in the area excluding the pivot pixel should be greater than the average of the pixels in the area minus the contrast value; (n) each of the pixels in the pivot area 64 should be smaller than the average of the pixels in the area plus the contrast value; and (in) the contrast value should be greater than 130 If the pixel is not important, it is either an irrelevant pixel or a pixel that borrows color from a neighbor. If the pivot is less than 220 and the contrast value is less than 130, the pivot pixel 65 is a pixel that borrows color from its neighbors

At the state 248, to fix the focus, color is added to important pixels and subtracted from neighboring pixels that took color from an important pixel The amount of color that is transferred depends on the pendent of the column. Equation 11, set forth below is may be used to calculate a new color of an important pixel. (1 1 ) Color = Color - ((7/5) ^* (2.4 ^* arctanfm^{4 *} 180/3.14))² + Color ^* m²), wherein: m = the pendant of the column where the pixel is placed, and

Color = each one of the red, green, or blue color values.

In one embodiment, irrelevant pixels are not modified. Color is removed from pixels that are classified as borrowing color from their neighbor Equation 12, set forth below, describes how to calculate a new color value of a pixel that borrows color from its neighbor.

(12) Color = Color ^* (1 + m²/55), wherein: m = the pendant of the column where the pixel is placed; and

Color = each one of the red, green, and blue colors. in one embodiment, when the pivot pixel is classified and the changes in color value of the pivot pixel are determined, the new values are not immediately set but are stored until the pixel area 64 has moved past the respective pixel. It is noted that after two lines of the captured image have been fixed, such as is shown in Figure 24, that the correction of these lines could affect results of correcting the focus distortion of an uncorrected pixel. However, when out of focus appears, pixels typically borrow information from neighbors and the correction of such pixels in the pixel window does not affect the cleanup of the focus distortion. Furthermore, as noted above, the original pixels of those pixels positioned to the left or right of an unfocused pixel are used in during the process of removing focus distortion, instead of their corrected values

The new and less distorted image data is then either saved on the computer or other device For it can be coupled to an output apparatus that will print an image of a less distorted image, than that which was originally presented. Although in this example, the image is captured using an optical sensor in a photocopying machine, a scanner, a digital copier, a camera, a digital camera, a line studio camera, a mopier, and other apparatuses may also be used to implement said process.

The process may be used with a particular machine as such, or it may be used as an intermediary step. The foregoing system and method may also be integrated with an OCR processing device Furthermore, the foregoing system and method could also be implemented as software routines that are written in a programming language such as C, C + +, Fortran, or Pascal.

The process may also be used to apply the correction factors to an image that has been previously acquired either by a photocopy or another device. Processing can also be adjusted to account for the textual content of the document image being vertically oriented. The process may be used both for color and for black and white text. The process may also be implemented by fragmenting the document image into blocks and processing the blocks sequentially or in a parallel manner

The above description is illustrative and not restrictive Many variations of the invention will become apparent to those of skilled in the art upon review of this disclosure, and which are intended to be encompassed by the following claims. Although not all of the possible implementations are discussed herein, the scope of the invention should be determined not only with the reference to the above description, but instead should be determined with reference to the appended claims along with their full scope of the equivalents.

Claims

WHAT IS CLAIMED IS:

1. A method of correcting for distortion in a captured image of an object, the method comprising: identifying one or more background sections of a captured image; generating a distortion coefficient based upon the background sections; and removing distortion from the captured image by use of the distortion coefficient.

2. The method of Claim 1 , additionally comprising capturing an image of an object.

3. The method of Claim 1 , additionally comprising: generating a histogram of the captured image; analyzing, line by line, the density of the colors of the pixels of the distorted image; and determining if there is a significant color difference amongst the pixels of the distorted image, wherein if the difference exists it constitutes foreground, if it is not found, then it constitutes the background of the distorted image.

4. The method of Claim 3, wherein the generating the histogram of the captured image comprises: selecting the identified background lines; finding the average of the pixel colors of each column of the background lines of the distorted image; and plotting the data in relation to the position of the column and the average of the column.

5 The method of Claim 1, wherein the generating the distortion coefficient comprises: analyzing the average of the columns of the background lines thereby determining where the straight line of the histogram terminates and starts to turn into a curved line; constructing a numeric circle to determine where a break point in the captured image is located, the break point identifying where the object separates from an image platen; determining the coefficient of the portion of the histogram that defines a curve; and applying the coefficient of the curve of the histogram to find the distortion coefficient of the object.

6. The method of Claim 1, wherein removing distortion additionally includes verifying that the coefficient values effectively correspond to the distortion.

7. The method of Claim 1 , additionally comprising removing spatial distortion from the captured image by use of the distortion coefficient.

8. The method of Claim 1 , additionally comprising removing compression distortion from the captured image by use of the distortion coefficient.

9. The method of Claim 1 , additionally comprising removing illumination distortion from at least a portion of the captured image by use of the distortion coefficient.

10. The method of Claim 1 , additionally comprising removing focus distortion from at least a portion of the captured image.

1 1. The method of Claim 1, additionally comprising removing glare from at least a portion of the image.

12. A method of removing distortion from a captured image of an object, the method comprising: capturing an image of an object; and automatically removing, in response to the capturing, compression distortion from the captured image.

13. The method of Claim 12, additionally comprising identifying compression distortion in the captured image.

14. A method of removing distortion from a captured image of a an object, the method comprising: capturing an image of an object; and automatically removing, in response to the capturing, illumination distortion from the at least a portion of the captured image.

15. The method of Claim 14, additionally comprising identifying illumination distortion in the captured image.

16. A method of removing distortion from a captured image, the method comprising- capturing an image of an object; and automatically removing, in response to the capturing, glare from at least a portion of the captured image.

17. The method of Claim 16, additionally comprising identifying glare in the captured image.

18. A method of removing distortion from a captured image of an object, the method comprising: capturing an image of an object; and automatically removing, in response to the capturing, focus distortion from at least a portion of the captured image.

19. The method of Claim 18, additionally comprising identifying focus distortion in the captured image.

20. A method of removing distortion from a captured image, the method comprising: capturing an image of an object; and automatically removing, in response to the capturing, spatial distortion from the captured image.

21. The method of Claim 20, additionally comprising identifying focus distortion in the captured image.

22. A method of removing distortion from a captured image, the method comprising: capturing an image of an object; automatically removing, in response to the capturing, compression distortion from at least a portion of the captured image; automatically removing, in response to the capturing, illumination distortion from at least a portion of captured image; automatically removing, in response to the capturing, glare from at least a portion of captured image; automatica y removing, in response to t e capturing, ocus istortion rom at least a portion of the captured image; and automatically removing, in response to the capturing, spatial distortion from at least a portion of the captured image

23. A distortion correction apparatus, comprising: a pixel color and illumination analyzer adapted to analyze the color and illumination of each pixel of a distorted image; a distortion coefficient generator, in data communication with the pixel color and illumination analyzer, adapted to generate values representative of a distortion coefficient of the distorted image; and a transformer, in data communication with the distortion coefficient generator, adapted to transform the distorted image by use of the distortion coefficient of the distorted image.

24. The apparatus of Claim 23, additionally comprising: an input device receiving the distorted image and formatting the distorted image for further processing by the distortion correction apparatus; and an output device, in data communication with the transformer, adapted to for output a corrected image.

25. The apparatus of Claim 23, wherein the pixel color and illumination analyzer analyzes and corrects the pixel color and illumination of the distorted image.

26. The apparatus of Claim 23, further comprising a histogram generator adapted to model the background lines of the distorted image.

27. The apparatus of Claim 23, further comprising a spine locator adapted to determine a darkest color pixel of at least one line in the distorted image thereby identifying where a spine of the book is located.

28. The apparatus of Claim 28, additionally comprising a background area identifier adapted to determine which pixels are background portions of the distorted image.

29 The apparatus of Claim 28, wherein the background area identifier segments the captured image into one or more segments, and wherein the background area identifier determines whether each of the segments include background data.

30. The apparatus of Claim 23, wherein the distortion coefficient generator further comprises: a background area column analyzer adapted to determine where a straight portion of the histogram terminates and starts to turn into a curved line, wherein the curved line represents the part of a scanned object that is not in proximate contact with a scanner platen; and a software code adapted to determine the coefficients of the curved line.

31. The apparatus of Claim 23, wherein the distorted image is captured using an optical sensor in a photocopying machine.

32. The apparatus of Claim 23, wherein the transformer includes a verifier adapted to verify that calculated coefficient values are correct.

33. The apparatus of Claim 23, wherein the transformer removes spatial distortion.

34. The apparatus of Claim 23, wherein the transformer removes compression distortion from at least a portion of the captured image

35. The apparatus of Claim 23, wherein the transformer removes glare from at least a portion of the captured image.

36. The apparatus of Claim 23, wherein the transformer removes illumination distortion from at least a portion of the captured image.

37. The apparatus of Claim 23, wherein the transformer corrects the focus of at least a portion of the captured image.