WO1998052119A1 - Method and system for image retrieval - Google Patents
Method and system for image retrieval Download PDFInfo
- Publication number
- WO1998052119A1 WO1998052119A1 PCT/US1997/009256 US9709256W WO9852119A1 WO 1998052119 A1 WO1998052119 A1 WO 1998052119A1 US 9709256 W US9709256 W US 9709256W WO 9852119 A1 WO9852119 A1 WO 9852119A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- similarity
- database
- image
- regions
- images
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 19
- 238000010586 diagram Methods 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5854—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using shape and object relationship
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
Definitions
- This invention relates to computerized image retrieval and, more specifically, to retrieval based on image database querying .
- the images can be included in an image database.
- querying based on image content can be combined with querying based on spatial location.
- queries can be directed to region feature similarity and region spatial location similarity in combination. If desired, the relative spatial arrangement of regions can also be taken into account.
- Fig. 1 is an example of an image with regions for inclusion in a database.
- Fig. 2 is a tabular display of a representation of the regions of Fig. 1.
- Fig. 3 is an example of an image with regions for database querying.
- Fig. 4 is a tabular display of a representation of the regions of Fig. 3.
- Fig. 5 is a flow diagram of database query processing for discriminating based on region feature and region absolute spatial location.
- Fig. 6 is a flow diagram of database query processing for discriminating based on the relative location of regions.
- the following description is primarily in terms of method steps for execution by a suitable processor under program control .
- the program may originate as software, or, for greater efficiency, it may be embodied at least in part in dedicated firmware or hardware.
- a prototype system embodying features as described has been formulated in the JAVA language .
- the system can operate on suitable hardware such as a SUN Workstation, a Silicon Graphics Workstation, or a PC with a Pentium processor, for example.
- an image database to be queried has tabular form, with each record or table entry representing a region of an image.
- a record includes an image identifier, a region identifier, a region attribute and, for geometric characterization, the x- and y- coordinates of the centroid of the region, the width and height of the region, and the area of the region.
- the table may be generated by manual keyboard entry based on visual inspection of images. Alternatively, if a suitable pattern recognition system is available, table generation may be automated. To illustrate database entries, the image 10 shown in Fig. 1 and having been given the identifier "T" can be represented by the table entries shown in Fig. 2.
- region 100 (t 0 , stretching across the bottom of the image, below a broken line) ; region 101 (t 1( bounded by a rectangle drawn with broken lines) ; region 102 (t 2 , bounded by a rectangle drawn with chain-dotted lines) , region 103 (t 3 , bounded by a rectangle drawn with broken lines) ; and region 104 (t 4 , stretching across the top of the image, above a broken line) .
- the x,y-coordinates, the width w, and the height h of each region are given in percent of the respective maximal values.
- the values x, y, w and h define a "bounding rectangle" for each region, so that the area of a region is less than or equal to w times h. As illustrated, regions may overlap, and their union need not cover the image.
- the attribute f may simply represent color, for example, with color being represented by known means, e.g., by a color histogram or by color sets.
- Other simple attributes which may be used include texture and shape, and such attributes may be combined into more complex attributes.
- a search query is expressed correspondingly.
- a query region table may be formed as shown in Fig. 4.
- Fig. 5 illustrates query processing for finding database entries based on the query. The general aim is to find images that contain arrangements of regions similar to those in the query.
- the database regions are searched for a feature match (step 51) and a spatial match (step 52) .
- a suitable metric for comparing the spatial information such as x, y, h, w and area of the query region with the corresponding information for the database regions .
- Suitable metrics include Euclidean distance and other Minkowski distances, and quadratic metrics whose definition involves a square matrix which expresses the relative similarity between the components of a vector.
- a metric can also include weights which may be different for each of the geometric parameters .
- thresholds are applied to the computed feature and spatial distances. Thus, if a distance exceeds the threshold, the database region is not included for further consideration. Instead of, or in addition to using separate thresholds for spatial and feature similarity as shown in Fig. 5, thresholding can be applied also to the combined region distance or score, i.e. before saving a region match in step 53. Distances may be combined by simple addition, or by suitable weighting followed by addition, for example.
- "k-loop" feature similarity processing analogous to step 51 and spatial similarity processing analogous to step 52 may be carried out in parallel instead of pair-wise sequentially as illustrated in Fig. 5.
- Parallel processing then yields two sets of regions, namely (i) those which meet feature similarity regardless of spatial similarity, and (ii) those which meet spatial similarity regardless of feature similarity.
- a "join" operation will be required. After joining, a final thresholding operation can be performed.
- multiple processors may also be used for parallel processing within steps 52 and 53.
- Image matches are obtained as a result of the "join" operation in step 54, producing all those database images which meet each one of the region requirements of the query.
- a query may result in an image being saved in step 54 more than once, namely for different combinations of its regions which satisfy the query. Such multiplicity may be helpful to a user of the system; otherwise, duplicates can be deleted by a simple one-pass search of the saved images . If the relative spatial location or arrangement of regions is not important to a user, the computation may terminate at this point (jS) , though preferably after the saved images are sorted by score.
- a process can be used as illustrated by Fig. 6, using so-called 2-D strings.
- Generation of 2-D strings at this point, i.e. after similarity processing, may be termed "query-time 2-D string generation" .
- a 2-D string For a query image, a 2-D string includes the x- coordinates of the centroids of the regions, arranged as an increasing sequence, followed by the y-coordinates of the centroids, also arranged as an increasing sequence.
- the coordinates of those regions are used which were matched against the query image regions .
- the 2-D string of the query image is formed in step 61, and, in step 62, this string is matched against the 2-D strings from each of the saved images.
- step 63 only in case of a match, the database image is saved, so that only those images are ultimately sorted and produced in step 64 which have a 2-D string which matches the 2-D string of the query image.
- 2-D strings can be produced after rotation of the coordinate system, e.g. by 45°.
- Such 2-D strings are defined analogously, using coordinates x' and y' of the centroids in the rotated coordinate system.
Landscapes
- Engineering & Computer Science (AREA)
- Library & Information Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP54918598A JP2001525959A (en) | 1997-05-16 | 1997-05-16 | Image search method and system |
PCT/US1997/009256 WO1998052119A1 (en) | 1997-05-16 | 1997-05-16 | Method and system for image retrieval |
CA002290445A CA2290445A1 (en) | 1997-05-16 | 1997-05-16 | Method and system for image retrieval |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/US1997/009256 WO1998052119A1 (en) | 1997-05-16 | 1997-05-16 | Method and system for image retrieval |
Publications (1)
Publication Number | Publication Date |
---|---|
WO1998052119A1 true WO1998052119A1 (en) | 1998-11-19 |
Family
ID=22260985
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US1997/009256 WO1998052119A1 (en) | 1997-05-16 | 1997-05-16 | Method and system for image retrieval |
Country Status (3)
Country | Link |
---|---|
JP (1) | JP2001525959A (en) |
CA (1) | CA2290445A1 (en) |
WO (1) | WO1998052119A1 (en) |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1999067695A3 (en) * | 1998-06-22 | 2000-04-13 | Koninkl Philips Electronics Nv | Image retrieval system |
WO1999067696A3 (en) * | 1998-06-23 | 2000-04-13 | Koninkl Philips Electronics Nv | A scalable solution for image retrieval |
FR2801992A1 (en) * | 1999-12-03 | 2001-06-08 | Canon Kk | Image searching method from database in computer, involves comparing two data items indicating region of interest in new image and representing visual content characteristic of example image |
EP1184796A1 (en) * | 2000-08-29 | 2002-03-06 | Sudimage | Method of associative navigation in a multimedia database |
EP1195062A1 (en) * | 1999-05-17 | 2002-04-10 | Samsung Electronics Co., Ltd. | Color image processing method |
FR2815741A1 (en) * | 2000-10-24 | 2002-04-26 | Canon Kk | Indexing method for digital images and method for searching for digital images contained in a database based on the index which is for predefined regions of interest within the digital images |
GB2381615A (en) * | 2001-08-23 | 2003-05-07 | Hewlett Packard Co | System and method for facilitating image retrieval |
US6782395B2 (en) | 1999-12-03 | 2004-08-24 | Canon Kabushiki Kaisha | Method and devices for indexing and seeking digital images taking into account the definition of regions of interest |
US6801661B1 (en) | 2001-02-15 | 2004-10-05 | Eastman Kodak Company | Method and system for archival and retrieval of images based on the shape properties of identified segments |
GB2417801A (en) * | 2004-09-07 | 2006-03-08 | Pepperdog Ltd | Image processing apparatus |
US7245762B2 (en) | 1999-05-17 | 2007-07-17 | Samsung Electronics Co., Ltd. | Color image processing method |
AT514355A1 (en) * | 2013-05-17 | 2014-12-15 | Ait Austrian Inst Technology | Used to select digital images from an image database |
CN106815272A (en) * | 2015-12-02 | 2017-06-09 | 杭州海康威视数字技术股份有限公司 | A kind of image search method, apparatus and system |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4641414B2 (en) * | 2004-12-07 | 2011-03-02 | キヤノン株式会社 | Document image search apparatus, document image search method, program, and storage medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5428774A (en) * | 1992-03-24 | 1995-06-27 | International Business Machines Corporation | System of updating an index file of frame sequences so that it indexes non-overlapping motion image frame sequences |
US5493677A (en) * | 1994-06-08 | 1996-02-20 | Systems Research & Applications Corporation | Generation, archiving, and retrieval of digital images with evoked suggestion-set captions and natural language interface |
US5546572A (en) * | 1991-08-28 | 1996-08-13 | Hitachi, Ltd. | Method for retrieving database of image information |
US5557728A (en) * | 1991-08-15 | 1996-09-17 | International Business Machines Corporation | Automated image retrieval and scaling into windowed displays |
US5615112A (en) * | 1993-01-29 | 1997-03-25 | Arizona Board Of Regents | Synthesized object-oriented entity-relationship (SOOER) model for coupled knowledge-base/database of image retrieval expert system (IRES) |
-
1997
- 1997-05-16 CA CA002290445A patent/CA2290445A1/en not_active Abandoned
- 1997-05-16 JP JP54918598A patent/JP2001525959A/en not_active Ceased
- 1997-05-16 WO PCT/US1997/009256 patent/WO1998052119A1/en active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5557728A (en) * | 1991-08-15 | 1996-09-17 | International Business Machines Corporation | Automated image retrieval and scaling into windowed displays |
US5546572A (en) * | 1991-08-28 | 1996-08-13 | Hitachi, Ltd. | Method for retrieving database of image information |
US5428774A (en) * | 1992-03-24 | 1995-06-27 | International Business Machines Corporation | System of updating an index file of frame sequences so that it indexes non-overlapping motion image frame sequences |
US5615112A (en) * | 1993-01-29 | 1997-03-25 | Arizona Board Of Regents | Synthesized object-oriented entity-relationship (SOOER) model for coupled knowledge-base/database of image retrieval expert system (IRES) |
US5493677A (en) * | 1994-06-08 | 1996-02-20 | Systems Research & Applications Corporation | Generation, archiving, and retrieval of digital images with evoked suggestion-set captions and natural language interface |
US5617119A (en) * | 1994-06-08 | 1997-04-01 | Systems Research & Applications Corporation | Protection of an electronically stored image in a first color space by the alteration of a digital component in a second color space |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1999067695A3 (en) * | 1998-06-22 | 2000-04-13 | Koninkl Philips Electronics Nv | Image retrieval system |
WO1999067696A3 (en) * | 1998-06-23 | 2000-04-13 | Koninkl Philips Electronics Nv | A scalable solution for image retrieval |
EP1195062A4 (en) * | 1999-05-17 | 2006-11-08 | Samsung Electronics Co Ltd | COLOR IMAGE PROCESSING |
EP1195062A1 (en) * | 1999-05-17 | 2002-04-10 | Samsung Electronics Co., Ltd. | Color image processing method |
US7245762B2 (en) | 1999-05-17 | 2007-07-17 | Samsung Electronics Co., Ltd. | Color image processing method |
FR2801992A1 (en) * | 1999-12-03 | 2001-06-08 | Canon Kk | Image searching method from database in computer, involves comparing two data items indicating region of interest in new image and representing visual content characteristic of example image |
US6782395B2 (en) | 1999-12-03 | 2004-08-24 | Canon Kabushiki Kaisha | Method and devices for indexing and seeking digital images taking into account the definition of regions of interest |
EP1184796A1 (en) * | 2000-08-29 | 2002-03-06 | Sudimage | Method of associative navigation in a multimedia database |
FR2815741A1 (en) * | 2000-10-24 | 2002-04-26 | Canon Kk | Indexing method for digital images and method for searching for digital images contained in a database based on the index which is for predefined regions of interest within the digital images |
US6801661B1 (en) | 2001-02-15 | 2004-10-05 | Eastman Kodak Company | Method and system for archival and retrieval of images based on the shape properties of identified segments |
GB2381615A (en) * | 2001-08-23 | 2003-05-07 | Hewlett Packard Co | System and method for facilitating image retrieval |
GB2417801A (en) * | 2004-09-07 | 2006-03-08 | Pepperdog Ltd | Image processing apparatus |
AT514355A1 (en) * | 2013-05-17 | 2014-12-15 | Ait Austrian Inst Technology | Used to select digital images from an image database |
AT514355B1 (en) * | 2013-05-17 | 2017-01-15 | Ait Austrian Institute Of Technology Gmbh | Used to select digital images from an image database |
CN106815272A (en) * | 2015-12-02 | 2017-06-09 | 杭州海康威视数字技术股份有限公司 | A kind of image search method, apparatus and system |
Also Published As
Publication number | Publication date |
---|---|
JP2001525959A (en) | 2001-12-11 |
CA2290445A1 (en) | 1998-11-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Smith et al. | Integrated spatial and feature image query | |
Niblack et al. | QBIC project: querying images by content, using color, texture, and shape | |
Fournier et al. | Retin: A content-based image indexing and retrieval system | |
US6192150B1 (en) | Invariant texture matching method for image retrieval | |
US8055103B2 (en) | Object-based image search system and method | |
US9007647B2 (en) | Methods and apparatus to identify images in print advertisements | |
US6512850B2 (en) | Method of and apparatus for identifying subsets of interrelated image objects from a set of image objects | |
EP1516264B1 (en) | Image retrieval by generating a descriptor for each spot of an image the cells of which having visual characteristics within a selected tolerance | |
Yoo et al. | Visual information retrieval system via content-based approach | |
WO1998052119A1 (en) | Method and system for image retrieval | |
JP2001521250A (en) | Information search and search system | |
CA2397424A1 (en) | Content-based image retrieval using positive and negative examples | |
Yang | Content-based image retrieval: a comparison between query by example and image browsing map approaches | |
Shih et al. | An intelligent content-based image retrieval system based on color, shape and spatial relations | |
JP3952592B2 (en) | Image search apparatus and method | |
Laaksonen et al. | Content-based image retrieval using self-organizing maps | |
James | Face Image retrieval with HSV color space using clustering techniques | |
Keyvanpour et al. | Feature weighting for improving document image retrieval system performance | |
JPH08263514A (en) | Method for automatic classification of document, method for visualization of information space, and information retrieval system | |
Fauqueur et al. | New image retrieval paradigm: logical composition of region categories | |
Chalechale et al. | An abstract image representation based on edge pixel neighborhood information (EPNI) | |
Buijs et al. | Visual learning of simple semantics in imagescape | |
Golshani et al. | Content-based image indexing and retrieval system in imageroadmap | |
Syeda-Mahmood | Extracting indexing keywords from image structures in engineering drawings | |
Kulkarni | Natural language based fuzzy queries and fuzzy mapping of feature database for image retrieval |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): CA JP US |
|
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
ENP | Entry into the national phase |
Ref document number: 2290445 Country of ref document: CA Ref country code: CA Ref document number: 2290445 Kind code of ref document: A Format of ref document f/p: F |
|
ENP | Entry into the national phase |
Ref country code: JP Ref document number: 1998 549185 Kind code of ref document: A Format of ref document f/p: F |
|
WWE | Wipo information: entry into national phase |
Ref document number: 09423770 Country of ref document: US |