+

WO2018142493A1 - Dispositif de traitement d'image, procédé de traitement d'image, programme de traitement d'image, procédé de capture d'image et objet mobile - Google Patents

Dispositif de traitement d'image, procédé de traitement d'image, programme de traitement d'image, procédé de capture d'image et objet mobile Download PDF

Info

Publication number
WO2018142493A1
WO2018142493A1 PCT/JP2017/003488 JP2017003488W WO2018142493A1 WO 2018142493 A1 WO2018142493 A1 WO 2018142493A1 JP 2017003488 W JP2017003488 W JP 2017003488W WO 2018142493 A1 WO2018142493 A1 WO 2018142493A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
subject
distance
cameras
feature point
Prior art date
Application number
PCT/JP2017/003488
Other languages
English (en)
Japanese (ja)
Inventor
創輔 山尾
山 姜
成幸 小田嶋
Original Assignee
富士通株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 富士通株式会社 filed Critical 富士通株式会社
Priority to PCT/JP2017/003488 priority Critical patent/WO2018142493A1/fr
Publication of WO2018142493A1 publication Critical patent/WO2018142493A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C15/00—Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/02—Details
    • G01C3/06—Use of electric means to obtain final indication

Definitions

  • the present invention relates to an image processing device, an image processing method, an image processing program, an image photographing method, and a moving body.
  • Image processing technology that simultaneously performs self-position estimation and environmental map creation is known. For example, a technique for calibrating a plurality of cameras or restoring an observed image using a parameter block acquired from images captured by a plurality of cameras is known.
  • the image processing apparatus projects the acquired three-dimensional coordinates onto the three-dimensional map by using the same camera and the three-dimensional coordinates and the projection function acquired at different times.
  • processing for generating a three-dimensional map using the correspondence relationship of three-dimensional coordinates acquired at different times by the same camera may be referred to as “intra-camera processing”.
  • a three-dimensional map is generated by setting a baseline length that assumes changes in the moving distance and direction of the camera.
  • a different baseline length is used for each camera. May be set.
  • an object is to provide an image processing apparatus, an image processing method, an image processing program, an image capturing method, and a moving body that can integrate a three-dimensional map reflecting a real scale.
  • the image processing device associates each of the images captured by the first camera and the second camera with each of the images captured at the first timing by the first camera.
  • a distance from the first or second camera to the feature point of the subject is calculated based on the obtained feature point and information on the position of the first and second cameras stored in the storage unit, Information about the distance from the first camera to the subject generated using the same feature point associated with each of the images taken by the camera at the first and second timings is calculated.
  • the correction is made based on the distance from the second camera to the subject.
  • FIG. 1 is a diagram illustrating an example of a plurality of cameras mounted on a moving body.
  • FIG. 2 is a diagram illustrating an example of camera movement between different time points.
  • FIG. 3 is a diagram illustrating an example of subject shooting by the first camera in the background art.
  • FIG. 4 is a diagram illustrating an example of intra-camera processing using the first camera in the background art.
  • FIG. 5 is a diagram illustrating an example of an intra-camera process using the second camera in the background art.
  • FIG. 6 is a diagram illustrating an example of an image processing system according to the first embodiment.
  • FIG. 7 is a diagram illustrating an example of functional blocks according to the first embodiment.
  • FIG. 8 is a diagram illustrating an example of the feature point storage unit according to the first embodiment.
  • FIG. 1 is a diagram illustrating an example of a plurality of cameras mounted on a moving body.
  • FIG. 2 is a diagram illustrating an example of camera movement between different time points.
  • FIG. 3 is
  • FIG. 9 is a diagram illustrating an example of a three-dimensional map storage unit according to the first embodiment.
  • FIG. 10 is a diagram illustrating an example of the relative position and orientation storage unit according to the first embodiment.
  • FIG. 11 is a diagram illustrating an example of the scale storage unit according to the first embodiment.
  • FIG. 12 is a diagram illustrating an example of inter-camera processing according to the first embodiment.
  • FIG. 13 is a diagram illustrating an example of correction processing of coordinates acquired by the first camera in the first embodiment.
  • FIG. 14 is a diagram illustrating an example of a correction process of coordinates acquired by the second camera according to the first embodiment.
  • FIG. 15 is a diagram illustrating an example of the three-dimensional map integrated in the first embodiment.
  • FIG. 16 is a flowchart illustrating an example of a three-dimensional map generation process according to the first embodiment.
  • FIG. 17 is a diagram illustrating an example of a computer that executes an information processing program.
  • FIG. 1 is a diagram illustrating an example of a plurality of cameras mounted on a moving body.
  • a plurality of cameras 100a, 100b, and 100c are mounted on the moving body 10.
  • the cameras 100a, 100b, and 100c are installed separately from each other.
  • the cameras 100a, 100b, and 100c shown in FIG. 1 capture images in different directions, but may be configured such that each camera captures the same direction.
  • the camera 100a is an example of a first camera
  • the camera 100b is an example of a second camera.
  • the camera 100a acquires a plurality of images for performing intra-camera processing by capturing the same feature point at different timings while moving.
  • the feature point is, for example, a boundary point between a road and a white line, which is taken from a traveling automobile, and will be described in detail later.
  • FIG. 2 is a diagram illustrating an example of camera movement between different time points. As shown in FIG. 2, the moving body having the camera 100a moves by a distance S from the time point t0 to the time point t1 after the time point t0. In this case, the camera 100a captures a feature point reflected in the position and orientation 110 at time t0. In addition, the camera 100a captures the same feature point that appears in the position and orientation 111 at the time point t1.
  • FIG. 3 is a diagram illustrating an example of subject shooting by the first camera in the background art.
  • the camera 100a captures the feature points 3000a and 3000b of the subject 3000 at the time point t0.
  • the camera 100a captures the feature points 3000a and 3000b of the subject 3000 at the time point t1.
  • the camera 100a is moved by the distance S and rotated by the angle r from the time point t0 to the time point t1.
  • the position / orientation 110 of the camera 100a changes to the position / orientation 111.
  • the moving distance and the rotation angle are obtained from the coordinates of the feature points in the acquired image. Is estimated.
  • the estimation of the movement distance and the rotation angle may be referred to as “posture estimation”.
  • FIG. 4 is a diagram showing an example of intra-camera processing using the first camera in the background art.
  • the distance Se to the feature point 3100a acquired by the camera 100a is not equal to the distance Sr from the actual camera 100a to the feature point 3000a.
  • the image processing apparatus considers that the camera 100a has moved or rotated by a predetermined baseline “1” based on the distance Se.
  • the baseline “1” is not equal to the actually moved distance S and the rotated angle r
  • the position / orientation 111 ′ of the camera 100a at the time t1 obtained by the attitude estimation by the image processing apparatus is the actual camera. It is different from the position and orientation 111 at the time t1 of 100a.
  • the subject position 3100 recognized by the image processing apparatus is different from the actual subject 3000 position.
  • FIG. 5 is a diagram illustrating an example of an intra-camera process using the second camera in the background art. Similar to the example shown in FIG. 4, the distance to the feature points 3200a and 3200c acquired by the camera 100b is not equal to the actual distance to the feature points 3000a and 3000c. In this case, the image processing apparatus also regards the image acquired by the camera 100b as having moved or rotated by a predetermined baseline “1 ′” that is different from the distance S that the camera 100b has actually moved.
  • the baseline “1” estimated for the camera 100a does not always match the baseline “1 ′” estimated for the camera 100b.
  • the feature points acquired by the camera 100a and the feature points acquired by the camera 100b do not overlap correctly.
  • the map generated by the feature points acquired by the camera 100a may not overlap with the map generated by the feature points acquired by the camera 100b.
  • inter-camera process a process of generating a three-dimensional map using correspondence relationships of three-dimensional coordinates acquired at different times by a plurality of cameras.
  • FIG. 6 is a diagram illustrating an example of an image processing system according to the first embodiment.
  • the image processing system 1 in this embodiment includes a plurality of cameras 100 a and 100 b, an SLAM (Simultaneous Localization and Mapping) device 200, and a scale processing device 300.
  • Each device is connected to be communicable with each other via a network such as the Internet or a bus.
  • the scale processing device 300 is an example of an image processing device.
  • the camera 100a and the camera 100b may be expressed as “camera 100” when they are expressed without distinction.
  • a configuration including two cameras 100 is illustrated, but the configuration is not limited thereto.
  • the image processing system includes three or more cameras 100. May be.
  • the camera 100 is an imaging device that is mounted on a moving body such as an automobile and images the surrounding environment of the moving body.
  • the camera 100 captures an image using, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor as an image sensor.
  • the camera 100 photoelectrically converts light received by the image sensor and performs A / D (Analog / Digital) conversion to generate a captured image.
  • the camera 100 transmits the generated captured image to the SLAM device 200.
  • the SLAM device 200 extracts feature points using an image acquired from the camera 100 and generates a three-dimensional map.
  • FIG. 7 is a diagram illustrating an example of functional blocks according to the first embodiment. As illustrated in FIG. 7, the SLAM apparatus 200 includes a communication unit 210, a storage unit 220, and a control unit 230.
  • the communication unit 210 receives the data transmitted from the camera 100 and outputs it to the control unit 230.
  • the storage unit 220 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.
  • the storage unit 220 includes a feature point storage unit 221 and a three-dimensional map storage unit 222.
  • the storage unit 220 stores, for example, a program executed by the control unit 230 and various types of information used for processing in the control unit 230.
  • the feature point storage unit 221 stores the coordinates of feature points extracted from the image acquired from the camera 100.
  • the feature point storage unit 221 is sequentially updated by the feature matching unit 233 described later when the camera 100 moves.
  • FIG. 8 is a diagram illustrating an example of a feature point storage unit according to the first embodiment.
  • the feature point storage unit 221 sets “acquisition time”, “three-dimensional coordinates (first camera)”, and “three-dimensional coordinates (second camera)” as “feature point IDs”. Store in association with each other.
  • feature point ID is an identifier for uniquely identifying a feature point extracted from an image.
  • acquisition time stores the time when the image from which the feature points are extracted is acquired.
  • the “three-dimensional coordinates (first camera)” stores the coordinates of feature points extracted from the image acquired at the acquisition time in the camera 100a.
  • “three-dimensional coordinates (second camera)” stores the coordinates of feature points extracted from the image acquired at the acquisition time in the camera 100b.
  • the three-dimensional coordinates are stored in different columns for each camera that has captured the image.
  • the present invention is not limited to this, and the feature point storage unit 221 further stores the identifier of the camera corresponding to the coordinates.
  • the configuration may be such that three-dimensional coordinates are stored in the same column.
  • the feature point storage unit 221 shown in FIG. 8 stores information about the same feature points photographed by different cameras at the same time, but in addition to this, the same camera has different times. Information on the same feature point photographed in is also stored.
  • the 3D map storage unit 222 stores a 3D map generated by a 3D map generation unit 235 described later.
  • the three-dimensional map storage unit 222 is sequentially updated by the three-dimensional map generation unit 235 when the camera 100 moves.
  • FIG. 9 is a diagram illustrating an example of a three-dimensional map storage unit according to the first embodiment.
  • the three-dimensional map storage unit 222 stores “acquisition time” and “three-dimensional coordinates (three-dimensional map after integration)” in association with “feature point IDs”.
  • three-dimensional coordinates (three-dimensional map after integration)” includes “three-dimensional coordinates (first camera)” and “three-dimensional coordinates (second camera)” integrated by the three-dimensional map generation unit 235.
  • the coordinates of the feature points after being stored are stored.
  • control unit 230 is realized, for example, by executing a program stored in an internal storage device using the RAM as a work area by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like.
  • the control unit 230 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).
  • control unit 230 includes an image data acquisition unit 231, a feature extraction unit 232, a feature matching unit 233, a posture estimation unit 234, and a three-dimensional map generation unit 235, and performs information processing described below. Implement or execute a function or action.
  • the internal configuration of the control unit 230 is not limited to the configuration illustrated in FIG. 7, and may be another configuration as long as the information processing described later is performed.
  • the image data acquisition unit 231 When the captured image is input from the camera 100, the image data acquisition unit 231 starts acquiring the input captured image.
  • the image data acquisition unit 231 acquires image data captured by the camera 100 and outputs the acquired image data to the feature extraction unit 232.
  • the image data acquisition unit 231 acquires an accurate time when image data is input using, for example, an RTC (Real-Time Clock) (not shown).
  • RTC Real-Time Clock
  • the feature extraction unit 232 extracts a feature point included in the input image data candidate range for each of the image data input from a plurality of cameras. It is. For example, the feature extraction unit 232 identifies a corner point by performing Harris corner detection. The feature extraction unit 232 may identify corner points based on other corner point detection methods other than Harris corner detection. The feature extraction unit 232 stores the corner points detected by the corner detection method in the corresponding camera row of the feature point storage unit 221 as feature points. Note that the method of extracting the feature points by the feature extraction unit 232 is not limited to this, and the feature points may be extracted by edge detection, for example.
  • the feature matching unit 233 acquires data used for the intra-camera process with reference to the stored feature points. For example, the feature matching unit 233 acquires the coordinates of the same feature point ID extracted from images acquired by the same camera at different time points stored in the feature point storage unit 221, that is, the feature point correspondence. . Then, the feature matching unit 233 outputs the acquired set pointtra corresponding to the feature points at different times to the posture estimation unit 234 and the three-dimensional map generation unit 235.
  • the feature matching unit 233 acquires the coordinates of the same feature point ID, that is, the feature point correspondences extracted from images acquired by different cameras at the same time stored in the feature point storage unit 221. Then, the feature matching unit 233 outputs to the scale processing device 300 the set pinter corresponding to the feature points at the acquired simultaneous points and the acquired set pintra corresponding to the feature points at different time points.
  • the posture estimation unit 234 performs posture estimation processing of the camera 100 using the set pintra output from the feature matching unit 233. For example, the posture estimation unit 234 estimates the initial position and posture T from the set pintra using a known five-point algorithm.
  • the posture estimation unit 234 when the posture estimation unit 234 receives the corrected scale input from the scale processing device 300, the posture estimation unit 234 corrects the estimated initial position and posture T. For example, the posture estimation unit 234 calculates the corrected position and posture T ′ by multiplying the initial position and posture T by the input scale s.
  • the posture estimation unit 234 outputs the initial position / posture T and the corrected position / posture T ′ to the three-dimensional map generation unit 235.
  • the three-dimensional map generation unit 235 generates a three-dimensional map using the pintra output from the feature matching unit 233 and the initial position / posture T or the corrected position / posture T ′ output from the posture estimation unit 234.
  • the 3D map generation unit 235 stores the coordinates of the generated 3D map in the 3D map storage unit 222.
  • the scale processing device 300 in this embodiment includes a storage unit 320 and a control unit 330.
  • the scale processing apparatus 300 is also realized by a device such as a computer, similarly to the SLAM apparatus 200, and may have various functional units included in a known computer in addition to the functional units illustrated in FIG.
  • the storage unit 320 stores programs executed by the control unit 330, various data, and the like.
  • the storage unit 320 corresponds to a semiconductor memory element such as a RAM, a ROM, and a flash memory, and a storage device such as an HDD.
  • the storage unit 320 includes a relative position / posture storage unit 321 and a scale storage unit 322. Further, the storage unit 320 stores, for example, a program executed by the control unit 330 and various types of information used for processing in the control unit 330.
  • the relative position / posture storage unit 321 stores information on the relative positions and postures of the cameras 100a and 100b. Information stored in the relative position and orientation storage unit 321 is set in advance by, for example, an administrator of the image processing system 1 (not shown).
  • FIG. 10 is a diagram illustrating an example of the relative position and orientation storage unit according to the first embodiment.
  • the relative position and orientation storage unit 321 stores “coordinates” and “angles” in association with “camera IDs”.
  • “camera ID” is an identifier for uniquely identifying a camera included in the image processing system 1.
  • “Coordinates” and “angles” store the difference between the position and orientation of the camera and the position and orientation of the reference camera.
  • the camera with the camera ID “C001” is used as a reference camera.
  • the coordinates of the camera with the camera ID “C001” are the origin (0, 0, 0), and there are no angles (0 °, 0 °, 0 °).
  • the coordinates of the camera with the camera ID “C002” are moved “100 mm” in the x direction from the origin, and the angle is shifted by “5 °” in the x axis direction.
  • the scale storage unit 322 stores the scale corrected by the scale correction unit 332 described later.
  • FIG. 11 is a diagram illustrating an example of the scale storage unit according to the first embodiment. As illustrated in FIG. 11, the scale storage unit 322 stores “scale” in association with “camera ID”. In FIG. 11, “scale” is a numerical value for correcting coordinates acquired by the camera with the corresponding camera ID. For example, the corrected three-dimensional coordinates are calculated by multiplying the three-dimensional coordinates by a scale.
  • control unit 330 is realized by executing a program stored in an internal storage device using the RAM as a work area, for example, by a CPU, MPU, or the like.
  • the control unit 330 may be realized by an integrated circuit such as an ASIC or FPGA, for example.
  • the control unit 330 includes a scale estimation unit 331 and a scale correction unit 332, and realizes or executes information processing functions and operations described below.
  • the internal configuration of the control unit 330 is not limited to the configuration illustrated in FIG. 7, and may be another configuration as long as the information processing described below is performed.
  • the scale estimation unit 331 is identical between the pintra and the pinter.
  • a set p ′ corresponding to the feature point that refers to the three-dimensional point is extracted.
  • the scale estimation unit 331 refers to the relative position / posture storage unit 321 and calculates a three-dimensional point group set M ′ of the set p ′. For example, the scale estimation unit 331 corrects the three-dimensional coordinates acquired by the camera 100b in the pinter using the relative position and orientation of the camera 100b acquired from the relative position and orientation storage unit 321. Then, the scale estimation unit 331 calculates M ′ that is a set of corrected three-dimensional coordinates.
  • the scale estimator 331 generates all three-dimensional points between a set M of three-dimensional points p in the initial three-dimensional map generated from pintra and M ′, which is a set of calculated p ′. , The distance ratio p / p ′ is calculated. Then, the scale estimation unit 331 employs the median value of all the calculated distance ratios p / p ′ as the scale s between M and M ′. The scale estimation unit 331 stores the adopted scale s in the scale storage unit 322 in association with the camera ID. Further, the scale estimation unit 331 similarly calculates the scale s ′ for the three-dimensional coordinate pbintra generated by the intra-camera process from the image acquired from the camera 100b.
  • the scale correction unit 332 corrects the initial three-dimensional map M and the initial position / posture T generated from the pintra using the calculated scale s. Specifically, when receiving the initial three-dimensional map M and the initial position and orientation T from the SLAM device 200, the scale correction unit 332 multiplies the scale s stored in the scale storage unit 322. Then, the scale correcting unit 332 outputs the corrected three-dimensional map M ′ and the corrected position / posture T ′ to the SLAM device 200. The scale correction unit 332 performs the same process for each camera 100.
  • FIG. 12 is a diagram illustrating an example of inter-camera processing according to the first embodiment.
  • the cameras 100a and 100b respectively photograph the same subject 3000 at the same time point t0.
  • the feature extraction unit 232 of the SLAM device 200 specifies the three-dimensional coordinates of the feature point 3000a from the images captured by the cameras 100a and 100b.
  • the feature extraction unit 232 specifies the coordinates for each of the images photographed by the cameras 100 a and 100 b and stores them in the feature point storage unit 221.
  • the feature matching unit 233 reads the feature point correspondence at the specified simultaneous point from the feature point storage unit 221, and outputs the set pinter corresponding to the feature point to the scale processing device 300.
  • the scale estimation unit 331 of the scale processing apparatus 300 uses the coordinates of the three-dimensional point p included in the input pinter and the position and orientation of the camera 100b stored in the relative position and orientation storage unit 321 to correct the coordinates.
  • the set p ′ is calculated.
  • the scale estimation unit 331 calculates the scale s using the calculated ratio between p ′ and p, and stores it in the scale storage unit 322.
  • the scale correction unit 332 multiplies the three-dimensional coordinates of the feature points, which are input from the SLAM device 200 and calculated by the intra-camera process on the image acquired from the camera 100a, by the scale s, and performs the correction.
  • the three-dimensional coordinates of the feature points are calculated.
  • FIG. 13 is a diagram illustrating an example of correction processing of coordinates acquired by the first camera in the first embodiment. As shown in FIG. 13, by correcting the three-dimensional coordinate of the feature point 3100a calculated by the intra-camera process with the scale s, the corrected three-dimensional coordinate 3110a becomes the three-dimensional coordinate of the actual feature point 3000a of the subject. And overlap.
  • the scale correction unit 332 also applies the three-dimensional feature points after correction to the three-dimensional coordinates of the feature points calculated by the intra-camera process for the image input from the SLAM device 200 and acquired from the camera 100b. Calculate the coordinates.
  • FIG. 14 is a diagram illustrating an example of a correction process of coordinates acquired by the second camera according to the first embodiment. As shown in FIG. 14, the corrected three-dimensional coordinates 3210a overlap with the three-dimensional coordinates of the actual feature point 3000a of the subject, and the corrected three-dimensional coordinates 3210c overlap with the three-dimensional coordinates of the feature point 3000c of the subject. .
  • FIG. 15 is a diagram illustrating an example of the three-dimensional map integrated in the first embodiment.
  • a three-dimensional map reflecting the actual scale can be integrated.
  • the three-dimensional coordinates 3110b and 3210c of the feature points after correction also overlap with the feature points 3000b and 3000c of the actual subject, respectively.
  • FIG. 16 is a flowchart illustrating an example of a three-dimensional map generation process according to the first embodiment.
  • the image data acquisition unit 231 of the SLAM apparatus 200 stands by until image data is acquired from the first camera 100a (S100: No).
  • the image data acquisition unit 231 outputs the acquired image data to the feature extraction unit 232.
  • the feature extraction unit 232 extracts the feature point pa from the input image data and stores it in the feature point storage unit 221 (S101).
  • the feature matching unit 233 refers to the feature point storage unit 221, obtains a set pintra corresponding to feature points at different time points, and outputs them to the posture estimation unit 234 and the three-dimensional map generation unit 235 (S102).
  • the posture estimation unit 234 estimates the initial position and posture T of the camera 100a using the pintra output from the feature matching unit 233, and outputs the estimated initial position / posture T to the three-dimensional map generation unit 235 (S103).
  • the three-dimensional map generation unit 235 generates an initial three-dimensional map Mi using the pintra output from the feature matching unit 233 and the initial position / posture T output from the posture estimation unit 234. It memorize
  • the image data acquisition unit 231 waits until image data is acquired from the second camera 100b (S110: No).
  • the image data acquisition unit 231 outputs the acquired image data to the feature extraction unit 232.
  • the feature extraction unit 232 extracts the feature point pb from the input image data and stores it in the feature point storage unit 221 (S111).
  • the feature matching unit 233 refers to the feature point storage unit 221 and acquires a set pinter corresponding to feature points of pa and pb at the same point. Then, the feature matching unit 233 outputs the acquired pintra and pinter to the scale processing device 300 (S112).
  • the scale estimation unit 331 of the scale processing apparatus 300 receives the input of the pintra and the pinter from the SLAM apparatus 200, the scale estimation unit 331 extracts a set p ′ corresponding to the feature point that refers to the same three-dimensional point between the pintra and the pinter. (S113). Next, the scale estimation unit 331 calculates the three-dimensional point group set M ′ of the set p ′ using the relative position and orientation stored in the relative position and orientation storage unit 321 (S ⁇ b> 114).
  • the scale estimation unit 331 acquires a set M of three-dimensional points p of the initial three-dimensional map generated from pintra (S115). Then, the scale estimation unit 331 calculates the scale s using the calculated M ′ and M, and stores it in the scale storage unit 322 (S116).
  • the scale correcting unit 332 calculates a corrected three-dimensional map M′i obtained by multiplying Mi ′ by s using the scale s (S117). In addition, the scale correction unit 332 calculates a corrected position / posture T ′ obtained by multiplying the initial position / posture T by s using the scale s (S118). Then, the scale correcting unit 332 outputs the calculated corrected three-dimensional map M′i and the corrected position / posture T ′ to the SLAM device 200.
  • the three-dimensional map generation unit 235 that has received the output of the corrected three-dimensional map M′i and the corrected position / posture T ′ applies the processing of S122 described below to all the coordinates acquired from the n cameras 100. (S121).
  • the three-dimensional map generation unit 235 sequentially generates the three-dimensional map Mk using the corrected position / posture T ′ (S122). The three-dimensional map generation unit 235 returns to S121 and repeats the processing until the processing for the coordinates acquired from all the cameras 100 is completed (S123: No).
  • the 3D map generation unit 235 When the process of S122 is completed for the coordinates acquired from all the cameras 100 (S123: Yes), the 3D map generation unit 235 superimposes all the generated 3D maps in the 3D map storage unit 222. Store (S124).
  • the transmission apparatus divides the data to be transmitted into a plurality of pieces of divided data, and the divided data and the own apparatus hold each of the plurality of pieces of divided data. Generate primary verification data from the key. Then, after transmitting the transmission data for verification including the generated primary verification data to the receiving device, the transmitting device transmits a plurality of pieces of divided data to the receiving device. Also, in the data transmission / reception method according to the present embodiment, when the receiving device receives the verification transmission data and receives the divided data, the verification data is generated from the divided data and the common key held by the own device. Then, the receiving device determines the validity of the divided data according to the comparison result between the generated verification data and each primary verification data corresponding to the divided data included in the verification transmission data. Thereby, the receiving apparatus can integrate the three-dimensional map reflecting the actual scale.
  • the SLAM device 200 and the scale processing device 300 have been described in the above embodiment, the SLAM device 200 may be configured to have the functions of the scale processing device 300. Further, the camera 100a and 100b, the SLAM device 200, and the scale processing device 300 may be configured to be mounted on one moving body such as an automobile. Thereby, for example, a three-dimensional map without deviation can be provided in real time to a driver of a car.
  • the three-dimensional map generation process can be performed, for example, when acquisition of image data is started, but the timing for performing the process is not limited to this.
  • processing may be performed at predetermined intervals such as every second, and processing is performed when the number of acquired three-dimensional coordinates exceeds a predetermined threshold or when a key frame is displayed. May be.
  • the processing may be performed when information about the position of the camera 100 is changed, such as when the position and orientation of the camera 100a and the camera 100b change due to an external impact or the like.
  • the process may be performed when the moving speed or the rotation angle of the camera 100a and the camera 100b is changed, and the process may be repeated every time the process is finished. Thereby, the shift
  • each component of each part shown in the figure is functionally conceptual and does not necessarily need to be physically configured as shown.
  • the specific form of distribution / integration of each unit is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed / integrated in arbitrary units according to various loads or usage conditions. Can be configured.
  • various processing functions performed by each device may be executed entirely or arbitrarily on a CPU (or a microcomputer such as an MPU or MCU (Micro Controller Unit)).
  • various processing functions may be executed in whole or in any part on a program that is analyzed and executed by a CPU (or a microcomputer such as an MPU or MCU) or on hardware based on wired logic. Needless to say, it is good.
  • each component of each part illustrated does not necessarily need to be physically configured as illustrated.
  • the specific form of distribution / integration of each unit is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed / integrated in arbitrary units according to various loads or usage conditions. Can be configured.
  • the image data acquisition unit 231 and the feature extraction unit 232 may be integrated.
  • the illustrated processes are not limited to the above-described order, and may be performed at the same time as long as the processing contents do not contradict each other, or may be performed by changing the order.
  • each device may be executed entirely or arbitrarily on a CPU (or a microcomputer such as MPU or MCU (Micro Controller Unit)).
  • various processing functions may be executed in whole or in any part on a program that is analyzed and executed by a CPU (or a microcomputer such as an MPU or MCU) or on hardware based on wired logic. Needless to say, it is good.
  • FIG. 17 is a diagram illustrating an example of a computer that executes an information processing program.
  • a computer having a function equivalent to that of the scale processing device 300 will be described, but the SLAM device 200 and the moving body 10 can also be realized by the same configuration.
  • the computer 900 includes a CPU 901 that executes various arithmetic processes, an input device 902 that receives data input, and a monitor 903.
  • the computer 900 also includes a medium reading device 904 that reads a program and the like from a storage medium, an interface device 905 for connecting to various devices, and a communication device 906 for connecting to other information processing devices and the like by wire or wirelessly.
  • Have The computer 900 also includes a RAM 907 that temporarily stores various information and a hard disk device 908. Each device 901 to 908 is connected to a bus 909.
  • the hard disk device 908 stores an information processing program having the same functions as the processing units of the scale estimation unit 331 and the scale correction unit 332 shown in FIG. Also, the hard disk device 908 stores a relative position and orientation storage unit 321 and a scale storage unit 322, and various data for realizing an information processing program.
  • the input device 902 receives input of various information such as operation information from a user of the computer 900, for example.
  • the monitor 903 displays various screens such as a display screen for the user of the computer 900, for example.
  • a camera or the like is connected to the interface device 905.
  • the communication device 906 has the same function as the communication unit 210 illustrated in FIG. 7, for example, and exchanges various types of information with the camera 100.
  • the CPU 901 reads out each program stored in the hard disk device 908, develops it in the RAM 907, and executes it to perform various processes.
  • these programs can cause the computer 900 to function as the scale estimation unit 331 and the scale correction unit 332 illustrated in FIG.
  • the computer 900 may read and execute a program stored in a storage medium readable by the computer 900.
  • the storage medium readable by the computer 900 corresponds to, for example, a portable recording medium such as a CD-ROM, a DVD disk, a USB (Universal Serial Bus) memory, a semiconductor memory such as a flash memory, a hard disk drive, and the like.
  • the information processing program may be stored in a device connected to a public line, the Internet, a LAN, or the like, and the computer 900 may read out and execute the information processing program therefrom.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Electromagnetism (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

La présente invention comprend : dans des images d'un sujet capturées par une première caméra et une seconde caméra, en fonction de points caractéristiques associés les uns aux autres respectivement à partir des images capturées par les première et seconde caméras à un premier instant et d'informations (321) sur les positions des première et seconde caméras, mémorisées dans une unité de mémorisation, le calcul (331) d'une distance entre la première ou la seconde caméra et le point caractéristique du sujet ; et la correction (332), en fonction de la distance calculée entre la première ou la seconde caméra et le sujet, des informations sur la distance entre la première caméra et le sujet, générées à l'aide desdits points caractéristiques associés les uns aux autres respectivement à partir d'images capturées par la première caméra aux premier et second moments. Ainsi, des cartes tridimensionnelles reflétant une échelle réelle peuvent être intégrées.
PCT/JP2017/003488 2017-01-31 2017-01-31 Dispositif de traitement d'image, procédé de traitement d'image, programme de traitement d'image, procédé de capture d'image et objet mobile WO2018142493A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2017/003488 WO2018142493A1 (fr) 2017-01-31 2017-01-31 Dispositif de traitement d'image, procédé de traitement d'image, programme de traitement d'image, procédé de capture d'image et objet mobile

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2017/003488 WO2018142493A1 (fr) 2017-01-31 2017-01-31 Dispositif de traitement d'image, procédé de traitement d'image, programme de traitement d'image, procédé de capture d'image et objet mobile

Publications (1)

Publication Number Publication Date
WO2018142493A1 true WO2018142493A1 (fr) 2018-08-09

Family

ID=63039396

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/003488 WO2018142493A1 (fr) 2017-01-31 2017-01-31 Dispositif de traitement d'image, procédé de traitement d'image, programme de traitement d'image, procédé de capture d'image et objet mobile

Country Status (1)

Country Link
WO (1) WO2018142493A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110673947A (zh) * 2019-08-12 2020-01-10 江苏博人文化科技有限公司 一种减少激光slam建图所需内存的方法
CN113474819A (zh) * 2019-03-27 2021-10-01 索尼集团公司 信息处理装置、信息处理方法和程序

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005049999A (ja) * 2003-07-30 2005-02-24 Ricoh Co Ltd 画像入力装置、画像入力方法、この方法を情報処理装置上で実行可能に記述されたプログラム、及びこのプログラムを記憶した記憶媒体
JP2007263657A (ja) * 2006-03-28 2007-10-11 Denso It Laboratory Inc ３次元座標取得装置
WO2012172870A1 (fr) * 2011-06-14 2012-12-20 日産自動車株式会社 Dispositif de mesure de distance et appareil de génération de carte d'environnement
JP2016516249A (ja) * 2014-02-20 2016-06-02 エヌイーシー ラボラトリーズ アメリカ インクＮＥＣ Ｌａｂｏｒａｔｏｒｉｅｓ Ａｍｅｒｉｃａ， Ｉｎｃ． 単一カメラを用いた３ｄでの移動物体の位置測定
US20160314593A1 (en) * 2015-04-21 2016-10-27 Hexagon Technology Center Gmbh Providing a point cloud using a surveying instrument and a camera device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005049999A (ja) * 2003-07-30 2005-02-24 Ricoh Co Ltd 画像入力装置、画像入力方法、この方法を情報処理装置上で実行可能に記述されたプログラム、及びこのプログラムを記憶した記憶媒体
JP2007263657A (ja) * 2006-03-28 2007-10-11 Denso It Laboratory Inc ３次元座標取得装置
WO2012172870A1 (fr) * 2011-06-14 2012-12-20 日産自動車株式会社 Dispositif de mesure de distance et appareil de génération de carte d'environnement
JP2016516249A (ja) * 2014-02-20 2016-06-02 エヌイーシー ラボラトリーズ アメリカ インクＮＥＣ Ｌａｂｏｒａｔｏｒｉｅｓ Ａｍｅｒｉｃａ， Ｉｎｃ． 単一カメラを用いた３ｄでの移動物体の位置測定
US20160314593A1 (en) * 2015-04-21 2016-10-27 Hexagon Technology Center Gmbh Providing a point cloud using a surveying instrument and a camera device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
OZOG, PAUL ET AL.: "On the Importance of Modeling Camera Calibration Uncertainty in Visual SLAM", 2013 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA, 6 May 2013 (2013-05-06), pages 3777 - 3784, XP032506698, ISSN: 1050-4729 *
vol. 2014-CVT, no. 2, pages 1 - 8 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113474819A (zh) * 2019-03-27 2021-10-01 索尼集团公司 信息处理装置、信息处理方法和程序
CN110673947A (zh) * 2019-08-12 2020-01-10 江苏博人文化科技有限公司 一种减少激光slam建图所需内存的方法
CN110673947B (zh) * 2019-08-12 2022-04-05 江苏博人文化科技有限公司 一种减少激光slam建图所需内存的方法

Similar Documents

Publication Publication Date Title
EP3665506B1 (fr) Appareil et procédé permettant de générer une représentation d'une scène
CN101630406B (zh) 摄像机的标定方法及摄像机标定装置
JP2017531976A (ja) アレイカメラを動的に較正するためのシステム及び方法
CN107843251B (zh) 移动机器人的位姿估计方法
KR101617078B1 (ko) 무인 항공기 영상과 지도 영상에 대한 영상 정합 장치 및 방법
CN105758426A (zh) 移动机器人的多传感器的联合标定方法
EP3100234A1 (fr) Système de traitement de donnée et procédé d'étalonnage d'un système de visualisation des alentours d'un véhicule
JP6566768B2 (ja) 情報処理装置、情報処理方法、プログラム
JP2008506953A5 (fr)
CN105335955A (zh) 对象检测方法和对象检测装置
WO2019019819A1 (fr) Dispositif électronique mobile et procédé de traitement de tâches dans une région de tâche
CN106908764B (zh) 一种多目标光学跟踪方法
CN105139401A (zh) 一种深度图中深度的可信度的评估方法
JP2014186004A (ja) 計測装置、方法及びプログラム
CN112422848B (zh) 一种基于深度图和彩色图的视频拼接方法
JP2013246606A (ja) データ導出装置、及び、データ導出方法
KR20210087511A (ko) 광각 이미지로부터의 디스패리티 추정
WO2018142493A1 (fr) Dispositif de traitement d'image, procédé de traitement d'image, programme de traitement d'image, procédé de capture d'image et objet mobile
US9538161B2 (en) System and method for stereoscopic photography
CN106683133B (zh) 一种获取目标深度图像的方法
JP2010258897A (ja) 判定プログラムおよびキャリブレーション装置
JP2017059998A (ja) 画像処理装置およびその方法、並びに、撮像装置
CN115797466A (zh) 一种快速的三维空间标定方法
US11166005B2 (en) Three-dimensional information acquisition system using pitching practice, and method for calculating camera parameters
JP4132068B2 (ja) 画像処理装置及び三次元計測装置並びに画像処理装置用プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17895390

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17895390

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载