CN115697652A - Camera Position Deviation Measuring Method - Google Patents

Abstract

The method for measuring the positional deviation of the camera is applied to a robot having a robot arm attached to the front end of the robot arm via a rotation axis, a hand attached to the robot arm so that the optical axis is parallel to the rotation axis, and a camera attached to the robot arm. The method comprises: step (a), attaching a tool with markings to the hand; step (b), rotating the hand around a rotation axis and photographing the markings with a camera at a plurality of rotational positions; and step (c), based on multiple The difference between the position of the mark captured in each captured image captured at each rotation position and the position of the ideal mark is used to find the positional deviation of the camera.

Description

Camera Position Deviation Measuring Method

technical field

This specification discloses a method for measuring positional deviation of a camera.

Background technique

Conventionally, among robots equipped with arms, a robot that performs calibration of a camera provided independently of the arm has been proposed (for example, refer to Patent Document 1). The camera calibration processing procedure first sets three rotation axes X, Y, and Z of the jaw coordinate system, and operates the arm so that the calibration pattern held at the end of the hand rotates around each rotation axis. Next, pattern images of the calibration pattern at a plurality of rotational positions rotated about each rotational axis are captured by a camera. Then, a coordinate transformation matrix between the gripper coordinate system and the camera coordinate system is estimated using these pattern images. According to this processing procedure, external parameters capable of calculating the coordinate transformation between the gripper coordinate system and the camera coordinate system can be obtained, so that position detection of an object using a camera can be performed.

prior art literature

Patent Document 1: Japanese Patent Laid-Open No. 2019-14031

Contents of the invention

The problem to be solved by the invention

In the above-mentioned Patent Document 1, there is description about performing calibration of a camera installed independently of the arm, but there is no mention about performing calibration (measurement of positional deviation) of the camera attached to the arm.

The main purpose of the present disclosure is to measure the positional deviation of a camera attached to an arm with a simple method.

means to solve the problem

The method for measuring the amount of positional deviation of a camera of the present disclosure employs the following means in order to achieve the above-mentioned main purpose.

The method for measuring the amount of positional deviation of a camera of the present disclosure measures the positional deviation of the camera in a robot having a robot arm, a hand attached to the front end of the robot arm via a rotation axis, and a camera such that the optical axis is aligned with the camera. The above-mentioned rotating shaft is installed on the above-mentioned robot arm in a parallel manner,

The method for measuring the positional deviation of the above-mentioned camera has the following steps:

Step (a), installing a marked tool to the hand;

step (b), rotating said hand about said axis of rotation and photographing said mark with said camera at a plurality of rotational positions; and

Step (c), calculating the positional deviation of the camera based on the difference between the position of the mark captured in each of the captured images captured at the plurality of rotation positions and an ideal position of the mark.

The method for measuring the amount of positional deviation of a camera of the present disclosure measures the positional deviation of a camera in a robot having a robot arm, a hand attached to the front end of the robot arm via a rotation axis, and a camera whose optical axis is aligned with the rotation axis. Mounted on the robot arm in parallel. In this method, a tool with a marker is attached to a hand, the hand is rotated around a rotation axis, and the marker is photographed with a camera at a plurality of rotational positions. Then, the positional deviation of the camera is obtained based on the difference between the position of the mark captured in each of the captured images captured at the plurality of rotation positions and the position of the ideal mark. Thereby, the measurement of the positional deviation of the camera attached to an arm can be performed with a simple process.

Description of drawings

FIG. 1 is an external perspective view of a working robot.

Fig. 2 is a side view of the main body of the robot.

Fig. 3 is a block diagram showing the electrical connection relationship between the robot main body and the control device.

FIG. 4 is an explanatory view showing an example of a camera positional deviation measurement step.

FIG. 5 is an explanatory diagram showing a state of attachment of the marking tool.

FIG. 6 is an explanatory view showing a designed camera field of view and an actual camera field of view when the camera is misaligned.

FIG. 7 is an explanatory diagram illustrating positional deviation (X _c , Y _c ) and rotational deviation θ _c in the X _vr Y _vr coordinate system based on the actual camera field of view.

FIG. 8 is an explanatory diagram illustrating positional deviation (X _ci , Y _ci ) in the X _vi Y _vi coordinate system based on the designed camera field of view.

Detailed ways

Next, an embodiment for implementing the present disclosure will be described with reference to the drawings.

FIG. 1 is an external perspective view of a working robot. Fig. 2 is a side view of the main body of the robot. Fig. 3 is a block diagram showing the electrical connection relationship between the robot main body and the control device.

The work robot 1 is configured as a horizontal articulated robot that performs predetermined operations on the workpiece W (for example, conveyance operation of picking up and conveying the workpiece W, assembly operation of picking up the workpiece W and assembling it to an object, etc.). The working robot 1 includes a robot main body 10 (see FIGS. 1 to 3 ) and a control device 70 (see FIG. 3 ) that controls the robot main body 10 . As shown in FIGS. 1 and 2 , the robot main body 10 includes a base 11 and a multi-joint arm 20 .

The base 11 is fixed to the workbench 2 and supports the base end side of the multi-joint arm 20 . The multi-joint arm 20 includes a first arm 21 , a first arm drive unit 30 , a second arm 22 , a second arm drive unit 40 , a shaft 23 , a shaft drive unit 50 , and a camera 60 . The first arm 21 is configured such that the base end thereof is connected to the base 11 via the first joint axis J1 , and can rotate (horizontally swivel) relative to the base 11 in the horizontal plane by the rotation of the first joint axis J1 . The second arm 22 is configured such that its base end is connected to the front end of the first arm 21 via the second joint axis J2, and can rotate (horizontally swivel) relative to the first arm 21 in the horizontal plane by the rotation of the second joint axis J2. . The shaft 23 is connected to the front end portion of the second arm 22 via the third joint axis J3, and is configured to be rotatable around the axis of the third joint axis J3 and to be able to move along the axial direction of the third joint axis J3 with respect to the second arm 22. Lift and lower. A tool holder 24 for holding various tools for working on the workpiece W is provided at the tip of the shaft 23 .

As shown in FIG. 3 , the first arm drive unit 30 includes a motor 32 and an encoder 34 . The rotation shaft of the motor 32 is connected to the first joint shaft J1 via a reduction gear not shown. The first arm drive unit 30 horizontally turns the first arm 21 with the first joint axis J1 as a fulcrum using the torque transmitted to the first joint axis J1 via the reduction gear by the drive motor 32 . The encoder 34 is configured as a rotary encoder that is attached to the rotating shaft of the motor 32 and that detects the amount of rotational displacement of the motor 32 .

The second arm drive unit 40 includes a motor 42 and an encoder 44 similarly to the first arm drive unit 30 . The rotation shaft of the motor 42 is connected to the second joint shaft J2 via a reduction gear not shown. The second arm drive unit 40 horizontally turns the second arm 22 around the second joint axis J2 by using the torque transmitted to the second joint axis J2 via the reduction gear by the drive motor 42 . The encoder 44 is configured as a rotary encoder that is attached to the rotating shaft of the motor 42 and that detects the amount of rotational displacement of the motor 42 .

As shown in FIG. 3 , the shaft drive unit 50 includes motors 52a, 52b and encoders 54a, 54b. The rotating shaft of the motor 52a is connected to the shaft 23 via a belt (not shown), and the shaft 23 is rotated around the shaft. The rotating shaft of the motor 52b is connected to a ball screw nut (not shown) passing through the shaft 23 via a belt, and the shaft 23 is vertically raised and lowered by rotating the ball screw nut. The encoder 54 a is configured as a rotary encoder that detects the amount of rotational displacement of the shaft 23 . The encoder 54b is configured as a linear encoder that detects the elevation position of the shaft 23 .

The camera 60 is attached to the front end side of the second arm 22 so that its optical axis is parallel to the axis of the shaft 23 . The camera 60 photographs the workpiece W as a work object, and outputs the photographed image to the control device 70 . The control device 70 recognizes the position of the workpiece W by processing the captured image.

As shown in FIG. 3 , the control device 70 includes a CPU 71 , a ROM 72 , a RAM 73 , a nonvolatile memory (storage device 74 ), and an input/output interface (not shown). Position signals from the encoders 34 , 44 , 54 a , and 54 b , image signals from the camera 60 , and the like are input to the control device 70 via the input/output interface. Drive signals and the like for the motors 32, 42, 52a, 52b are output from the control device 70 via the input/output interface.

Next, the operation of the working robot 1 configured in this way will be described. The CPU 71 of the control device 70 first controls the first arm drive unit 30 , the second arm drive unit 40 , and the shaft drive unit 50 so that the camera 60 moves above the workpiece W so that the tip of the hand moves to a predetermined target position. And the workpiece W is photographed by the camera 60 . Next, the CPU 71 performs image processing on the processed captured image to measure the position of the workpiece W in the camera coordinate system. The image processing is performed by measuring the coordinate values of the workpiece W captured in the captured image, and performing positional misalignment correction and rotational misalignment correction on the measured coordinate values using correction values. Here, the correction value is used to correct a deviation in the positional relationship between the camera 60 and the end of the hand caused by a processing error or an assembly error of the camera 60 . The correction value is measured in advance in a camera position deviation measurement step described later, and is stored in the storage device 74 . Next, the CPU 71 converts the position of the workpiece W into the robot coordinate system, and sets the target position of the tip of the hand for picking up the workpiece W based on the converted position of the workpiece W. Then, the CPU 71 controls the first arm drive unit 30 , the second arm drive unit 40 , and the shaft drive unit 50 so as to move the hand tip to a set target position. The workpiece W is picked up by the end of the hand (work holding unit 24 ) as the end of the hand moves to the target position.

Next, the camera position deviation measurement step will be described. FIG. 4 is an explanatory view showing an example of a camera positional deviation measurement step. The camera position deviation measurement step is executed through steps S100 to S150.

In step S100 , as shown in FIG. 5 , the marking tool 100 is held on the tool holding portion 24 (hand end) provided at the tip of the shaft 23 . The marked tool 100 has a mark M at a position away from the attachment portion attached to the tool holder 24 in a direction perpendicular to the axis 23 . In step S110 , the shaft drive unit 50 is controlled so that the shaft 23 rotates by a predetermined angle (for example, 10 degrees each), and the camera 60 images the mark M every time the shaft 23 rotates by a predetermined angle. Thus, captured images of the mark M are obtained at a plurality of rotational positions n (n=1, 2, 3 . . . ). In step S120, _the marker position ( _{X vn ,} Y _vn ) (n=1, 2, 3...). The measurement of the marker position (X _vn , Y _vn ) is performed by processing the captured image to obtain the center coordinates of the marker M captured in the image.

In step S130, based on the marker position (X _vn , Y _vn ) (n ₌ 1, 2, 3...) measured in the X vr Y _vr coordinate system and the previously stored in ROM 72 in the X _vr Y _vr coordinate system The position deviation (X _c , Y _c ) and the rotation deviation θ _c of the camera 60 are derived from the difference of the ideal marker position (X _vi , Y _vi ) (n=1, 2, 3 . . . ). The positional deviations (X _c , Y _c ) represent the amount of positional deviation in the direction of the X _vr axis and the amount of positional deviation in the direction of the Y _vr axis, respectively. In addition, the rotation deviation θ _c indicates the amount of deviation in the rotational direction between the actual camera field of view including the positional deviation of the camera 60 and the designed camera field of view not including the positional deviation of the camera 60 . In this embodiment, step S130 derives _{the difference f xk and} The square sum F of the difference f _yk in the direction of the Y _vr axis is the smallest combination of the position deviation (X _c , Y _c ) and the rotation deviation θ _c . In step S140, using the rotation deviation θ _c , the positional deviation (X _c , Y _c ) is changed from the coordinate system based on the actual camera field of view (X _vr Y _vr coordinate system ) is transformed into a coordinate system (X _vi Y _vi coordinate system) based on the designed camera field of view. Thus, in the X _vi Y _vi coordinate system, the positional deviation (X _ci , Y _ci ) of the camera 60 in the X _vi Y _vi axis direction is derived. In step S150, a positional offset value equal to and opposite to the positional deviation (X _ci , Y _ci ) and a rotational offset value equal to and opposite to the rotational deviation θ _c are registered in the storage device as correction values. 74. Thus, when picking up the workpiece W, by performing offset correction (positional offset and rotational offset) on the position of the workpiece W measured by photographing the workpiece W with the camera 60 , it is possible to accurately recognize The position of the workpiece W. As a result, the accuracy of work (pickup) can be further improved.

[number 1]

f _xk ＝x _in -{(x _rn +x _c )cosθ _c -(y _rn +y _c )sinθ _c }…(1)

f _yk ＝y _in -{(x _rn +x _c )sinθ _c +(y _rn +y _c )cosθ _c }…(2)

x _ci ＝x _c cosθ _c -y _c sinθ _c …(4)

y _ci ＝x _c sinθ _c +y _c cosθ _c …(5)

Here, in step S130, between the measured mark position (X _vn , Y _vn ) and the ideal mark position (X _vi , Y _vi ), theoretically, the following equations (6) and (7) hold true . Therefore, the positional deviation (X _c , Y _c ) and the rotational deviation θ _c of the camera 60 can be derived using equations (6) and (7). _{However, in reality, various errors are included in the relationship between the two. Therefore, it is possible to obtain the positional deviation (X c ,} Y _c ) _and Combination of rotation deviation θ _c to further improve the correction accuracy.

[number 2]

x _in ＝(x _rn +x _c )cosθ _c -(y _rn +y _c )sinθ _c …(6)

y _in ＝(x _rn +x _c )sinθ _c +(y _rn +y _c )cosθ _c …(7)

Here, the correspondence relationship between the main elements of the embodiment and the main elements of the present disclosure described in the claims will be described. That is, in the present embodiment, the multi-joint arm 20 corresponds to a robot arm, the tool holder 24 corresponds to a hand, and the camera 60 corresponds to a camera.

In addition, this disclosure is not limited at all by the above-mentioned embodiment, It goes without saying that it can implement in various aspects as long as it belongs to the technical scope of this disclosure.

For example, in the above-mentioned embodiments, the method of measuring the positional deviation of the camera according to the present disclosure was applied to a horizontal articulated robot and described. However, the invention is not limited thereto, as long as it has a robot arm such as a vertical articulated robot, a hand attached to the front end of the robot arm via a rotation axis, and a camera attached to the robot arm so that the optical axis is parallel to the rotation axis. structure, it can be applied to robots of any structure.

As described above, the camera positional deviation measurement method of the present disclosure measures the positional deviation of the camera in a robot having a robot arm, a hand attached to the front end of the robot arm via a rotation shaft, and a camera. The method for measuring the position deviation of the above-mentioned camera includes the following steps: step (a), attaching a tool with a mark to the above-mentioned hand; step (b), making the above-mentioned The hand rotates around the above-mentioned rotation axis and uses the above-mentioned camera to take pictures of the above-mentioned marks at a plurality of rotational positions; and step (c), based on the position and The difference between the ideal marker positions is used to find the positional deviation of the camera above.

The method for measuring the amount of positional deviation of a camera of the present disclosure measures the positional deviation of a camera in a robot having a robot arm, a hand attached to the front end of the robot arm via a rotation axis, and a camera whose optical axis is aligned with the rotation axis. Mounted on the robot arm in parallel. In this method, a tool with a marker is attached to a hand, the hand is rotated around a rotation axis, and the marker is photographed with a camera at a plurality of rotational positions. Then, the positional deviation of the camera is obtained based on the difference between the position of the mark captured in each of the captured images captured at the plurality of rotation positions and the position of the ideal mark. Thereby, the measurement of the positional deviation of the camera attached to an arm can be performed with a simple process.

In the position deviation measurement method of the camera of the present disclosure, it may also be set as follows: in the above step (c), in the X _vr Y _vr coordinate system based on the actual camera field of view, the above step (b) will be The positions of the markers that are imaged and measured at each rotation position n (n=1, 2...) are taken as (X _vn , Y _vn ), and the ideal position at each rotation position n (n=1, 2...) The position of the mark is set as (X _in , Y _in ), the positional deviations of the above-mentioned cameras in the X _vr axis direction and the Y _vr axis direction are set as (X _c , Y _c ), and the actual camera field of view is compared with the design When _the rotation deviation in the rotation direction of the camera _{field of} view above is set to θ _{c ,} use The above-mentioned position deviation (X _c , Y _c ) and the above-mentioned rotation deviation θ _c are obtained by using the relational expression, and the coordinate system of the above-mentioned position deviation (X _c , _{Y c )} is changed from the above-mentioned X _vr Y _vr coordinates based on the above-mentioned rotation deviation θ c The positional deviation of the above-mentioned camera is obtained by transforming the system into an X _vi Y _vi coordinate system based on the field of view of the camera in the above-mentioned design. In this way, the positional deviation of the camera can be obtained by simple processing based on the position of the mark measured by the camera at each rotation position n.

In this case, in the above step (c), a predetermined minimization algorithm may be used to obtain the positional deviation (X _c , Y _c ) and the above-mentioned rotation deviation θ _c , the coordinate system of the above-mentioned position deviation (X _c , Y _c ) is transformed from the above-mentioned X _vr Y _vr coordinate system to the above-mentioned X _vi Y by using equations (4) and (5) _vi coordinate system, thereby obtaining the position deviation (X _ci , Y _ci ) in the above X _vi Y _vi coordinate system. In this way, it is possible to further improve the measurement accuracy of the positional deviation and the rotational deviation of the camera.

In addition, in the positional deviation measurement method of the camera of the present disclosure, it is also possible to register an offset value corresponding to the positional deviation and rotational deviation of the camera obtained in the above step (c) as a value used for correction. The correction value of the position of the object measured by the above-mentioned camera imaging. In this way, the position of the object can be recognized with high precision using the camera.

Industrial availability

The present disclosure can be utilized in the robot manufacturing industry and the like.

Explanation of reference signs

1 working robot, 2 working table, 10 robot main body, 11 base, 20 multi-joint arm, 21 first arm, 22 second arm, 23 axis, 30 first arm driving part, 32 motor, 34 encoder, 40 Two-arm drive unit, 42 motor, 44 encoder, 50 shaft drive unit, 52a, 52b motor, 54a, 54b encoder, 60 camera, 70 control device, 71CPU, 72ROM, 73RAM, 74 storage device, J1 first joint axis , J2 second joint axis, J3 third joint axis.

Claims (4)

1. A method for measuring positional deviation of a camera, which measures positional deviation of the camera in a robot having a robot arm, a hand attached to a distal end portion of the robot arm via a rotation axis, and the camera attached to the robot arm such that an optical axis is parallel to the rotation axis,

the method for measuring the positional deviation of the camera comprises the following steps:

a step (a) of mounting a tool having a mark to the hand;

a step (b) of rotating the hand around the rotation axis and photographing the mark with the camera at a plurality of rotation positions; and

and (c) finding a positional deviation of the camera based on a difference between a position of a marker captured in each captured image captured at each of the plurality of rotational positions and a position of an ideal marker.

2. The method for measuring positional deviation of a camera according to claim 1,

in the step (c), X is based on the actual camera view _vr Y _vr In the coordinate system, the position of the marker measured by the step (b) by imaging at each rotational position n (n =1,2 …) is (X) _vn ，Y _vn ) And setting the position of the ideal mark at each rotation position n (n =1,2 …) as (X) _in ，Y _in ) X is then reacted with _vr Axial direction and Y _vr Each positional deviation of the camera in the axial direction is set to (X) _c ，Y _c ) And setting a rotational deviation in a rotational direction of the actual camera view with respect to the designed camera view to θ _c Using a marker position (X) representing said determination _vn ，Y _vn ) With said ideal mark position (X) _in ，Y _in ) The positional deviation (X) is obtained from the relational expression of the relationship between the two _c ，Y _c ) And the rotational deviation theta _c Based on said rotational deviation θ _c To deviate the position (X) _c ，Y _c ) From said X coordinate system _vr Y _vr The coordinate system is transformed into X relative to the field of view of the camera on the design _vi Y _vi A coordinate system, thereby finding a positional deviation of the camera.

3. The method for determining positional deviation of a camera according to claim 2,

in the step (c), the positional deviation (X) that minimizes the sum of squares F defined by the following equations (1) to (3) is obtained using a predetermined minimization algorithm _c ，Y _c ) And the rotational deviation theta _c In the combination of (1) and (5), the position is deviated by the following formulas (4) and (5)Difference (X) _c ，Y _c ) From said X coordinate system _vr Y _vr Coordinate system is transformed into said X _vi Y _vi Coordinate system, thereby finding said X _vi Y _vi Position deviation (X) in a coordinate system _ci ，Y _ci )，

f _xk ＝x _in -{(x _rn +x _c )cosθ _c -(y _rn +y _c )sinθ _c }...(1)

f _yk ＝y _in -{(x _rn +x _c )sinθ _c +(y _rn +y _c )cosθ _c }...(2)

x _ci ＝x _c cosθ _c -y _c sinθ _c ...(4)

y _ci ＝x _c sinθ _c +y _c cosθ _c ...(5)。

4. The method for measuring positional deviation of a camera according to any one of claims 1 to 3,

the method for measuring positional deviation of a camera includes the following step (d): registering offset values corresponding to the positional deviation and the rotational deviation of the camera obtained in the step (c) as correction values for correcting the position of the object measured by the image capturing with the camera.

Images