WO2008047872A1

WO2008047872A1 - Manipulator

Info

Publication number: WO2008047872A1
Application number: PCT/JP2007/070360
Authority: WO
Inventors: Saku Egawa
Original assignee: Hitachi, Ltd.
Priority date: 2006-10-20
Filing date: 2007-10-18
Publication date: 2008-04-24
Also published as: JPWO2008047872A1; CN101522377B; CN101522377A

Abstract

The position and shape of an object (109) held by a holding section (102) is detected based on an image of the periphery of the holding section (102) acquired by an image input device (2) provided on an arm (103) of a manipulator (101) and on a positional change of the manipulator (101) detected by an angle sensor (122) provided at a joint (112) between the holding section (102) and the arm (103). Further, a contact possibility determination device (4) detects the shape and position of the object (109) and compares them with the position of a peripheral object (108), detected by a periphery monitoring device (3), to determine the possibility of contact between the objects. Further, when there is a possibility that the object (109) comes into contact with the peripheral object (108), movement of the manipulator (101) is stopped or the approach between the held object (109) and the peripheral object (108) are warned by alarm means (5).

Description

Specification

Manipulator

Technical field

The present invention relates to a manipulator that holds an object with an arm and performs positioning and transportation.

Background art

[0002] A manipulator is a device in which a joint and an arm (arm) are combined like a human arm, and is a generic name for a device that grips an object and performs positioning and transportation. In general, manipulators include a gripping mechanism that grips an object and an arm mechanism that moves the gripping mechanism. The manipulator includes a mechanism that automatically controls movement of the arm and a mechanism that is operated by a person. Examples of manipulators whose arms are automatically controlled include industrial robot arms used for parts transportation and assembly in factories, and service robots that perform tasks such as housework and nursing care in public space 'office' homes. There are arms. Examples of manipulators that operate the arm by humans include construction machines that handle large 'heavy objects', master-slave manipulators that are used in space environments and nuclear power facilities, and medical operation support manipulators. .

[0003] Conventionally, safety techniques for preventing the manipulator from coming into contact with surrounding objects and people have been proposed. For arm-type robots, it has been proposed to install an intrusion detection sensor around the work area of the robot arm and to make an emergency stop when an obstacle enters! In addition, in construction machines, workers are equipped with infrared light emitters, and infrared light receivers are attached to the construction machines. When workers enter the alarm area set around the construction machines, infrared receivers are used. A construction machine warning device that detects workers and issues warnings has been proposed.

[0004] On the other hand, it is necessary to prevent the object held by the manipulator from coming into contact with surrounding objects and people. However, in the past, safety techniques related to the contact of gripped objects that can handle the case of gripping an unknown object that has not been given a shape in advance have been proposed! / ヽ. [0005] In order to prevent an unknown gripping object from coming into contact with the surroundings, a means for knowing the dimension of the object being gripped is required. As a technology related to this, there is a technology for obtaining information on the shape of an object for the purpose of grasping the object by a manipulator. As the technology, based on the image information acquired by the visual sensor provided in the arm type robot, the object to be grasped is applied to a simple shape by image recognition, its size and orientation are calculated, and the grasping method is obtained based on it. A method of reliably grasping an object of arbitrary shape has been proposed.

[0006] In Japanese Patent Application Laid-Open No. 2000-202790, an ultrasonic sensor provided in an arm type robot main body detects a surrounding moving object and moves the robot arm when the distance from the main body falls within a certain range. A robot apparatus that performs speed reduction control is disclosed.

[0007] Further, in Japanese Patent Laid-Open No. 2005-128959, an object region image indicating a learning target object is cut out by moving a learning target object by bringing a movable part such as an arm unit into contact with the learning target object, A robot apparatus that extracts features from the object region image and registers them in the object model database and a learning method thereof are disclosed. Here, the means for extracting the image of the target object uses a method of extracting an area that has changed before and after the target object is moved from the captured image.

[0008] Conventionally, when a manipulator is holding an unknown object, there has been no appropriate safety technique for preventing the gripped object from coming into contact with surrounding objects or people. In addition, the conventional method of obtaining information related to the shape of the object to be grasped by image recognition can be applied to a prepared environment such as a factory, but it can be applied to a complex environment such as a general office 'home or outdoors. There is a problem that it is difficult to separate the object to be grasped from the object in the background!

More specifically, in a general office 'complex environment such as at home or outdoors, surrounding objects other than the object to be gripped may move. As disclosed in Japanese Patent Laid-Open No. 2005-128959, as a method for extracting an object from an image when the object is moving, a method for extracting a changed portion from a plurality of images at different times is known. If the surrounding object moves, the moving background part will be mistaken for the object, and the object cannot be extracted correctly. [0010] As described above, in order to safely handle an unknown object, it is necessary to recognize the object being grasped. However, in the conventional technique, the object to be grasped is surely recognized in a general environment. It was difficult. This made it difficult to put into practical use robots with automatic manipulators that handle unknown objects, such as office and home service robots. In the case of operation-type manipulators such as construction machines, safety has to be ensured by the operator's attention, which has been a heavy burden on the operator.

Patent Document 1: Japanese Patent Application Laid-Open No. 2000-202790

Patent Document 2: Japanese Patent Laid-Open No. 2005-128959

Disclosure of the invention

Problems to be solved by the invention

[0012] The present invention has been made in view of the force and the point, and in the situation where the object being grasped and the surrounding object can move arbitrarily, the shape of the unknown grasped object can be reliably recognized and recognized. In order to solve the problem, an object is to perform a work based on the shape and to provide a manipulator capable of preventing the gripped object from coming into contact with surrounding objects and people and thereby improving safety. Means

[0013] The manipulator of the present invention includes an arm, arm driving means for driving the arm, a gripping portion provided on the arm, an image input means for acquiring a peripheral image of the gripping portion, and the image input Gripper relative position detection means for detecting the relative position of the gripper with respect to the means, a plurality of images acquired by the image input means, and the image input means detected by the gripper relative position detection means Storage means for storing the relative position of the gripping part, and based on the plurality of images stored in the storage means and the relative position of the gripping part to the image input means! It detects the position and shape of an object.

The invention's effect

[0014] According to the present invention, it is possible to recognize the shape of the gripping object of the manipulator, prevent the gripping object from coming into contact with surrounding objects, and improve the safety of the manipulator. Yes

Brief Description of Drawings

FIG. 1 is a schematic diagram showing an example of the overall configuration according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a system configuration example of a manipulator device according to an embodiment of the present invention.

FIG. 3 is a flowchart showing an overall processing example of contact possibility determination processing according to an embodiment of the present invention.

FIG. 4 is a flowchart showing a processing example of a gripping object position ′ shape determination process according to an embodiment of the present invention.

FIG. 5 is an explanatory diagram showing an example of a grayscale image according to an embodiment of the present invention.

FIG. 6 is an explanatory view showing an example of image processing in the gripping object position ′ shape determination processing according to the embodiment of the present invention.

FIG. 7 is a flowchart showing an example of a contact possibility determination process according to an embodiment of the present invention.

FIG. 8 is a schematic diagram showing an example of the overall configuration according to another embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

FIG. 1 shows an example of the overall configuration of the manipulator in the present embodiment, and FIG. 2 shows an example of the system configuration of the manipulator in the present embodiment. First, an overall configuration example of the present embodiment will be described with reference to FIG.

[0018] The manipulator 101 in this embodiment includes a base 105, a plurality of arms (arms) 103 and 104 that are directly connected to the base, and a grip portion 102 attached to the tip of the arm. In Fig. 1, a force with two arms may be provided. Between the grip and the arms 103, 104 and the base 105, there are movable joints 112, 113, 114. Each joint has actuators 132, 133, and 134 (shown in FIG. 1 overlaid on joints 112, 113, and 114) and joints to measure the amount of change in the movable part.

Angle sensors 122, 123, 124 that measure the angle of 112, 113, 114 (in Fig. 1, joint 112,

113 and 114). Where the joint has multiple rotational degrees of freedom In this case, the angle sensor acquires rotation angles in a plurality of directions. In addition, a control device 106 that inputs angle information acquired by the angle sensor and controls the actuator to move the manipulator 101 is provided. The control device 106 has various installation positions depending on the use and shape of the manipulator 101. FIG. 1 shows the control device 106 in the vicinity of the manipulator 101.

[0019] In general, the manipulator mainly controls the position of the gripping part by using an arm connected in series from the base, and rotates the joint between the arm at the tip and the gripping part, so that the posture of the gripping part is increased. To control. Here, the arm 103 in front of the grip portion 102 is referred to as the forearm 103 for comparison with the human structure. In addition, the joint 112 between the forearm 103 and the grip portion 102 is called a wrist joint 112. Even when the wrist joint 112 is rotated, the position of the tip of the grip portion 102 moves, but since the length of the grip portion is shorter than that of the arms 103 and 104, the amount of change in the position is relatively small. Note that the part corresponding to the wrist joint is composed of multiple joints, short arms, and arms! /, Which may be a force S. Here, these are collectively regarded as one wrist joint.

Next, a system configuration example of the manipulator 101 in the present embodiment will be described with reference to FIG. 1 and FIG. The manipulator 101 of the present embodiment mounted on the manipulator 101 includes an image input device 2 that is an image input unit that acquires a three-dimensional image of an object near the grip portion 102, and a surrounding monitoring unit that can detect the position of a surrounding object 108, Ambient monitoring device 3 is provided. In this embodiment, the image input device 2 is attached to the forearm 103, and the surroundings monitoring device 3 is provided on the base 105 of the manipulator 101 so as to acquire a wider range of surrounding images than the image input device 2. The image input device 2 may be provided on the arm 104 as long as it can acquire an image in the vicinity of the grip portion 102, but is preferably provided on the forearm (arm) 103 if the joint 113 can be inserted therebetween. Further, an angle sensor 122 that measures the angle of the wrist joint 112 is used as a gripper relative position detection unit that detects a change in the position of the gripper 102 relative to the image input device 2. The image storage unit 301 is a storage unit that stores the stereoscopic image data acquired by the image input device 2 and the angle information of the angle sensor 122 of the wrist joint 112 acquired from the control device 106! /, The In addition, the image of the gripping object 109 is extracted using the plurality of images stored in the image storage unit 301 and the relative position / posture information of the gripping unit, and the position and shape of the gripping object are detected. An image extraction unit 302, and based on the detected position and shape of the gripping object 109 and the position information of the surrounding object 108 detected by the surrounding monitoring device 3, the gripping object 109 and the surrounding object 108 A contact possibility determination device 4 which is a contact possibility determination means for determining the possibility of contact with the user, and an alarm device 5 which is an alarm means for notifying the possibility of contact by sound / image. Further, the contact possibility determination device 4 notifies the control device 106 of the possibility of contact, thereby restricting the operation of the manipulator and preventing contact. Here, in FIG. 1, the contact possibility determination device 4 and the alarm device 5 are illustrated separately from the manipulator 101 and the control device 6, but may be configured to be incorporated in the manipulator 101 and the control device 6.

The image input apparatus 2 according to the present embodiment has a function of acquiring a distance image including a grayscale image and distance information to each point of the image. The image input device 2 can be a stereo camera that measures the distance by image shift using two or more cameras, or a round trip until the laser beam hits an object and returns after scanning in one or two dimensions. A laser radar that measures distance according to time, a distance image camera equipped with an image sensor that can measure distance according to the round-trip time of light, and a combination of these devices with a normal two-dimensional CCD camera can be used. . A combination of a plurality of the above image input devices may be used. Similar to the image input device 2, the surroundings monitoring device 3 has a function of acquiring a distance image. As the warning device 5, a buzzer, a lamp, a display, etc. are used.

Here, the distance image is a term used in the field of image processing technology, and means data in which the value of the distance to the object is stored for each pixel of the image. A normal image (grayscale image) is a two-dimensional representation of the brightness of light in a number of directions divided into a grid pattern, that is, the intensity of light reflected by an object in that direction. On the other hand, the distance image stores the distance to the object in that direction instead of the brightness of the light for each point in the image. In other words, a distance image is two-dimensional array data that stores the distance to an object, and is generally used to store the output of a sensor that can obtain depth information such as a stereo camera.

The manipulator 101 has a function of recognizing the position and shape of the gripping object 109 when the gripping unit 102 is gripping the target gripping object 109. The following is the grip The description will be made on the assumption that the body 109 is held. As a method of gripping the gripping object 109, in the case of an automatic manipulator, for example, the target gripping object is recognized by recognizing the image using the shape feature based on the image information acquired by the image input device 2. The gripping object 109 is gripped by measuring the position of the easy-to-grip point in 109 and positioning the gripping part at that position. In the case of an operation type manipulator, the operator visually determines the position and shape of the gripping object 109 and moves the manipulator by manual operation to grip the gripping object 109.

[0024] The manipulator 101 in this embodiment moves the manipulator 101 in a state where the gripping object 109 is gripped, and the movement of the gripping object in the image near the gripping part 102 is compared with the gripping part 102 and the image input device 2 The gripping object 109 is predicted from the image by predicting the gripping object 109 based on the change in the relative position of the image, and extracting the part that moves the same as the prediction from the image of the image input device 2. Extract high.

Next, the operations of the image storage unit 301, the image extraction unit 302, and the contact possibility determination device 4 will be described with reference to FIG. FIG. 3 is a flowchart showing an overall outline of a processing example of the image storage unit 301, the image extraction unit 302, and the contact possibility determination device 4.

This process starts when the gripping object 109 is gripped by the gripping part 102 of the manipulator 101. First, stereoscopic image information around the grip portion 102, that is, a grayscale image 12 and a distance image 13 are acquired from the image input device 2, and at the same time, the wrist joint 112 measured by the angle sensor 122 through the control device 106 is obtained. The angle 14 is acquired and stored in the image storage unit 301 (step Sl). Information acquired before the movement of the manipulator 101 is referred to by the grayscale image 12a, the distance image 13a, and the wrist joint angle 14a in the subsequent processing. Next, in order to move the gripping object 109, the manipulator 101 starts moving and waits until the gripping part 102 moves a little (step S2). After the gripper 102 is moved, the gray scale image 12 and the distance image 13 around the gripper 102 are acquired from the image input device 2, and the angle 14 of the wrist joint 112 is acquired from the control device 106 and stored in the image storage unit 301. (Step S3). The information acquired after the movement is referred to by the grayscale image 12b, the distance image 13b, and the wrist joint angle 14b in the subsequent processing. Next, the position / shape of the grasped object 109 is determined by the image extraction unit 302 based on the information acquired in Step 1 and Step 3 (Step S4). next The contact possibility determination device 4 acquires the position information of the surrounding object from the surrounding monitoring device 3, and compares it with the position of the gripping object 109 determined in step S4. (Step S5). The result of the contact possibility determination process in step S5 is determined (step S6), and if there is a possibility of contact, an alarm is output by the alarm device 5 (step S7). As an alarm output method, for example, in the case of an automatic manipulator, the control device 106 is notified and the manipulator is stopped, and in the case of an operation type manipulator, the manipulator operation is performed by voice output or lighting of a warning lamp. It is possible to tell the person. As a result of the contact possibility determination process, if there is no possibility of contact, the process proceeds to the next process. Finally, it is determined whether or not the grip portion 102 of the manipulator 101 is gripping the gripped object 109 (step S8). If the object is being gripped, the grayscale image 12b, distance image 13b, Replace the information of angle 14b of wrist joint 112 with the information before the movement (step S9), return to the process of step S2, repeat the process, and continue to monitor the possibility of contact between the gripped object and surrounding objects . If it is determined in step S8 that the gripper 102 has not already gripped the object, the process ends.

[0027] Each of the above processes will be described in more detail. First, a processing example of the gripping object position / shape determination process (step S4) in the image extraction unit 302 will be described with reference to the flowchart of FIG.

[0028] In this process, the position 'shape of the grasped object 109 is determined based on the grayscale image, the distance image, and the wrist contact angle information acquired in step S1 and step S3. First, as shown in FIG. 6, the moved gray image 12b and distance image 13b (not shown) are divided into grid-like blocks (step S31). The size of the block is determined in advance. The smaller the block, the higher the position resolution, and the larger the block, the better the image matching accuracy described later. Usually, the block is set to about 5 X 5 pixels (pixels) to 25 X 25 pixels. In this embodiment, since the image input device 2 is attached to the forearm 103, the forearm 103 appears in the image as shown in the grayscale image example in FIG. Even if the grip portion 102 moves, the forearm 103 always appears in the same position on the image, so that portion (hatched portion in FIG. 5) is excluded from the subsequent processing targets. [0029] Next, one piece of block information of the divided gray image 12b after movement is extracted, and the following processing is performed as block B (i) (step S32). The subsequent image processing will be described with reference to the image processing example shown in FIG. First, the spatial position Pb (i) of the point Q (i) in the block B (i) is obtained (step S33). That is, the point of the object in the center of block B (i) is Q (i), and the position (two-dimensional coordinates) of the point Q (i) on the image is obtained and Rb (i). Then, based on the distance information of each pixel in the block B (i) of the distance image 13b after movement, the distance value of each pixel is averaged, and the point Q ( The relative distance of point Q (i) relative to the image input device 2 is obtained by backprojection transformation (J1 in Fig. 6).

Next, the spatial position Pa (i) of Q (i) before movement when it is assumed that the point Q (i) is fixed to the grip portion 102 is obtained (step S34). That is, assuming that the point Q (i) is a point on the grasped object 109 fixed to the grasping part 102, the angle 14 of the wrist joint 112 is changed from the wrist angle 14b after the current movement to the wrist angle 14a before the movement 14a. The three-dimensional spatial position (relative to image input device 2) when rotated to the position is obtained by coordinate transformation, and this is defined as Pa (i) (J2 in Fig. 6). Next, the position (two-dimensional coordinate) Ra (i) where the spatial position Pa (i) appears on the image of the image input device 2 is obtained by projection transformation (step S35) (J3 in FIG. 6). Note that the spatial position Pa (i) is also a relative position with respect to the image input device 2 in the same manner as P b (i).

[0031] Next, a partial image 21 having the same size as the block, centered on the image of the block B (i) on the gray image 12b after movement and the position Ra (i) on the gray image 12a before movement, is obtained. A comparison is made to determine whether the images match (step S36). If the images match as a result of the determination (step S37), a gripping object mark indicating that the object is on the gripping object 109 is attached to block B (i) (step S38).

[0032] Here, the position Ra (i) is assumed that the point Q (i) shown in the block B (i) on the grayscale image 12b after movement is a part of the grasped object 109. It is presumed that the point Q (i) appears in the gray image 12 a before moving. Therefore, if the point Q (i) is actually a part of the grasped object 109, the image of the block B (i) and the image of the partial image 21 should match. On the other hand, if Q (i) is different from the assumption, if the object is a background object rather than a part of the grasped object 109, the images do not match. This is because the manipulator 101 moves when Q (i) is a background object. Since the position of the image input device 2 attached to the forearm 103 changes with the operation, the point Q (i) appears at a different location from the position Ra (i) on the gray image 12a before movement. It is. Therefore, when the images match, it can be determined that the point Q (i) is a part of the grasped object 109.

Note that an image matching technique such as normalized correlation is used for determining image matching.

For example, the normalized correlation value between the image of the block B (i) on the gray image 12b after movement and the partial image 21 at the position Ra (i) on the gray image 12a before movement is obtained, and the value is determined in advance. When it is larger than the threshold! /, It is determined that the images match.

[0034] Finally, it is determined whether or not all blocks have been processed (step S39). If there is a block that has not been processed yet, the above steps S32 to S38 are repeated to process all blocks. When it is completed, this process ends. With the above processing, the 3D spatial position of the point on the grasped object can be detected, and the position 'shape of the grasped object can be known.

[0035] In the image extraction procedure described above, the manipulator is moved while the gripping object is gripped, and the gripping object motion in the image at that time is determined using the relative position of the gripping unit and the image input device measured by the sensor. By extracting the part that moves in the same way as the prediction from the image, only the grasped object is extracted from the image. In addition, when this method is changed, the relative position between the grip unit 102 and the image input device 2 is measured, and the image is captured so that the grip unit 102 shown in the image does not move before and after the movement of the manipulator 101. It is also a method of extracting only the gripped object 109 gripped by the gripping unit 102 from the images by comparing the deformed images and extracting the matching parts.

Next, a processing example of the contact possibility determination process (step S5) of FIG. 3 will be described with reference to the flowchart of FIG.

[0037] This process is performed by the contact possibility determination device 4. Compare the position of the point on the gripped object 109 extracted by the image extraction process described above with the position of the surrounding object obtained from the surrounding monitoring device 4, and if there is a close point, the possibility of contact It is determined that there is.

First, position information of surrounding objects is obtained from the surrounding monitoring device 3 (step S41). Next, among the blocks B (i) on the grayscale image 12b after the movement, the block with the gripping object mark is extracted (step S42). Block B with the extracted gripping object mark The spatial position Pb (i) corresponding to (i) is compared with the positions of surrounding objects acquired in step S41 (step S43). As a result, if the distance force is smaller than a predetermined threshold value, it is determined that the position is close (step S44), and if there is even one block that is close in position, the gripping object 109 and surrounding objects are detected. If there is a possibility of contact (step S45), the process is terminated. As a result of the determination in step S44, if it is determined that the position is not close, it is determined whether or not all blocks that have been gripped are marked and processed (step S46). When the block processing is completed, it is determined that there is no possibility of contact (step S47), and the processing is terminated. If there is a block that has not been processed yet, the above steps S42 to S44 are repeated.

As described above, in the manipulator 101 of the present embodiment, the gripping in the image is performed by image matching using the difference in the movement of the gripping object 109 and the surrounding object 108 on the image when the manipulator 101 is moved. Since the portion of the object 109 is detected, the shape 'position of the grasped object 109 can be reliably detected even when the background is complicated or there is a moving portion in the background. For this reason, even in a general usage environment, the possibility of approach between the gripped object 109 and surrounding objects is reliably determined, and if there is an object approaching the gripped object 109 using the determined result, the manipulator 101 is operated. Can be warned.

In the present embodiment, since the image input device 2 is attached to the forearm 103 of the manipulator, the grasped object 109 can always be grasped at the center of the visual field. For this reason, compared to the case where the image input device 2 is attached to the base 105 or the like, the grasped object 109 does not move much in the field of view (in the image), so that the grasped object 109 can be detected reliably and the viewing angle is limited Because it can do S, high resolution information can be obtained. Also, the distance force S between the image input device 2 and the gripping object 109 is moderately separated compared to the case where the image input device 2 is attached to the gripping part 102, so the overall shape of the gripping object 109 can be easily monitored. it can. On the other hand, since the image input device 2 is attached to the forearm 103, there is a problem that the image of the grasped object 109 moves within the field of view when the wrist joint 112 moves. The wrist joint angle obtained by the joint angle sensor 122 14 Using this information, image matching processing is performed in consideration of the movement of the grasped object 109 within the field of view! /, So the wrist joint 112 moves! / wear. In this embodiment, the force manipulator is used to determine the approaching state with the surrounding object 108 using the current position of the gripping object 109 detected based on the image information acquired from the image input device 2. From the motion of 101, it is possible to predict and calculate the position where the gripping object 109 will arrive within a certain period of time in the future, and to check the approaching state with the surrounding object 108 at that position. As a result, the possibility of contact between the gripped object 109 and the surrounding object 108 can be determined in advance and a warning can be issued earlier, so that the safety during operation of the manipulator 101 is increased.

In the present embodiment, the surroundings monitoring device 3 is provided separately from the image input device 2. However, instead of providing the surroundings monitoring device 3 as an independent device, the surrounding object 108 is connected to the image input device 2. By providing a function for acquiring an image, a configuration that also serves as a surrounding monitoring device may be used. Further, map information prepared in advance may be stored in the contact possibility determination device 4, and position information of surrounding objects may be acquired by referring to the map information.

[0043] Specifically, when the image input device 2 has a function of acquiring an image of the surrounding object 108, the image is extracted from the stereoscopic image input by the image input device 2 by the above method. The surrounding object 108 is extracted from the background excluding the gripped object 109, and the three-dimensional coordinates of the object are stored. When the background force excluding the extracted grasped object 109 is also extracted as the surrounding object 108, the position of the surrounding object 108 is not basically changed by the operation of the manipulator 101. Therefore, the image input device 2 and the surrounding monitoring device 3 It is only necessary to obtain a surrounding distance image.

In the present embodiment, a stereoscopic image sensor such as a stereo camera is used as the image input device 2, but instead, a monocular camera such as a CCD may be used as a simple configuration. In this case, since depth information cannot be detected, the gripping object 109 is detected and the possibility of contact with the surroundings is determined on the assumption that the gripping object 109 is at a certain distance.

Alternatively, the distance may be estimated from the image of the monocular camera by assuming that the gripping object 109 is on one virtual plane fixed to the gripping portion 102. In this case, the virtual surface is preferably a place where the probability that the gripping object 109 is high is preferably set to a surface that passes through the tip of the gripping portion 102 and is orthogonal to the gripping portion.

In the present embodiment, the image input device 2 can be attached to other places with the force attached to the forearm 103 of the manipulator. The image input device 2 was attached to the base 105. In step S34 in the flowchart of the shape determination process shown in Fig. 4, the spatial position Pa (i) before the movement of the point Q (i) can be obtained considering the angle change of joints other than the wrist. That's fine. In this case, there is a problem that the grasped object 109 moves greatly in the field of view. The position of the image input device 2 is fixed, so that the structure becomes simple and the image input device 2 also serves as the surrounding monitoring device 3. There is an advantage that you can. On the other hand, when the image input device 2 is attached to the gripping part 102, the gripping object 109 does not move within the field of view, so in step S34, the spatial position Pa (i ) Should be equal to the position Pb (i) after movement. In this case, the problem that the grasped object 109 is too close to the image input device 2 has an advantage that a certain force calculation process is simplified.

Furthermore, in this embodiment, an angle sensor that measures the angle of the wrist joint is used as the gripper relative position detection unit that detects the relative positional relationship between the image input device and the gripper. Alternatively, the relative position relationship may be obtained by detecting the position of the image input device and the gripping portion by other methods. In addition to the method of measuring the joint angle of the arm, the method of measuring the position / posture includes the method of obtaining the position / posture by photographing with a camera with the object to be measured outside.

[0048] Next, an embodiment in which the manipulator of the present embodiment is mounted on a construction machine is shown. Fig. 8 shows a configuration example when the manipulator of this embodiment is mounted on a work machine used for forestry or demolition work.

[0049] The work machine 201 according to the present embodiment includes a grapple 2002, which is a gripping part, an arm 203, and an arm 204 as manipulators. The work machine 201 is used for applications such as grabbing an object by a grappet nozzle 202 for disassembly and transportation.

[0050] The image input device 2 constituting the manipulator of the present embodiment is attached to the bottom surface of the arm 203 corresponding to the forearm. This location is suitable for capturing the object gripped by the grapple 202 because the positional relationship with the grapple 202 does not change greatly and is appropriately separated from the grapple 202. The surrounding monitoring device 3 is attached to the upper part of the cabin 209 to cover a wide field of view. An angle sensor 222 is attached to the joint 212 corresponding to the wrist as a wrist angle sensor. In this embodiment, the contact possibility determination device 4 and the alarm device 5 are illustrated to be installed inside the cabin 209. It may be installed in other places as long as it can communicate with each device that does not interfere with the operation of force S and manipulator.

[0051] In the present embodiment, the contact possibility determination device 4 uses the stereoscopic image around the grip nore 202 acquired by the image input device 2 and the angle of the joint 212 acquired from the angle sensor 222, so that the grapple 202 The gripping object is detected and compared with the position of the surrounding object detected by the surroundings monitoring device 3 to determine whether there is a possibility of contact. If there is a possibility of contact, an alarm device 5 informs the operator. There are methods for the alarm device 5 to notify the operator of the possibility of contact, such as giving vibration to the operation lever in addition to the notification method by voice or image. In addition, a series of image information related to processing in the contact possibility determination device 4 may be displayed by a display means, not shown.

Thus, by applying the manipulator of this embodiment, the burden on the operator can be reduced. Also, as in this example, in a complex work environment with many obstacles such as forestry, demolition work, construction work, etc., by warning in advance the possibility of contact between the grasped object and the surrounding object 108, Work safely and quickly.

As described above, according to the embodiment of the present invention, even when an unknown object is gripped in a complicated environment, the shape of the gripped object of the manipulator can be reliably recognized, and the gripping object is a surrounding object. This can increase the safety of the manipulator.

Claims

The scope of the claims

[1] An arm, an arm driving unit that drives the arm, a gripping unit provided in the arm, an image input unit that acquires a peripheral image of the gripping unit, and the gripping unit for the image input unit A relative position detecting means for detecting a relative position of the gripper, a plurality of images acquired by the image input means, and a relative position of the gripper with respect to the image input means detected by the gripper relative position detecting means. Based on a plurality of images stored in the storage unit and a relative position of the gripping unit with respect to the image input unit! /, Based on the position and shape of the gripping object. Mayupile to detect.

[2] The manipulator according to claim 1, wherein the surrounding monitoring means for detecting the position of the surrounding object, the detected position and shape of the gripping object, and the position of the surrounding object detected by the surrounding monitoring means Based on the contact possibility determination means for determining the possibility of contact between the gripped object and the surrounding object, and alarm means for issuing an alarm based on the determination result of the contact possibility determination means; A manipulator comprising:

[3] The manipulator according to claim 1 or 2, wherein the image input means is provided on an arm of the manipulator, and the grip portion relative position detection means is provided with the image input means. A manipulator characterized in that it is means for detecting the amount of change of a movable part provided between an arm and the grip part.

[4] The manipulator according to any one of claims 1 to 3, wherein the image input means is a stereoscopic image input means capable of acquiring a grayscale image and a distance image.

[5] The manipulator according to claim 2, wherein the contact possibility determination means determines a future position of the gripping object based on a change state of the relative position of the gripping part detected by the gripping part relative position detection means. A manipulator that predicts and determines the possibility of contact with the surrounding object.