WO2015092977A1

WO2015092977A1 - Sight line detection device and sight line detection method

Info

Publication number: WO2015092977A1
Application number: PCT/JP2014/005903
Authority: WO
Inventors: 洋一成瀬
Original assignee: 株式会社デンソー
Priority date: 2013-12-18
Filing date: 2014-11-26
Publication date: 2015-06-25
Also published as: JP6322991B2; JP2015118579A

Abstract

Provided are a sight line detection device and a sight line detection method which are disposed upon a vehicle and whereby a sight line location whereto a driver's sight line reaches is detected. This sight line detection device comprises: a sight line detection unit (11a) which detects the sight line location of the driver; a correction unit (11b) which corrects the sight line location which is detected by the sight line detection unit (11a); and a peripheral environment detection unit (11c) which detects the brightness of the vehicle periphery. When a degree of change of the brightness of the vehicle periphery exceeds a prescribed degree, the correction unit (11b) corrects the sight line location.

Description

Gaze detection device and gaze detection method

Cross-reference of related applications

This application is based on Japanese Patent Application No. 2013-261878 filed on December 18, 2013, the contents of which are incorporated herein by reference.

The present disclosure relates to a technique for detecting a line-of-sight position where a driver's line of sight arrives.

A technique for detecting the driver's gaze position by analyzing the driver's face image is known. If this technique is used, it is possible to monitor the driver's aside driving. Furthermore, if the detection accuracy of the line-of-sight position is sufficiently high, various devices (such as a car navigation system) mounted on the vehicle can be operated with the line of sight (for example, Patent Document 1).

If it is possible to operate various devices with a line of sight, the driver does not need to reach out and operate the device while driving, so the driver's burden can be reduced. On the other hand, when the detection accuracy of the line-of-sight position decreases, there is a possibility that an operation different from the driver's intention may be performed. Therefore, for example, when a forward vehicle is detected during driving, a technique has been proposed that calibrates the line-of-sight position so that the detected line-of-sight position becomes the position of the vehicle ahead, assuming that the driver is gazing at the vehicle ahead. (Patent Document 2).

The inventors of the present application have found the following regarding the line-of-sight detection device and the line-of-sight detection method. With the proposed technique, even if the detection accuracy of the line-of-sight position is lowered, calibration cannot be performed when the preceding vehicle is not detected, and therefore there is a possibility that the detection accuracy of the line-of-sight position cannot be kept high.

Japanese published patent publication 2013-143012 Japanese Published Patent Publication No. 2009-015533

The present disclosure has been made in view of the above-described problems, and an object thereof is to provide a technique capable of keeping high eye-gaze position detection accuracy.

The line-of-sight detection device and the line-of-sight detection method according to an aspect of the present disclosure are detected by the line-of-sight detection unit when the brightness around the vehicle is detected and the amount of change in brightness around the vehicle exceeds a predetermined amount. Calibrate the gaze position. The gaze detection device includes a gaze detection unit that detects the gaze position of the driver, a calibration unit that calibrates the gaze position detected by the gaze detection unit, and a surrounding environment detection unit that detects brightness around the vehicle. . Further, the gaze detection method includes a step of detecting the gaze position of the driver, a step of calibrating the gaze position detected by the step of detecting the gaze position, and a step of detecting the brightness around the vehicle. .

It is considered that the detection accuracy of the line-of-sight position is likely to decrease as the amount of change in brightness around the vehicle increases. Therefore, if the brightness around the vehicle is detected, and the amount of change in the brightness around the vehicle exceeds a predetermined amount, the line-of-sight position detected by the line-of-sight detection unit is calibrated. Calibration can be performed at the timing when the accuracy is lowered to increase the detection accuracy.

According to the line-of-sight detection device and the line-of-sight detection method according to the present disclosure, the detection accuracy of the line-of-sight position can be kept high.

The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. In the attached drawings
FIG. 1A is an explanatory diagram illustrating the configuration of the line-of-sight detection device according to the present embodiment. FIG. 1B is a block diagram illustrating functions of the CPU. FIG. 2 is a flowchart of a gaze position detection process performed by the gaze detection device of the present embodiment. FIG. 3A is an explanatory diagram illustrating a pupil and a corneal reflection image in an image obtained by imaging the right eye of the driver, FIG. 3B is an explanatory diagram illustrating a pupil and a corneal reflection image in an image obtained by imaging the right eye of the driver; FIG. 3C is an explanatory diagram illustrating a pupil and a cornea reflection image captured in an image obtained by imaging the right eye of the driver; FIG. 4 is an explanatory diagram showing a method of detecting the relative pupil position, FIG. 5 is an explanatory diagram illustrating a line-of-sight position determination table. FIG. 6 is a flowchart of the calibration process performed by the line-of-sight detection apparatus according to the present embodiment. FIG. 7 is an explanatory diagram illustrating a gaze image displayed to the driver. FIG. 8 is an explanatory diagram conceptually showing how the line-of-sight position determination table is calibrated. FIG. 9 is an explanatory view showing another example of a gaze image displayed to the driver, FIG. 10 is an explanatory diagram showing the relationship between the brightness around the vehicle and the detection accuracy of the line-of-sight position in the present embodiment.

Hereinafter, examples will be described in order to clarify the contents of the disclosure of the present application described above. The line-of-sight detection apparatus according to the present embodiment is an apparatus that is mounted on a vehicle and detects a line-of-sight position where the line of sight of the driver of the vehicle reaches.
(Device configuration)
FIG. 1A shows the configuration of the line-of-sight detection device 10 of the present embodiment. As shown in the figure, the line-of-sight detection device 10 is connected to a flash memory 12 that stores a program executed by the CPU 11 and a RAM 13 that is a work area of the CPU 11 via a bus 14. ing. The CPU 11 reads out various programs and data necessary for executing the programs from the flash memory 12 and performs processing for detecting the line-of-sight position.

Various methods can be adopted as a method for detecting the line-of-sight position. In this embodiment, the line-of-sight position is detected based on the positional relationship between the cornea reflection image captured in the driver's eye image and the pupil. We will adopt the method to do.

Corresponding to adopting such a method, the bus 14 includes a near-infrared light camera 16 for imaging using near-infrared light, and a near-infrared light irradiation LED (near-infrared light LED). 17 is connected via the camera interface 15. The near-infrared light camera 16 and the near-infrared light LED 17 are provided with the driver's seat side facing the front of the driver's seat in order to image the driver. The CPU 11 controls the near-infrared light camera 16 and the near-infrared light LED 17 to acquire an image of the driver's eyes imaged in a state where the near-infrared light is irradiated.

Further, an illuminance sensor 19 is connected to the bus 14 via an illuminance sensor interface 18, and the CPU 11 detects the brightness around the vehicle using the illuminance sensor 19. Further, a HUD device (head-up display device) 21 is connected to the bus 14 via a HUD device interface 20. The HUD device 21 can project a predetermined image on the windshield (hereinafter “windshield”) of the vehicle based on an instruction from the CPU 11.

FIG. 1B is a block diagram showing functions of the CPU 11. These functions are realized by the CPU 11 executing a program stored in the flash memory 12. As shown in FIG. 1B, the CPU 11 includes a gaze position detection function 11a that performs a gaze position detection process, a calibration function 11b that calibrates the gaze position detected by the detection process, and a vehicle periphery, which will be described in detail later. A surrounding environment detection function 11c for detecting the brightness of the vehicle, and an image display function 11d for displaying a predetermined image to the driver.

The line-of-sight position detection function 11a according to the present embodiment corresponds to the “line-of-sight detection unit” of the present disclosure, and the calibration function 11b according to the present embodiment corresponds to the “calibration unit” of the present disclosure. The environment detection function 11c corresponds to the “peripheral environment detection unit” of the present disclosure, and the image display function 11d of the present embodiment corresponds to the “image display unit” of the present disclosure.
(Gaze position detection processing)
FIG. 2 shows a flowchart of the eye gaze position detection process executed by the CPU 11 of this embodiment. In the line-of-sight position detection process, a process for detecting the driver's line-of-sight position is performed. This process is started when a predetermined condition is satisfied (for example, when a vehicle engine is started or when a predetermined start operation is performed by the driver).

When starting the line-of-sight position detection process, the CPU 11 first controls the near-infrared light camera 16 and the near-infrared light LED 17 to image the eyes of the driver while irradiating near-infrared light. The obtained eye image is stored in a predetermined address of the RAM 13 (S100).

Subsequently, the CPU 11 detects the center of the “pupil” and the center of the “corneal reflection image” from the eye image captured in the process of S100 using a known method such as edge detection (S102). The “corneal reflection image” is a light reflected from the near-infrared light LED 17 reflected by the cornea (so-called Purkinje image), and is brightly reflected in the cornea.

FIG. 3A, FIG. 3B, and FIG. 3C exemplify “pupil” and “corneal reflection image” in an image obtained by imaging the right eye of the driver. Among these, FIG. 3A shows an image captured when the line of sight is directed to the front. In the illustrated example, a “corneal reflection image” is shown directly below the “pupil”. The CPU 11 detects the centers of the “pupil” and “corneal reflection image” in the eye image in this way.

Here, the positional relationship between the “pupil” in the eye image and the “corneal reflection image” changes according to the line-of-sight position. For example, roughly speaking, as illustrated in FIG. 3A, when the line of sight is directed to the front, even if the pupil is positioned “directly above” the corneal reflection image, As illustrated in FIG. 3B, the pupil is positioned on the “upper right” of the cornea reflection image. When the line of sight is directed to the right side, as illustrated in FIG. 3C, the pupil is positioned at the “upper left” of the cornea reflection image.

Since the positional relationship between the “corneal reflection image” and the “pupil” changes in accordance with the line-of-sight position in this way, the line-of-sight position is detected based on “the position of the center of the pupil relative to the center of the cornea reflection image”. Can do.

Therefore, when the CPU 11 detects the center of the “pupil” and the “corneal reflection image” in the process of S102, subsequently, “the position of the center of the pupil relative to the center of the cornea reflection image” (hereinafter, “pupil relative position”). Is detected (S104). As shown in FIG. 4, the relative pupil position is represented by the distance in the horizontal direction (x direction) and the distance in the vertical direction (y direction) from the center of the cornea reflection image to the center of the pupil. These distances are represented by the number of pixels.

Thus, when the pupil relative position is detected (S104), the CPU 11 refers to the gaze position determination table and detects the gaze position of the driver (S106). In the line-of-sight position determination table, as illustrated in FIG. 5, the line-of-sight positions (bx, by) of the driver are set for all values (ax, ay) that the relative position of the pupil can take. The CPU 11 refers to this line-of-sight position determination table and detects the line-of-sight position corresponding to the pupil relative position obtained by the process of S104 described above.

The line-of-sight position determination table is generated as follows before starting the line-of-sight position detection process, such as after the engine of the vehicle is started, and stored in the flash memory 12 in advance. First, the driver's eyes are imaged while the driver is gazing at a predetermined position (for example, the image displayed by the HUD device 21 at the predetermined position), and the pupil is captured in the same manner as described above. Detect relative position. Then, the gaze position (line-of-sight position) and the obtained pupil relative position are stored in association with each other. Such processing is repeated a plurality of times (for example, 5 to 10 times) while changing the position to be watched. In addition, for a position (gaze position) that is not gazed, the pupil relative position is calculated by performing known interpolation processing, extrapolation processing, or the like. Thereby, a correspondence relationship between the “pupil relative position” and the “line-of-sight position”, that is, a line-of-sight position determination table is generated.

As described above, the line-of-sight detection apparatus 10 of the present embodiment detects the line-of-sight position based on the pupil relative position obtained from the eye image. The CPU 11 outputs data relating to the line-of-sight position detected in this way to the device in response to requests from various driving support devices (not shown). Thereby, various driving assistances using the driver's line-of-sight position are performed.

For example, when the driver's line-of-sight position is greatly deviated from the traveling direction of the vehicle detected by the steering angle sensor (not shown), the driver is warned to prevent the driver from looking aside. Alternatively, a device such as a car navigation system is operated by the movement of the driver's line of sight. In this case, the physical burden can be reduced as compared with the case of operating with a finger, and the device can be operated even when the hand is closed.

As described above, after performing a series of processes for detecting the line-of-sight position (S100 to S106), the CPU 11 performs a calibration process (S200 in FIG. 2). In this calibration process, when it is estimated that the detection accuracy of the gaze position has been lowered, the above-described gaze position determination table is calibrated in order to increase the detection accuracy of the gaze position from the next time.
(Calibration process)
FIG. 6 shows a flowchart of the calibration process performed by the CPU 11. When starting the calibration process, the CPU 11 detects the brightness around the vehicle using the illuminance sensor 19 (S202). Then, it is determined whether or not the amount of change in brightness around the vehicle since the previous calibration exceeded a predetermined amount (S204). As will be described in detail later, in the visual line detection device 10 of the present embodiment, the brightness around the vehicle is stored in the RAM 13 every time the visual line position determination table is calibrated. Therefore, in the process of S204, the brightness stored in the RAM 13 when the previous calibration was performed is compared with the brightness detected this time, and it is determined whether or not the difference between these brightnesses exceeds a predetermined amount. (S204). Note that examples of the predetermined amount include several hundred lx to several thousand lx.

Here, when the brightness around the vehicle changes, the detection accuracy of the line-of-sight position may decrease for the following reason. That is, when the brightness around the vehicle changes, the brightness of light entering the eyes of the driver in the passenger compartment also changes, so the pupil diameter of the driver's eyes changes. Then, even if the actual line-of-sight position does not change, the “pupil relative position” obtained from the eye image may change. Since the “pupil relative position” is associated with the line-of-sight position corresponding to the “pupil relative position”, the corresponding line-of-sight position is naturally different if the pupil relative position is different. As described above, when the brightness around the vehicle changes, the “pupil relative position” obtained from the eye image changes even if the actual line-of-sight position does not change, and is thus detected based on the “pupil relative position”. The sight line position may be shifted from the actual sight line position.

Therefore, if it is determined in the determination process of S204 described above that the amount of change in brightness around the vehicle has exceeded a predetermined amount (S204: yes), it can be estimated that the detection accuracy of the line-of-sight position has decreased. Therefore, when the CPU 11 determines that the amount of change in brightness around the vehicle has exceeded a predetermined amount (S204: yes), the brightness around the vehicle detected in the processing of S202 is stored in the RAM 13 as "brightness at the time of current calibration". After storing (S206), the gaze position determination table is calibrated as follows in order to increase the gaze position detection accuracy.

First, the CPU 11 outputs a predetermined control signal to the HUD device 21 to display the “gazing image 30a” in front of the driver (S208). FIG. 7 illustrates a state in which the gaze image 30a is displayed on the windshield in front of the driver. FIG. 7 shows a state when the front of the vehicle is viewed from the driver's seat, and a rectangular broken line represents an image display area on the windshield by the HUD device 21. In the illustrated example, the gaze image 30a is displayed in the lower right part of the image display area. Based on the control signal from the CPU 11, the HUD device 21 displays the gaze image 30a in a manner that stands out from the image that is normally displayed (for example, blinking or increasing the brightness). When the gaze image 30a is displayed in front of the driver in this manner, it is considered that the driver driving while looking at the front gazes at the gaze image 30a.

Therefore, the CPU 11 detects the pupil relative position when the driver is gazing at the gaze image 30a, that is, the pupil relative position when the gaze image 30a is displayed (S210). The detection of the pupil relative position is performed by the same method as the visual line position detection process described above with reference to FIG. That is, the driver's eyes are imaged in the state of irradiating near infrared light, and the pupil relative position is detected based on the pupil and the center of the cornea reflection image detected from the eye image.

Thus, when the “pupil relative position” when the gaze image 30a is displayed is detected (S210), the CPU 11 obtains the position where the gaze image 30a is displayed and the “pupil” obtained when the gaze image 30a is displayed. Based on the “relative position”, the line-of-sight position determination table is calibrated (S212). In other words, the “pupil relative position” obtained this time can be estimated to be the “pupil relative position” in a state where the driver is gazing at the gaze image 30a, and thus the “pupil relative position” is detected. In this case, the line-of-sight position corresponding to the “pupil relative position” in the line-of-sight position determination table is set to “the position where the line-of-sight image 30a is displayed” so that the “position where the line-of-sight image 30a is displayed” is detected as the line-of-sight position. To the line-of-sight position corresponding to “”.

For example, as shown in FIG. 8, the value of the “relative pupil position” obtained this time (in other words, the “relative pupil position obtained when the gaze image 30a is displayed) is (120, 250). In this case, the line-of-sight position corresponding to the pupil relative position (120, 250) is updated to the line-of-sight position (bx80, by80) corresponding to the “position where the gaze image 30a is displayed”.

Thus, when the line-of-sight position corresponding to the “pupil relative position” obtained when the gaze image 30a is displayed is updated, the line-of-sight positions corresponding to the other “pupil relative positions” are also known. It is updated by performing interpolation processing and extrapolation processing. By calibrating the line-of-sight position determination table in this manner, the deviation between the line-of-sight position detected in the line-of-sight position detection process and the actual line-of-sight position is eliminated, and the line-of-sight position detection accuracy is improved.

In this embodiment, the accuracy of detecting the eye gaze position can be improved by calibrating the eye gaze position determination table. Therefore, in the present embodiment, “calibrating the line-of-sight position determination table” corresponds to “calibrating the line-of-sight position” in the present disclosure.

Note that, instead of calibrating the line-of-sight position determination table as described above, the line-of-sight position determination table may be regenerated. That is, as shown in FIG. 9, the process of displaying the gaze image 30a described in the process of S208 is repeated a plurality of times while changing the display position. In the example shown in FIG. 9, the gaze images 30b to 30f are sequentially displayed at five locations (four corners and a central portion) of the image display area of the HUD device 21. Since the driver is considered to gaze at the sequentially displayed gaze images 30b to 30f, the CPU 11 detects the pupil relative position from the eye image captured when the gaze images 30b to 30f are displayed. Then, the display positions (line-of-sight positions) of the images for gaze 30b to 30f and the detected pupil relative positions are stored in association with each other. Further, for positions (line-of-sight positions) where the gazing images 30b to 30f are not displayed, pupil relative positions are calculated by performing known interpolation processing, extrapolation processing, and the like.

Thus, when the line-of-sight position determination table is regenerated, the deviation between the line-of-sight position detected by the line-of-sight position detection process and the actual line-of-sight position is surely eliminated, and the line-of-sight position detection accuracy can be reliably increased. When the line-of-sight position determination table is regenerated as described above, “re-generation of the line-of-sight position determination table” corresponds to “calibrate the line-of-sight position” in the present disclosure.

As described above, the line-of-sight detection device 10 according to the present embodiment displays the gaze image 30a to the driver when the line-of-sight position determination table is calibrated (or regenerated). The process of displaying the gaze image 30a for the driver can be performed at any time using the HUD device 21. Further, it is considered that the driver who drives while looking forward looks at the gaze image 30a displayed to the driver (displayed in front of the driver). For these reasons, the driver's “actual line-of-sight position” can be acquired at a desired timing by the process of displaying the gaze image 30a to the driver.

Therefore, as described above, in the line-of-sight detection device 10 of the present embodiment, the gaze image 30a is displayed to the driver, the position where the gaze image 30a is displayed, and the gaze position detection process (similar to the same). The line-of-sight position determination table is calibrated based on the pupil relative position detected in (Method). In this way, the driver's “actual gaze position” is acquired without missing the timing when the amount of change in brightness around the vehicle exceeds the predetermined amount, and the “actual gaze position” is detected by the gaze position detection process. The line-of-sight position determination table can be calibrated to be detected by As a result, the detection accuracy of the line-of-sight position can be reliably kept high.

In this embodiment, the gaze position determination table is calibrated based on the position where the gaze image 30a is displayed and the “pupil relative position” detected by the gaze position detection process (similar method). The “pupil relative position” corresponds to the “line-of-sight position” detected by the line-of-sight position detection process. Therefore, in the present embodiment, “calibrating the gaze position determination table based on the position where the gaze image 30a is displayed and the“ pupil relative position ”detected by the gaze position detection process (similar method)” This corresponds to “calibrating the line-of-sight position based on the position where the gaze image is displayed and the“ line-of-sight position ”detected by the line-of-sight detection unit” in the present disclosure.

In addition, a driver who drives while looking forward looks at the gaze image 30a displayed to the driver (displayed in front of the driver) unconsciously without greatly moving his line of sight. Conceivable. For this reason, since a driver | operator's driving | operation is not prevented, the calibration of a gaze position determination table can be performed safely.

Further, in the above description, the gaze image 30a is displayed in order to cause the driver to gaze at a predetermined position. However, instead of using a dedicated image like the gaze image 30a, the HUD device 21 is used. The image displayed normally may be used. In particular, a warning image for warning the driver (for example, a warning image for warning that an obstacle has been detected ahead or a warning image for displaying a road sign to call attention) is generally used by the driver. A display mode that attracts the attention of is adopted. For this reason, the warning image can make the driver gaze at a predetermined position in the same manner as the gaze image 30a.

Therefore, the CPU 11 may calibrate the line-of-sight position determination table based on the display position of the warning image when the amount of change in brightness around the vehicle exceeds a predetermined amount. As described above, if the warning image displayed normally is used instead of the dedicated image (gaze image 30a), the image displayed to the driver does not change at all. For this reason, the line-of-sight position determination table can be calibrated more safely without disturbing the driving of the driver.

Further, as described above, in the line-of-sight detection device 10 according to the present embodiment, it is estimated that the detection accuracy of the line-of-sight position has decreased when the amount of change in brightness around the vehicle exceeds a predetermined amount, and the line-of-sight position determination table It was decided to calibrate. In the following, this point will be supplementarily described.

FIG. 10 conceptually shows the relationship between “brightness around the vehicle” and “detection accuracy of the line-of-sight position” in this embodiment. The upper side in FIG. 10 shows the brightness around the vehicle, and the lower side in FIG. 10 shows the detection accuracy of the line-of-sight position.

When the vehicle is traveling during the daytime, as shown at time t1 in FIG. 10, the brightness around the vehicle may be greatly reduced by entering the tunnel. As a result, when the brightness of light entering the eyes of the driver in the passenger compartment decreases, the relative position of the pupil obtained from the eye image changes even if the line-of-sight position does not actually change. The detection accuracy may be reduced. Therefore, if the CPU 11 determines that the amount of change in brightness around the vehicle has exceeded a predetermined amount (decreased beyond a predetermined amount), the CPU 11 calibrates the line-of-sight position determination table. Thereby, as shown at time t2 in FIG. 10, the detection accuracy of the line-of-sight position can be improved.

Then, as shown at time t3 in FIG. 10, when the vehicle goes outside the tunnel, the brightness around the vehicle returns to the original brightness, and the brightness of the light entering the driver's eyes is also based on the brightness. Return. Then, even if the line-of-sight position does not actually change, the pupil relative position changes, and the line-of-sight position detection accuracy may decrease. Therefore, when the CPU 11 determines that the amount of change in brightness around the vehicle has exceeded a predetermined amount (increased beyond the predetermined amount), the CPU 11 calibrates the line-of-sight position determination table. Thereby, as shown at time t4 in FIG. 10, the detection accuracy of the line-of-sight position can be increased.

As described above, if the gaze position determination table is calibrated when the amount of change in brightness around the vehicle exceeds a predetermined amount, calibration is performed at the timing when the gaze position detection accuracy is reduced, and the detection accuracy is increased. Can be increased. As a result, the detection accuracy of the line-of-sight position can be kept high.

Further, when the vehicle is traveling at sunset, the brightness around the vehicle may gradually decrease as shown from time t5 to time t6, t7, t8 in FIG. In this case, for example, it may not be possible to appropriately determine whether or not the amount of change in brightness around the vehicle exceeds a predetermined amount only by the amount of change in brightness per unit time.

Therefore, the line-of-sight detection device 10 (CPU 11) of the present embodiment stores the brightness around the vehicle when the line-of-sight position determination table is calibrated in the RAM 13, and the line-of-sight position determination table is calibrated by detecting the vehicle periphery. And the brightness stored in the RAM 13 is greater than a predetermined amount.

In this way, even if the brightness around the vehicle gradually changes, the stored brightness (that is, the brightness at the time of previous calibration) and the brightness detected this time are compared. Therefore, it is possible to appropriately determine whether or not the amount of change in brightness around the vehicle exceeds a predetermined amount. As a result, even when the gaze position detection accuracy gradually decreases due to the gradual change in brightness around the vehicle, calibration is performed before the detection accuracy greatly decreases from the state that was increased by the previous calibration. Thus, the detection accuracy of the line-of-sight position can be kept high (see times t6, t7, and t8 in FIG. 10). The RAM 13 in this embodiment corresponds to a “storage unit” in the present disclosure.

As described above, in this embodiment, as a method for detecting the line-of-sight position, the method using the positional relationship between the cornea reflection image reflected in the driver's eye image and the pupil has been described. However, the method for detecting the line-of-sight position is not limited to this method. That is, in various methods for detecting the line-of-sight position, it is considered that the detection accuracy of the line-of-sight position is likely to decrease when the amount of change in brightness around the vehicle increases. Therefore, in various methods for detecting the line-of-sight position, when the amount of change in the brightness around the vehicle exceeds a predetermined amount, if the processing performed in the line-of-sight position detection is calibrated, it is the same as the above-described embodiment. The effect of can be obtained. It should be noted that the calibration of the processing performed when detecting the line-of-sight position is performed by a method corresponding to the detection method. For example, when the line-of-sight position is detected based on a predetermined relational expression instead of the line-of-sight position determination table, the relational expression is calibrated.

In the present embodiment, the gaze position determination table is taken as an example to calibrate the “processing performed when the gaze position is detected”, but of course, the “gaze line” that is a result obtained by detection is used. The “position” may be calibrated. That is, in the present disclosure, “calibrate the line-of-sight position” refers to whether to “calibrate the process performed when detecting the line-of-sight position” or to calibrate the “line-of-sight position” obtained as a result of detection. Regardless of whether or not, as a result, the calibration is performed so that the detection accuracy of the line-of-sight position is improved. In addition, as a method of calibrating the “sight line position” which is a result obtained by detection, based on the position where the gaze image 30a is displayed (that is, the actual line-of-sight position) and the detected line-of-sight position. A method of offsetting the line-of-sight position so that the line-of-sight position becomes the “actual line-of-sight position” can be exemplified.

According to one aspect of the present disclosure, the visual line detection device and the visual line detection method detect brightness around the vehicle and detect the brightness around the vehicle when the amount of change in brightness around the vehicle exceeds a predetermined amount. The sight line position is calibrated.

It is considered that the detection accuracy of the line-of-sight position is likely to decrease as the amount of change in brightness around the vehicle increases. Therefore, if the brightness around the vehicle is detected, and the amount of change in the brightness around the vehicle exceeds a predetermined amount, the line-of-sight position detected by the line-of-sight detection unit is calibrated. Calibration can be performed at the timing when the accuracy is lowered to increase the detection accuracy. As a result, the detection accuracy of the line-of-sight position can be kept high.

Further, in the gaze detection device of the present disclosure, the following may be performed. The brightness around the vehicle when the line-of-sight position is calibrated is stored, and the line-of-sight position calibration is performed when the difference between the detected brightness around the vehicle and the stored brightness exceeds a predetermined amount. Do.

The brightness around the vehicle may change not only suddenly when entering and exiting a tunnel but also gradually changing over time. Even in such a case, if the brightness around the vehicle when the line-of-sight position is calibrated is memorized, and the brightness around the vehicle changes from the brightness at that time by a predetermined amount or more, the calibration is performed. The line-of-sight position can be calibrated at the required timing. As a result, the detection accuracy of the line-of-sight position can be kept high.

Further, in the gaze detection device of the present disclosure, the following may be performed. The gaze image is displayed to the driver, and the gaze position is calibrated based on the position where the gaze image is displayed and the gaze position detected by the gaze detection unit.

If the gaze image is displayed, it can be considered that the driver moves the line of sight to the gaze image. Therefore, if the gaze image is displayed, the line-of-sight position can be calibrated at the required timing. As a result, the detection accuracy of the line-of-sight position can be reliably kept high.

Here, the flowchart described in this application or the processing of the flowchart is configured by a plurality of steps (or referred to as sections), and each step is expressed as S100, for example. Further, each step can be divided into a plurality of sub-steps, while a plurality of steps can be combined into one step. Further, each step configured in this way can be referred to as a device, module, or means.

In addition, each of the plurality of steps described above or a combination thereof includes not only (i) a software step combined with a hardware unit (eg, a computer) but also (ii) hardware (eg, an integrated circuit, As a step of the wiring logic circuit), it can be realized with or without including the functions of related devices. Further, the hardware steps can be configured inside the microcomputer.

The embodiments, configurations, and aspects of the present disclosure have been illustrated above, but the embodiments, configurations, and aspects according to the present disclosure are not limited to the above-described embodiments, configurations, and aspects. For example, embodiments, configurations, and aspects obtained by appropriately combining technical units disclosed in different embodiments, configurations, and aspects are also included in the scope of the embodiments, configurations, and aspects according to the present disclosure.

Claims

A gaze detection device that is provided in a vehicle and detects a gaze position where a driver's gaze arrives,
A line-of-sight detector (11a) for detecting the line-of-sight position of the driver;
A calibration unit (11b) for calibrating the line-of-sight position detected by the line-of-sight detection unit (11a);
A surrounding environment detection unit (11c) for detecting brightness around the vehicle,
The said calibrating part (11b) is a gaze detection apparatus which calibrates the said gaze position, when the variation | change_quantity of the brightness around the said vehicle exceeds predetermined amount.
The line-of-sight detection device according to claim 1,
A storage unit (13) for storing brightness around the vehicle when the line-of-sight position is calibrated;
In the calibration unit (11b), the difference between the brightness around the vehicle detected by the surrounding environment detection unit (11c) and the brightness stored in the storage unit (13) is larger than the predetermined amount. A line-of-sight detection device for calibrating the line-of-sight position.
The line-of-sight detection device according to claim 1 or 2,
An image display unit (11d) for displaying a gaze image for the driver;
The calibration unit (11b) calibrates the line-of-sight position based on the position where the image display unit (11d) displays the gaze image and the line-of-sight position detected by the line-of-sight detection unit (11a). Gaze detection device.
A line-of-sight detection method that is provided in a vehicle and detects a line-of-sight position that a driver's line of sight reaches
Detecting the driver's line-of-sight position (S106);
Calibrating the line-of-sight position detected by the step of detecting the line-of-sight position (S212);
Detecting the brightness around the vehicle (S202),
The step of calibrating the line-of-sight position (S106) is a line-of-sight detection method for calibrating the line-of-sight position when the amount of change in brightness around the vehicle exceeds a predetermined amount.