+

WO2018163493A1 - Dispositif, procédé et programme de surveillance - Google Patents

Dispositif, procédé et programme de surveillance Download PDF

Info

Publication number
WO2018163493A1
WO2018163493A1 PCT/JP2017/036706 JP2017036706W WO2018163493A1 WO 2018163493 A1 WO2018163493 A1 WO 2018163493A1 JP 2017036706 W JP2017036706 W JP 2017036706W WO 2018163493 A1 WO2018163493 A1 WO 2018163493A1
Authority
WO
WIPO (PCT)
Prior art keywords
driver
steering wheel
monitoring
determination unit
video
Prior art date
Application number
PCT/JP2017/036706
Other languages
English (en)
Japanese (ja)
Inventor
相澤　知禎
初美 青位
岡地　一喜
啓 菅原
充恵 鵜野
光司 滝沢
Original Assignee
オムロン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オムロン株式会社 filed Critical オムロン株式会社
Publication of WO2018163493A1 publication Critical patent/WO2018163493A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present invention relates to a monitoring device, a method, and a program for monitoring the state of a driver sitting in a driver seat of a vehicle.
  • levels 0 to 4 are defined according to the degree of automation.
  • the vehicle of the automatic driving level 4 that is the highest level controls all of acceleration, steering, and braking including the emergency response on the vehicle side. It is assumed that a driver who rides a vehicle at an automatic driving level 4 does not need to be involved in driving, and thus pays attention to a book, a portable terminal, etc. even though he is sitting in the driver's seat. .
  • surveillance cameras are often installed near the meter panel on the back side of the steering wheel as seen from the driver in order to photograph the driver's facial expression from the front.
  • the photographing range of the monitoring camera is blocked by the support member of the steering wheel.
  • the shooting range of the surveillance camera is blocked.
  • the shooting range of the surveillance camera is obstructed by an obstacle, it is considered to make an announcement or the like to prompt the driver to remove the obstacle.
  • the obstacle includes not only an object held by the driver such as a book or a portable terminal, but also a steering wheel support member as described above. For this reason, it is not appropriate to make an announcement simply because the shooting range of the surveillance camera is shielded.
  • the present invention is intended to provide a monitoring device, method, and program capable of determining the cause when the shooting range of the monitoring camera is shielded.
  • a first aspect of the present invention is a monitoring device for monitoring a driver's state, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel when viewed from the driver.
  • a determination unit that determines whether the image acquired by the monitoring camera includes a support member for the steering wheel based on the steering angle of the steering wheel is provided.
  • the determination unit analyzes the video to determine whether the video includes a steering wheel support member based on the steering angle detected by the steering sensor. I have to. For this reason, the determination unit can determine whether or not the obstacle blocking the imaging range of the surveillance camera is a support member of the steering wheel without performing complicated image analysis.
  • the determination unit analyzes an image acquired by the monitoring camera, thereby determining whether or not an object other than the driver is included in the image. And whether the object determined to be included in the video by the image analysis unit is a support member of the steering wheel or other object, the angle width in which the steering angle is set in advance And a shielding determination unit for determining based on whether or not it is included.
  • the image analysis unit determines whether or not an obstacle is included in the video by analyzing the video.
  • the occlusion determination unit determines whether or not the steering angle of the steering wheel when an image determined to include an obstacle is included within a preset angle width.
  • the shielding determination unit determines that the obstacle included in the image is not a support member for the steering wheel. For this reason, the determination unit can accurately determine whether the object blocking the imaging range of the surveillance camera is a support member of the steering wheel or other obstacles without performing complicated image analysis. It becomes possible.
  • the determination unit further includes a concentration level determination unit that determines a concentration level based on the video, and the image analysis unit cannot determine the concentration level by the concentration level determination unit. Determining whether or not an object other than the driver is included in the video used for the determination of the concentration level, and the shielding determination unit acquires the video used for the determination of the concentration level Based on the steering angle of the steering wheel at the time, it is determined whether the object determined to be included in the video by the image analysis unit is a support member of the steering wheel or other object It is what I did.
  • the image analysis unit analyzes the image notified from the concentration degree determination unit, so that the image includes an obstacle. It is determined whether or not.
  • the shielding determining unit obtains the image when it is determined that the degree of concentration cannot be determined whether or not the obstacle determined to be included in the image is a support member of the steering wheel. The determination is made based on the steering angle of the wheel. For this reason, it becomes possible to reduce the process in a determination part.
  • a notification control unit that outputs a control signal for generating notification information that prompts removal is further provided.
  • the notification control unit removes the object when the shielding determination unit determines that the obstacle determined to be included in the video is not the steering wheel support member. A control signal for announcing a message prompting the user is output. For this reason, the determination unit can prompt the driver to remove the obstacle after determining whether the obstacle is a support member of the steering wheel.
  • each aspect of the present invention it is possible to provide a monitoring device, a method, and a program that can determine the cause when the photographing range of the monitoring camera is shielded.
  • FIG. 1 is a diagram showing an overall configuration of an automatic operation control system including an operation mode switching control device according to an embodiment of the present invention.
  • FIG. 2 is a block diagram showing a functional configuration of the operation mode switching control device shown in FIG.
  • FIG. 3 is a diagram showing the installation position of the driver camera shown in FIG.
  • FIG. 4 is a flowchart showing a procedure when a control unit provided in the operation mode switching control device shown in FIG. 2 notifies the driver to move an obstacle via the notification device.
  • FIG. 5 is a diagram illustrating an obstacle that blocks the imaging range of the driver camera assumed in the present embodiment.
  • FIG. 1 is a diagram showing an example of the overall configuration of an automatic driving control system provided with an operation mode switching control device according to an embodiment of the present invention.
  • the automatic driving control system is mounted on a vehicle 1 such as a passenger car.
  • the vehicle 1 includes, as basic equipment, a power unit 2 including a power source and a transmission, a steering device 3 equipped with a steering wheel 4, and further includes a manual operation mode and an automatic operation mode as operation modes. Yes.
  • An engine and / or a motor is used as the power source.
  • the manual driving mode is a mode in which the vehicle 1 is driven mainly by a driver's manual driving operation, for example.
  • the manual operation mode includes, for example, an operation mode for driving the vehicle based only on the driver's driving operation, and an operation mode for performing driving operation support control for supporting the driving operation of the driver while mainly driving the driver's driving operation. Is included.
  • the driving operation support control assists the steering torque so that the driver's steering becomes an appropriate steering amount based on the curvature of the curve when the vehicle 1 is traveling on the curve, for example.
  • the driving operation support control includes control for assisting a driver's accelerator operation (for example, operation of an accelerator pedal) or brake operation (for example, operation of a brake pedal), manual steering (manual operation of steering), and manual speed adjustment (speed). Adjustment manual operation) is also included.
  • a driver's accelerator operation for example, operation of an accelerator pedal
  • brake operation for example, operation of a brake pedal
  • manual steering manual operation of steering
  • speed manual speed adjustment
  • Adjustment manual operation is also included.
  • manual steering the vehicle 1 is steered mainly by the driver's operation of the steering wheel 4.
  • the speed of the vehicle is adjusted mainly by the driver's accelerator operation or brake operation.
  • the driving operation support control does not include control for forcibly intervening in the driving operation of the driver and automatically driving the vehicle.
  • the driving operation of the driver is reflected in the driving of the vehicle within a preset allowable range, but forcibly intervenes in the driving of the vehicle under certain conditions (for example, deviation from the lane of the vehicle). Control to do is not included.
  • the automatic driving mode is, for example, a mode that realizes an operating state in which the vehicle automatically travels along the road on which the vehicle travels.
  • the automatic driving mode includes, for example, a driving state in which the vehicle automatically travels toward a preset destination without driving by the driver.
  • the automatic driving mode it is not always necessary to automatically control all of the vehicle, and the driving state in which the driving operation of the driver is reflected in the driving of the vehicle within the preset allowable range is also included in the automatic driving mode. That is, the automatic driving mode includes control for forcibly intervening in driving of the vehicle under certain conditions, while reflecting the driving operation of the driver in driving of the vehicle within a preset allowable range.
  • the automatic operation control system shown in FIG. 1 has an automatic operation control device 5 for executing operation control in the automatic operation mode.
  • the automatic driving control device 5 acquires sensing data from the steering sensor 9, the accelerator pedal sensor 10, the brake pedal sensor 11, the GPS receiver 12, the gyro sensor 13, and the vehicle speed sensor 14, respectively. And these sensing data, route information generated by a navigation system (not shown), traffic information acquired by road-to-vehicle communication, and information obtained by a peripheral monitoring system that monitors the positions and movements of surrounding people and vehicles Based on the above, the traveling of the vehicle 1 is automatically controlled.
  • Automatic control includes, for example, automatic steering (automatic steering operation) and automatic speed adjustment (automatic driving of speed).
  • Automatic steering is an operating state in which the steering device 3 is automatically controlled.
  • Automatic steering includes Lane Keeping Assist (LKA).
  • LKA Lane Keeping Assist
  • the LKA automatically controls the steering device 3 so that the vehicle 1 does not deviate from the traveling lane even when the driver does not perform the steering operation.
  • the driver's steering operation may be reflected in the steering of the vehicle in a range where the vehicle 1 does not deviate from the travel lane (allowable range).
  • automatic steering is not limited to LKA.
  • Automatic speed adjustment is an operating state in which the speed of the vehicle 1 is automatically controlled.
  • Automatic speed adjustment includes Adaptive Cruise Control (ACC). For example, when there is no preceding vehicle ahead of the vehicle 1, ACC performs constant speed control that causes the vehicle 1 to travel at a constant speed at a preset speed, and when the preceding vehicle exists ahead of the vehicle 1. Is a follow-up control that adjusts the vehicle speed of the vehicle 1 in accordance with the inter-vehicle distance from the preceding vehicle.
  • the automatic operation control device 5 decelerates the vehicle 1 according to the driver's brake operation (for example, operation of the brake pedal) even when ACC is being executed.
  • the automatic operation control device 5 can perform the driver's accelerator operation (for example, accelerator) up to a preset maximum allowable speed (for example, the maximum speed legally determined on the traveling road) even during execution of ACC.
  • the vehicle can be accelerated according to the pedal operation.
  • the automatic speed adjustment is not limited to ACC, but also includes Cruise Control (CC) that performs only constant speed control.
  • CC Cruise Control
  • the automatic operation control system shown in FIG. 1 includes an operation mode switching control device 6, a driver camera 7, a notification device 8, and a device for switching settings between the manual operation mode and the automatic operation mode. And a steering sensor 9.
  • the driver camera 7 is disposed, for example, near the meter panel on the back side of the steering wheel as viewed from the driver.
  • the driver camera 7 acquires monitoring video data that is data acquired by photographing the driver.
  • the driver camera 7 outputs the acquired monitoring video data to the operation mode switching control device 6.
  • the notification device 8 is realized by a speaker or the like, for example, and notifies the driver of a message based on voice data output from the operation mode switching control device 6.
  • the steering sensor 9 detects the steering angle of the steering wheel 4.
  • the steering sensor 9 outputs steering angle data representing the detected steering angle to the driving mode switching control device 6.
  • FIG. 2 is a block diagram showing an example of a functional configuration of the operation mode switching control device 6 shown in FIG.
  • the operation mode switching control device 6 is a device that comprehensively controls the automatic operation control device 5 to switch between the manual operation mode and the automatic operation mode.
  • the operation mode switching control device 6 can also be called a monitoring device in this embodiment.
  • the operation mode switching control device 6 includes a control unit 61, an input / output interface unit 62, and a storage unit 63.
  • the input / output interface unit 62 receives the monitoring video data output from the driver camera 7 and the steering angle data output from the steering sensor 9.
  • the input / output interface unit 62 converts the received monitoring video data and steering angle data according to a predetermined method, and outputs the converted data to the control unit 61. Further, the input / output interface unit 62 outputs the audio data output from the control unit 61 to the notification device 8. Further, the input / output interface unit 62 outputs an operation mode switching control signal output from the control unit 61 to the automatic operation control device 5.
  • the storage unit 63 is a non-volatile memory that can be written and read as needed, such as Solid State Drive (SSD) and Hard Disk Drive (HDD), and volatile like Random Access Memory (RAM). This is realized by a memory or the like.
  • SSD Solid State Drive
  • HDD Hard Disk Drive
  • RAM Random Access Memory
  • the control unit 61 includes a central processing unit (CPU) that constitutes a computer, and a program memory. In the control unit 61, by causing the CPU to execute a program stored in the program memory, a monitoring video acquisition control unit 611, a concentration degree determination unit 612, which are control functions necessary for carrying out the present embodiment, An image analysis unit 613, a shielding determination unit 614, a notification control unit 615, and an operation mode switching control unit 616 are realized. In addition, the control unit 61 can be rephrased as a determination unit in the present embodiment.
  • the monitoring video acquisition control unit 611 has a function of acquiring video data acquired by the driver camera 7. For example, the monitoring video acquisition control unit 611 receives video monitoring data output from the input / output interface unit 62. The monitoring video acquisition control unit 611 stores the received monitoring video data in the monitoring video storage area 631 of the storage unit 63 at a predetermined sampling rate.
  • the concentration level determination unit 612 has a function of determining the concentration level based on the monitoring video data acquired by the monitoring video acquisition control unit 611. For example, the concentration degree determination unit 612 reads a preset number of monitoring video data from the monitoring video storage area 631 at a preset time interval. The concentration degree determination unit 612 recognizes the movement of the driver and / or the expression of the driver based on the read monitoring video data. The degree-of-concentration determination unit 612 collates the recognized driver movement and / or driver's facial expression with a predetermined criterion for determining the driver's movement and facial expression, and determines the driver's degree of concentration with respect to driving. judge. The concentration level determination unit 612 outputs information indicating the determination result of the concentration level to the operation mode switching control unit 616. Further, the concentration degree determination unit 612 stores information indicating the determination result of the concentration degree in the concentration degree storage area 632 with a time stamp.
  • the degree of concentration is an index indicating how suitable the driver is for driving.
  • the driver's movements are detected as forward gaze and gaze in a predetermined direction for viewing the rearview mirror or side mirror.
  • a state in which the detected driver's motion is almost occupied by forward gaze and gaze in a predetermined direction is considered that the driver is sufficiently concentrated on driving.
  • the driver is driving, he / she should not be looking closely ahead, talking, sleepy, carrying food in his mouth, or should not be mobile
  • the driver is not concentrated on driving.
  • the driver's facial expression for example, the eye open state of the driver's eyes, the frequency of blinking, or eye movements are detected.
  • the degree of concentration is defined as “high”.
  • 20% or more of the driver's movement is forward gaze or gaze in a predetermined direction, and the driver's movement includes operation of a mobile phone or smartphone and a state of a call.
  • the degree of concentration is defined as “low”.
  • the concentration degree determination unit 612 determines the movement of the driver recognized from the monitoring video data stored in the monitoring video storage area 631 with reference to the determination criterion.
  • the image analysis unit 613 has a function of analyzing the read monitoring video data and determining whether an object other than the driver is reflected in the monitoring video data. For example, the image analysis unit 613 reads monitoring video data from the monitoring video storage area 631. The image analysis unit 613 analyzes the read monitoring video data and determines whether or not an object other than the driver is reflected in the monitoring video data.
  • the shielding determination unit 614 has a function of determining whether or not the object determined to be reflected by the image analysis unit 613 is a support member for the steering wheel 4.
  • the notification control unit 615 receives steering angle data output from the input / output interface unit 62.
  • the notification control unit 615 is in a state where the driver camera 7 is shielded by the support member of the steering wheel 4 based on whether or not the steering angle represented by the received steering angle data is included in a preset shielding angle width. It is determined whether or not.
  • FIG. 3 is a schematic diagram showing an example of an installation position of the driver camera 7 shown in FIG.
  • the driver camera 7 images the driver from the back side of the steering wheel 4 toward the rear of the vehicle 1 when viewed from the driver. For this reason, depending on the steering angle of the steering wheel 4, the photographing range is blocked by the support members 41-1 to 41-3 of the steering wheel 4.
  • the shielding angle width represents the width of the steering angle at which the photographing range of the driver camera 7 is shielded by the support members 41-1 to 41-3.
  • the shielding angle width is set in advance based on, for example, the shape of the steering wheel 4 mounted on the vehicle 1.
  • the shielding determination unit 614 does not shield the imaging range of the driver camera 7 when the monitoring image data determined by the image analysis unit 613 that an object other than the driver is reflected is acquired. If it is determined that the object is determined to be included in the monitoring video data by the image analysis unit 613, it is determined that the object is not a support member of the steering wheel 4.
  • the notification control unit 615 has a function of controlling the notification device 8 so as to notify the driver that an obstacle placed in the shooting range of the driver camera 7 is moved.
  • the notification control unit 615 controls the notification device 8 so as to give an instruction to the driver to move the obstacle that is determined not to be the support member of the steering wheel 4 by the shielding determination unit 614.
  • the instruction at this time can be paraphrased as an instruction that prompts the driver's face to enter the shooting range of the driver camera 7.
  • the operation mode switching control unit 616 controls the automatic operation control device 5 so as to switch the setting between the manual operation mode and the automatic operation mode based on the concentration degree determined by the concentration degree determination unit 612.
  • the operation mode switching control unit 616 stores information indicating the switching control in the history storage area 633 with a time stamp.
  • FIG. 4 is a flowchart illustrating an example of a procedure when the control unit 61 provided in the operation mode switching control device 6 illustrated in FIG. 2 notifies the driver to move an obstacle via the notification device 8. .
  • the driver camera 7 and the steering sensor 9 are activated.
  • the driver camera 7 continuously captures a predetermined range including the driver's face.
  • the driver camera 7 outputs monitoring video data obtained by photographing the driver to the driving mode switching control device 6.
  • the driver camera 7 may intermittently photograph the driver at a cycle shorter than the time interval for determining the driver state.
  • the driver camera 7 or the input / output interface unit 62 may encode the video signal according to a predetermined encoding method. By doing so, it is possible to reduce the information amount of the monitoring video data and save the storage capacity of the monitoring video storage area 631.
  • the steering sensor 9 detects the steering angle of the steering wheel 4.
  • the steering sensor 9 outputs steering angle data representing the detected steering angle to the driving mode switching control device 6.
  • the image analysis unit 613 of the control unit 61 receives the monitoring video data output from the driver camera 7 via the input / output interface unit 62 (step S41). Further, the shielding determination unit 614 receives the steering angle data output from the steering sensor 9 via the input / output interface unit 62 (step S42).
  • the image analysis unit 613 analyzes the received monitoring video data (step S43). Any existing algorithm may be used as the algorithm for analyzing the monitoring video data.
  • the image analysis unit 613 uses a model fitting technique in which a three-dimensional face model composed of feature points of facial organs such as eyes, nose, and mouth is applied to the face portion in the monitoring image, and the position of the face, face The size of the face, the orientation of the face, and the position of each facial organ. Further, the image analysis unit 613 recognizes the position in the depth direction of the face (the optical axis direction of the camera) (that is, the distance between the driver camera 7 and the face) from the size of the face in the monitoring video data.
  • the image analysis unit 613 determines whether an object other than the driver is reflected in the monitoring video data based on the analysis result (step S44). For example, when the fitting of the three-dimensional face model is not performed on a part of the face in the monitoring video, the image analysis unit 613 determines that the shooting range of the driver camera 7 is blocked by an obstacle. In addition, when the driver is present in the driver's seat and the fitting of the three-dimensional face model is not performed on the entire face, the image analysis unit 613 has the shooting range of the driver camera 7 blocked by an obstacle. It is determined that
  • the image analysis unit 613 recognizes the driver's face by fitting a three-dimensional face model to a plurality of monitoring video data obtained in time series.
  • the image analysis unit 613 tracks the recognized face.
  • the image analysis unit 613 determines that the driver is present in the driver's seat.
  • the image analysis unit 613 determines that the driver has left the driver's seat.
  • the image analysis unit 613 When the face being tracked suddenly disappears due to a factor other than frame-out in the surveillance video, the image analysis unit 613 indicates that the driver is present in the driver's seat, but the shooting range of the driver camera 7 is obstructed by an obstacle. Determined to be blocked.
  • the image analysis unit 613 determines that the shooting range of the driver camera 7 is blocked by an obstacle (Yes in step S44)
  • the image analysis unit 613 generates an obstacle signal indicating that the shooting range of the driver camera 7 is blocked by the obstacle.
  • the data is output to the shielding determination unit 614.
  • the shielding determination unit 614 reads a preset shielding angle width from the storage unit 63 when the image analysis unit 613 generates a failure signal.
  • the shielding determination unit 614 determines whether or not the steering angle of the steering wheel 4 when the monitoring image including the obstacle is acquired is included in the shielding angle width (step S45).
  • the shielding determination unit 614 obtains the monitoring image including the obstacle when the steering wheel 4 is acquired.
  • the obstacle determined that the steering angle is included in the shielding angle width, that is, the image analysis unit 613 determines that it is included in the monitoring video data is determined to be the support member of the steering wheel 4 (step S46). ).
  • the shielding determination unit 614 performs steering when acquiring a monitoring image including an obstacle.
  • the obstacle determined that the steering angle of the wheel 4 is not included in the shielding angle width, that is, is included in the monitoring video data by the image analysis unit 613 is determined not to be a support member of the steering wheel 4. (Step S47).
  • the notification control unit 615 reads the audio data from the storage unit 63 (step S48). .
  • the voice data is, for example, as shown in FIG. 5 in a state in which an object that can be moved by the driver is placed on the steering wheel 4 or the direction in which the driver camera 7 captures the object that the driver has in his / her hand. It is created in advance assuming a state where is shielded.
  • FIG. 5 although the book is described as an example, it is not limited to a book.
  • the thing placed on the steering wheel 4 and the thing that the driver has in his hand may be any thing possessed by the driver, such as a smartphone or a tablet PC.
  • the voice data created in this way represents, for example, “Please move the shield.”, “Make sure the face faces the camera.”
  • the notification control unit 615 outputs, together with the read audio data, a control signal for reproducing the audio data to the notification device 8 (step S49).
  • the notification device 8 When the notification device 8 receives the voice data output from the notification control unit 615 and the control signal, the notification device 8 announces a message based on the voice data to the driver.
  • step S410 determines whether or not the operation is finished.
  • the control unit 61 ends the process when the operation ends (Yes in step S410).
  • the control unit 61 shifts the process to step S41, and repeats the operations from step S41 to step S49 until the driving of the vehicle 1 is finished.
  • step S42 If it is determined in step S42 that there is no obstacle, the control unit 61 proceeds to step S410.
  • the control unit 61 proceeds to step S410.
  • the control unit 61 analyzes the monitoring video data and the obstacle determined to be included in the monitoring video data is the support member of the steering wheel 4. Whether or not there is is determined based on the steering angle detected by the steering sensor 9. For this reason, the control unit 61 can determine whether or not the obstacle blocking the shooting range of the driver camera 7 is a support member of the steering wheel 4 without performing complicated image analysis.
  • the present embodiment it is possible to accurately determine whether the obstacle blocking the surveillance camera is an object possessed by the driver or a support member of the steering wheel. This makes it possible for the driver to accurately remove obstacles other than the steering wheel support member that are blocking the imaging range of the surveillance camera.For example, switching from the automatic operation mode to the manual operation mode is possible. In this case, it is possible to avoid a situation in which the driver's concentration cannot be determined because the shooting range of the driver camera 7 is blocked by an object owned by the driver.
  • the image analysis unit 613 determines whether or not an obstacle is included in the monitoring video by using a model fitting technique in which the three-dimensional face model is applied to the face portion in the monitoring video.
  • the shielding determination unit 614 determines whether or not the steering angle when the monitoring image determined to include an obstacle is included within a preset shielding angle width. When the steering angle is not included in the shielding angle width, the shielding determination unit 614 determines that the obstacle included in the monitoring image is not a support member for the steering wheel 4. Therefore, the control unit 61 accurately discriminates whether the object blocking the shooting range of the driver camera 7 is a support member of the steering wheel 4 or other obstacles without performing complicated image analysis. It becomes possible to do.
  • the notification control unit 615 determines that the obstacle determination unit 614 determines that the obstacle determined to be included in the monitoring video data is not the support member of the steering wheel 4.
  • a control signal for announcing a message prompting removal is output to the notification device 8. Therefore, the control unit 61 can prompt the driver to remove the obstacle after determining whether the obstacle is a support member of the steering wheel 4 or not.
  • control unit 61 performs the process shown in FIG. 4 when the operation is started has been described as an example.
  • the reason why the control unit 61 starts the process shown in FIG. 4 is not limited to when the operation is started.
  • the control unit 61 may perform the processing shown in FIG. 4 when the concentration level determination unit 612 cannot determine the concentration level.
  • the concentration degree determination unit 612 uses a predetermined number of pieces of monitoring video data among a predetermined number of pieces of monitoring video data used when determining the degree of concentration. If the driver's facial expression cannot be recognized, the degree of concentration cannot be determined. When the concentration level cannot be determined, the concentration level determination unit 612 notifies the image analysis unit 613 of the monitoring video data that caused the concentration level. The image analysis unit 613, the occlusion determination unit 614, and the notification control unit 615 perform the processing illustrated in FIG. 4 on the monitoring video data notified from the concentration degree determination unit 612. As a result, the process shown in FIG. 4 is performed only when the process is required, so that the process in the control unit 61 can be reduced.
  • the vehicle type, the function of the automatic operation control device 5, and the control function, control procedure, and control content of the operation mode switching control device 6 are variously modified and implemented without departing from the gist of the present invention. Is possible.
  • the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage.
  • various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.
  • a monitoring device is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver, and monitors the state of the driver,
  • a memory for storing a preset angle width for the steering angle of the steering wheel;
  • the at least one hardware processor comprises: By analyzing the video acquired by the monitoring camera, it is determined whether or not an object other than the driver is included in the video, Whether the steering angle of the steering wheel is included in a preset angular width, whether the object determined to be included in the video is a support member of the steering wheel or another object
  • a monitoring device that makes judgments based on

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Mathematical Physics (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Physics & Mathematics (AREA)
  • Traffic Control Systems (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

La présente invention présente la contre-mesure suivante, dans le but de permettre de discerner une cause de blocage lorsque la plage de capture d'une caméra de surveillance est bloquée. L'invention concerne une partie de détermination qui est dans un dispositif de surveillance, ledit dispositif de surveillance étant utilisé dans un véhicule dans lequel une caméra de surveillance est positionnée du côté arrière d'un volant, du point de vue du conducteur, et surveillant un état du conducteur, ladite partie de détermination déterminant, en fonction de l'angle d'orientation du volant, si une vidéo qui est acquise par la caméra de surveillance contient un élément de support du volant.
PCT/JP2017/036706 2017-03-09 2017-10-10 Dispositif, procédé et programme de surveillance WO2018163493A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017045215A JP2018151683A (ja) 2017-03-09 2017-03-09 監視装置、方法、及びプログラム
JP2017-045215 2017-03-09

Publications (1)

Publication Number Publication Date
WO2018163493A1 true WO2018163493A1 (fr) 2018-09-13

Family

ID=63447449

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/036706 WO2018163493A1 (fr) 2017-03-09 2017-10-10 Dispositif, procédé et programme de surveillance

Country Status (2)

Country Link
JP (1) JP2018151683A (fr)
WO (1) WO2018163493A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111152720A (zh) * 2018-11-08 2020-05-15 丰田自动车株式会社 车辆信息提供装置

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7595818B2 (ja) 2022-11-30 2024-12-06 三菱電機株式会社 情報処理装置、制御方法、及び制御プログラム

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004334786A (ja) * 2003-05-12 2004-11-25 Nissan Motor Co Ltd 状態検出装置及び状態検出システム
JP2008204107A (ja) * 2007-02-19 2008-09-04 Toyota Motor Corp 不注意警告装置、不注意警告装置の車両機器制御方法及び車両制御装置のプログラム
JP2008213823A (ja) * 2007-02-06 2008-09-18 Denso Corp 車両用制御装置
JP2009248777A (ja) * 2008-04-07 2009-10-29 Toyota Motor Corp 運転者状態判定装置
JP2010225089A (ja) * 2009-03-25 2010-10-07 Toyota Motor Corp 運転者監視装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004334786A (ja) * 2003-05-12 2004-11-25 Nissan Motor Co Ltd 状態検出装置及び状態検出システム
JP2008213823A (ja) * 2007-02-06 2008-09-18 Denso Corp 車両用制御装置
JP2008204107A (ja) * 2007-02-19 2008-09-04 Toyota Motor Corp 不注意警告装置、不注意警告装置の車両機器制御方法及び車両制御装置のプログラム
JP2009248777A (ja) * 2008-04-07 2009-10-29 Toyota Motor Corp 運転者状態判定装置
JP2010225089A (ja) * 2009-03-25 2010-10-07 Toyota Motor Corp 運転者監視装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111152720A (zh) * 2018-11-08 2020-05-15 丰田自动车株式会社 车辆信息提供装置
CN111152720B (zh) * 2018-11-08 2022-11-29 丰田自动车株式会社 车辆信息提供装置

Also Published As

Publication number Publication date
JP2018151683A (ja) 2018-09-27

Similar Documents

Publication Publication Date Title
JP7080598B2 (ja) 車両制御装置および車両制御方法
JP7006326B2 (ja) 自動運転システム
CN110192084B (zh) 自动驾驶辅助装置、方法及程序
JP6447647B2 (ja) 運転モード切替制御装置、方法およびプログラム
CN109844842B (zh) 驾驶模式切换控制装置、系统、方法以及存储介质
JP2018062308A (ja) 運転モード切替制御装置、方法およびプログラム
CN111699680B (zh) 行车记录仪、显示控制方法以及存储介质
CN109843690B (zh) 驾驶模式切换控制装置、系统以及方法
US20190210586A1 (en) Drive mode switch controller, method, and program
JP6593431B2 (ja) 記録制御装置、記録装置、記録制御方法及び記録制御プログラム
US20200017123A1 (en) Drive mode switch controller, method, and program
JP2005202876A (ja) 衝突可能性判定装置
JP6708785B2 (ja) 走行経路提供システムおよびその制御方法、並びにプログラム
JP6856086B2 (ja) 報知管理装置及び報知管理プログラム
JP2014160298A (ja) 運転支援システムおよび運転支援方法
WO2018168019A1 (fr) Dispositif de détermination de commutation de conduite, procédé de détermination de commutation de conduite, et programme de détermination de commutation de conduite
JP6627810B2 (ja) 運転モード切替制御装置、方法およびプログラム
WO2018163492A9 (fr) Dispositif, procédé et programme de commande de commutation de mode de conduite
WO2018163493A1 (fr) Dispositif, procédé et programme de surveillance
WO2018163474A1 (fr) Dispositif, procédé et programme de commande de commutation de mode de conduite
WO2022258150A1 (fr) Procédé mis en œuvre par ordinateur d'aide à un conducteur, produit-programme informatique, système d'aide à la conduite et véhicule
JP6835173B2 (ja) 記録制御装置、記録装置、記録制御方法及び記録制御プログラム
WO2018163454A1 (fr) Dispositif, procédé et programme de commande de commutation de mode de conduite
JP6648722B2 (ja) 故障判定装置、方法およびプログラム
JP2008305283A (ja) 運転支援装置および運転支援方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17900200

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17900200

Country of ref document: EP

Kind code of ref document: A1

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载