+

WO2018163492A9 - Dispositif, procédé et programme de commande de commutation de mode de conduite - Google Patents

Dispositif, procédé et programme de commande de commutation de mode de conduite Download PDF

Info

Publication number
WO2018163492A9
WO2018163492A9 PCT/JP2017/036705 JP2017036705W WO2018163492A9 WO 2018163492 A9 WO2018163492 A9 WO 2018163492A9 JP 2017036705 W JP2017036705 W JP 2017036705W WO 2018163492 A9 WO2018163492 A9 WO 2018163492A9
Authority
WO
WIPO (PCT)
Prior art keywords
concentration
monitoring video
shielding
driver
monitoring
Prior art date
Application number
PCT/JP2017/036705
Other languages
English (en)
Japanese (ja)
Other versions
WO2018163492A1 (fr
Inventor
相澤　知禎
初美 青位
岡地　一喜
啓 菅原
充恵 鵜野
光司 滝沢
Original Assignee
オムロン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オムロン株式会社 filed Critical オムロン株式会社
Publication of WO2018163492A1 publication Critical patent/WO2018163492A1/fr
Publication of WO2018163492A9 publication Critical patent/WO2018163492A9/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00—Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/04—Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present invention relates to an operation mode switching control device, method and program for switching a vehicle operation mode between a manual operation mode and an automatic operation mode.
  • the automatic driving mode includes, for example, information on a navigation system using Global Positioning System (GPS), traffic information acquired by road-to-vehicle communication, and information on a surrounding monitoring system that monitors the positions and movements of surrounding people and vehicles.
  • GPS Global Positioning System
  • the vehicle can be automatically driven by controlling the power unit, the steering device, the brake, and the like (see, for example, Japanese Patent Laid-Open No. 2015-141053).
  • the automatic driving mode can be expected to reduce the burden on the driving operation of the driver and alleviate the traffic congestion, it is practically difficult to automate all the driving operations from the start of travel to the end of travel. . For this reason, it is required to appropriately switch from the automatic operation mode to the manual operation mode.
  • the operation mode is switched from the automatic operation mode to the manual operation mode, it is necessary to confirm that the driver is in a state where manual operation can be performed prior to the switching.
  • a monitoring camera that monitors the driver's condition may be placed near the meter panel on the back of the steering wheel as viewed from the driver.
  • the shooting range of the camera is blocked by the support member of the steering wheel, and the driver's state monitoring cannot be continued. If the driver's condition cannot be confirmed, the driving operation after switching to the manual operation mode cannot be guaranteed, and the driver's concentration is high.
  • the automatic operation mode can be switched to the manual operation mode. Even in the state, the automatic operation mode is continued. In view of the above, there has been a demand for appropriate measures when the photographing range of the surveillance camera is blocked by the support member of the steering wheel.
  • This invention is intended to provide an operation mode switching control device, method and program capable of determining the state of the driver even when the photographing range of the surveillance camera is blocked by the support member of the steering wheel.
  • a first aspect according to the first embodiment of the present invention is a vehicle driving mode used in a vehicle in which a surveillance camera is installed on the back side of a steering wheel as viewed from a driver.
  • An operation mode switching control apparatus that performs switching control includes a control unit that determines a concentration degree of the driver based on a monitoring image acquired by the monitoring camera. The control unit determines whether or not the shooting range of the monitoring camera is shielded by the steering wheel support member based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel. Is determined, and the concentration degree determined immediately before entering the shielding state is defined as the concentration degree in the shielding state.
  • the control unit determines whether the photographing range of the monitoring camera is shielded by the steering wheel support member based on the steering angle detected by the steering sensor. Determine whether. Then, when the imaging range of the monitoring camera is shielded by the support member of the steering wheel, the control unit determines the concentration determined at the timing immediately before the shielding as the concentration level when the imaging range of the monitoring camera is shielded. The degree is adopted. For this reason, even when it is desired to use the degree of concentration at the timing when the photographing range of the surveillance camera is shielded by the support member of the steering wheel, for example, whether to switch from automatic driving to manual driving, etc. Can be carried out.
  • the control unit acquires monitoring video data at a predetermined cycle from the monitoring video acquired by the monitoring camera, and includes the acquired monitoring video data
  • a monitoring image acquisition control unit that determines monitoring image data acquired in the shielding state based on a steering angle detected by the steering sensor, and a concentration level in the shielding state is determined immediately before the shielding state is entered.
  • a concentration degree determination unit for determining based on the monitoring video data acquired during the shielding period.
  • the monitoring video acquisition control unit is configured to shield the photographing range of the monitoring camera by the steering wheel support member based on the steering angle detected by the steering sensor.
  • the monitoring video acquired at the time is determined.
  • the concentration level determination unit determines the concentration degree determined at the timing immediately before the shooting range of the monitoring camera is blocked. Is adopted as the degree of concentration while the screen is shielded. For this reason, the monitoring video acquisition control unit can efficiently sort the monitoring video acquired when the imaging range of the monitoring camera is blocked.
  • the control unit acquires a monitoring video data at a predetermined cycle from a monitoring video acquired by the monitoring camera, and the monitoring Among the monitoring video data acquired by the video acquisition control unit, the monitoring video data acquired in the shielding state is determined based on the steering angle detected by the steering sensor, and the degree of concentration in the shielding state is determined by A concentration degree determination unit for determining based on the monitoring video data acquired in the non-shielding period immediately before the shielding state.
  • the concentration degree determination unit uses the monitoring video data acquired when the monitoring camera is shielded by the steering wheel based on the steering angle detected by the steering sensor. Determine.
  • the concentration degree determination unit indicates the degree of concentration determined at the timing immediately before the shooting range of the monitoring camera is blocked. Is adopted as the degree of concentration while the screen is shielded. For this reason, the concentration level determination unit can select the monitoring video data acquired when the imaging range of the monitoring camera is shielded by the steering wheel, and the processing of the monitoring video acquisition control unit can be reduced.
  • the control unit acquires a monitoring video data at a predetermined cycle from a monitoring video acquired by the monitoring camera, and the acquisition A degree-of-concentration determination unit that determines the degree of concentration of the driver based on the monitored video data, and determines the shielding state based on a steering angle detected by the steering sensor, and immediately before entering the shielding state, An operation mode switching control unit that determines whether or not the operation mode can be switched is used as the concentration level determined by the concentration level determination unit as the concentration level in the shielding state.
  • the monitoring video acquisition control unit acquires monitoring video data from the monitoring video acquired by the monitoring camera at a predetermined cycle.
  • the concentration level determination unit determines the driver's concentration level based on the acquired monitoring video data.
  • the operation mode switching control unit determines the timing at which the photographing range of the monitoring camera is shielded by the steering wheel support member based on the steering angle detected by the steering sensor. Then, the operation mode switching control unit adopts the concentration determined immediately before entering the shielding state as the concentration when determining whether the operation mode can be switched in the shielding state. For this reason, it is possible to reduce the processing in the monitoring video acquisition control unit and the concentration degree determination unit.
  • a first aspect according to the second embodiment of the present invention is an operation mode switching control for switching a vehicle operation mode, which is used in a vehicle in which a surveillance camera is installed on the back side of a steering wheel when viewed from the driver.
  • the apparatus includes a control unit that determines the degree of concentration of the driver based on a monitoring image acquired by the monitoring camera.
  • the control unit predicts a shielding period in which the imaging range of the monitoring camera is shielded by the steering wheel support member based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel.
  • the degree of concentration in the predicted shielding period is determined based on the monitoring video acquired in the non-shielding period before the shielding period.
  • the control unit determines the period during which the imaging range of the monitoring camera is shielded by the steering wheel support member based on the steering angle detected by the steering sensor. Predict. Then, the control unit determines the concentration of the driver in the predicted period based on the monitoring video data acquired before the shielding period. For this reason, even if it is desired to use the degree of concentration at the timing when the shooting range of the surveillance camera is predicted to be shielded by the support member of the steering wheel, for example, whether or not switching from automatic driving to manual driving is possible. It is possible to determine whether or not switching is possible using the degree of concentration acquired immediately before it is predicted to be in the shielding state.
  • the control unit predicts the shielding period based on a steering angle detected by the steering sensor, and is preset from the start point of the shielding period.
  • a shielding prediction unit that outputs an operation signal at a time point just before the period, a monitoring image acquisition control unit that acquires monitoring image data from a monitoring image acquired by the monitoring camera at a timing according to the operation signal, and the operation
  • a concentration level determination unit that determines the concentration level of the driver based on the acquired monitoring video data at a timing according to a signal and sets the concentration level as the concentration level in the predicted shielding period. It is a thing.
  • the shielding prediction unit predicts the shielding period based on the steering angle detected by the steering sensor, and is preset from the predicted start time of the shielding period.
  • An operation signal is output when the previous time is reached.
  • the monitoring video acquisition control unit stores monitoring video data from the monitoring video acquired by the monitoring camera at the timing of receiving the operation signal.
  • the concentration level determination unit determines the concentration level based on the monitoring video data stored by the monitoring video acquisition control unit at the timing when the operation signal is received, and the concentration level is determined as the concentration level in the predicted shielding period. Like to do.
  • the operation mode can be switched at a timing at which the imaging range of the surveillance camera is predicted to be shielded by the support member of the steering wheel, it is set in advance immediately before it is predicted to enter the shielding state It is possible to use the degree of concentration acquired at a timing different from the sampling rate.
  • the control unit predicts the shielding period based on a steering angle detected by the steering sensor, and is preset from a start time of the shielding period.
  • a shielding prediction unit that outputs an operation signal at a time point just before the period, a monitoring video acquisition control unit that acquires monitoring video data in a predetermined cycle from the monitoring video acquired by the monitoring camera, and the acquired monitoring video data
  • the concentration level determined immediately before receiving the operation signal is set as the concentration level in the predicted shielding period.
  • a determination unit is used to determine the concentration level in the predicted shielding period.
  • the shielding prediction unit predicts the shielding period based on the steering angle detected by the steering sensor, and presets it from the start time of the predicted shielding period.
  • An operation signal is output at a time point before the period.
  • the monitoring video acquisition control unit samples monitoring video data from the monitoring video acquired by the monitoring camera according to a preset sampling rate.
  • the concentration level determination unit determines the concentration level of the driver at a preset cycle based on the monitoring video data sampled by the monitoring video acquisition control unit. Then, when receiving the operation signal, the concentration determination unit substitutes the concentration determined in the predicted shielding period with the concentration determined immediately before receiving the operation signal. For this reason, it is possible to quickly calculate the degree of concentration at the timing when the operation signal is received.
  • the control unit when the control unit receives the operation signal, the control unit determines whether or not the operation mode can be switched based on the concentration level determined by the concentration level determination unit.
  • the operation mode switching control unit is further provided with the determination result as the determination result in the shielding period.
  • the operation mode switching control unit when the operation mode switching control unit receives the operation signal output from the shielding prediction unit, the operation mode switching control unit represents the concentration level determined by the concentration level determination unit. Get information. The operation mode switching control unit determines whether or not the operation mode can be switched based on the acquired information. The operation mode switching control unit sets the determination result as the determination result in the predicted shielding period. For this reason, when the operation mode switching instruction is given when the surveillance camera is shielded by the steering wheel, the operation mode switching control unit can use the determination result determined immediately before entering the shielding state. Become. That is, it is possible to prepare for an operation mode switching instruction given when the surveillance camera is shielded by the steering wheel.
  • an operation mode switching control device, method, and program capable of determining the state of the driver even when the photographing range of the monitoring camera is blocked by the support member of the steering wheel. it can.
  • FIG. 1 is a diagram showing an example of the overall configuration of an automatic operation control system including an operation mode switching control device according to the first embodiment of the present invention.
  • FIG. 2 is a block diagram showing a functional configuration of the operation mode switching control apparatus according to the first embodiment of the present invention.
  • FIG. 3 is a diagram showing the installation position of the driver camera shown in FIG.
  • FIG. 4 is a flowchart showing a procedure when the monitoring video acquisition control unit shown in FIG. 2 stores the monitoring video data.
  • FIG. 5 is a flowchart illustrating a procedure when the concentration level determination unit illustrated in FIG. 2 determines the concentration level of the driver.
  • FIG. 6 is a block diagram showing a functional configuration of an operation mode switching control apparatus according to the second embodiment of the present invention.
  • FIG. 1 is a diagram showing an example of the overall configuration of an automatic operation control system including an operation mode switching control device according to the first embodiment of the present invention.
  • FIG. 2 is a block diagram showing a functional configuration of the operation mode switching control apparatus according to
  • FIG. 7 is a flowchart illustrating a procedure when the shielding prediction unit illustrated in FIG. 6 outputs an operation signal.
  • FIG. 8 is a flowchart showing a procedure when the monitoring video acquisition control unit shown in FIG. 6 stores the monitoring video data.
  • FIG. 9 is a flowchart illustrating a procedure when the concentration level determination unit illustrated in FIG. 6 determines the concentration level of the driver.
  • FIG. 1 is a diagram showing an example of the overall configuration of an automatic driving control system provided with an operating mode switching control device according to a first embodiment of the present invention.
  • the automatic driving control system is mounted on a vehicle 1 such as a passenger car.
  • the vehicle 1 includes, as basic equipment, a power unit 2 including a power source and a transmission, a steering device 3 equipped with a steering wheel 4, and further includes a manual operation mode and an automatic operation mode as operation modes. Yes.
  • An engine and / or a motor is used as the power source.
  • the manual driving mode is a mode in which the vehicle 1 is driven mainly by a driver's manual driving operation, for example.
  • the manual operation mode includes, for example, an operation mode for driving the vehicle based only on the driver's driving operation, and an operation mode for performing driving operation support control for supporting the driving operation of the driver while mainly driving the driver's driving operation. Is included.
  • the driving operation support control assists the steering torque so that the driver's steering becomes an appropriate steering amount based on the curvature of the curve when the vehicle 1 is traveling on the curve, for example.
  • the driving operation support control includes control for assisting a driver's accelerator operation (for example, operation of an accelerator pedal) or brake operation (for example, operation of a brake pedal), manual steering (manual operation of steering), and manual speed adjustment (speed). Adjustment manual operation) is also included.
  • a driver's accelerator operation for example, operation of an accelerator pedal
  • brake operation for example, operation of a brake pedal
  • manual steering manual operation of steering
  • speed manual speed adjustment
  • Adjustment manual operation is also included.
  • manual steering the vehicle 1 is steered mainly by the driver's operation of the steering wheel 4.
  • the speed of the vehicle is adjusted mainly by the driver's accelerator operation or brake operation.
  • the driving operation support control does not include control for forcibly intervening in the driving operation of the driver and automatically driving the vehicle.
  • the driving operation of the driver is reflected in the driving of the vehicle within a preset allowable range, but forcibly intervenes in the driving of the vehicle under certain conditions (for example, deviation from the lane of the vehicle). Control to do is not included.
  • the automatic driving mode is, for example, a mode that realizes an operating state in which the vehicle automatically travels along the road on which the vehicle travels.
  • the automatic driving mode includes, for example, a driving state in which the vehicle automatically travels toward a preset destination without driving by the driver.
  • the automatic driving mode it is not always necessary to automatically control all of the vehicle, and the driving state in which the driving operation of the driver is reflected in the driving of the vehicle within the preset allowable range is also included in the automatic driving mode. That is, the automatic driving mode includes control for forcibly intervening in driving of the vehicle under certain conditions, while reflecting the driving operation of the driver in driving of the vehicle within a preset allowable range.
  • the automatic operation control system shown in FIG. 1 has an automatic operation control device 5 for executing operation control in the automatic operation mode.
  • the automatic driving control device 5 acquires sensing data from the steering sensor 8, the accelerator pedal sensor 9, the brake pedal sensor 10, the GPS receiver 11, the gyro sensor 12, and the vehicle speed sensor 13, respectively. And these sensing data, route information generated by a navigation system (not shown), traffic information acquired by road-to-vehicle communication, and information obtained by a peripheral monitoring system that monitors the positions and movements of surrounding people and vehicles Based on the above, the traveling of the vehicle 1 is automatically controlled.
  • Automatic control includes, for example, automatic steering (automatic steering operation) and automatic speed adjustment (automatic driving of speed).
  • Automatic steering is an operating state in which the steering device 3 is automatically controlled.
  • Automatic steering includes Lane Keeping Assist (LKA).
  • LKA Lane Keeping Assist
  • the LKA automatically controls the steering device 3 so that the vehicle 1 does not deviate from the traveling lane even when the driver does not perform the steering operation.
  • the driver's steering operation may be reflected in the steering of the vehicle in a range where the vehicle 1 does not deviate from the travel lane (allowable range).
  • automatic steering is not limited to LKA.
  • Automatic speed adjustment is an operating state in which the speed of the vehicle 1 is automatically controlled.
  • Automatic speed adjustment includes Adaptive Cruise Control (ACC). For example, when there is no preceding vehicle ahead of the vehicle 1, ACC performs constant speed control that causes the vehicle 1 to travel at a constant speed at a preset speed, and when the preceding vehicle exists ahead of the vehicle 1. Is a follow-up control that adjusts the vehicle speed of the vehicle 1 in accordance with the inter-vehicle distance from the preceding vehicle.
  • the automatic operation control device 5 decelerates the vehicle 1 according to the driver's brake operation (for example, operation of the brake pedal) even when ACC is being executed.
  • the automatic operation control device 5 can perform the driver's accelerator operation (for example, accelerator) up to a preset maximum allowable speed (for example, the maximum speed legally determined on the traveling road) even during execution of ACC.
  • the vehicle can be accelerated according to the pedal operation.
  • the automatic speed adjustment is not limited to ACC, but also includes Cruise Control (CC) that performs only constant speed control.
  • CC Cruise Control
  • the automatic operation control system shown in FIG. 1 includes an operation mode switching control device 6, a driver camera 7, and a steering sensor 8 as devices for switching the setting between the manual operation mode and the automatic operation mode. I have.
  • the driver camera 7 is disposed, for example, near the meter panel on the back side of the steering wheel as viewed from the driver.
  • the driver camera 7 captures the driver and obtains a monitoring video about the driver.
  • the driver camera 7 outputs the acquired monitoring video as a video signal to the operation mode switching control device 6.
  • the steering sensor 8 detects the steering angle of the steering wheel 4.
  • the steering sensor 8 outputs the detected steering angle to the operation mode switching control device 6 as a steering angle detection signal.
  • FIG. 2 is a block diagram illustrating an example of a functional configuration of the operation mode switching control device 6 according to the first embodiment.
  • the operation mode switching control device 6 shown in FIG. 2 is a device that comprehensively controls the automatic operation control device 5 to switch between the manual operation mode and the automatic operation mode.
  • the operation mode switching control device 6 includes a control unit 61, an input / output interface unit 62, and a storage unit 63.
  • the input / output interface unit 62 receives the video signal output from the driver camera 7 and the steering angle detection signal output from the steering sensor 8.
  • the input / output interface unit converts the received video signal and steering angle detection signal into digital data. Further, the input / output interface unit 62 outputs an operation mode switching control signal output from the control unit 61 to the automatic operation control device 5.
  • the storage unit 63 is a non-volatile memory that can be written and read as needed, such as Solid State Drive (SSD) and Hard Disk Drive (HDD), and volatile like Random Access Memory (RAM). This is realized by a memory or the like.
  • SSD Solid State Drive
  • HDD Hard Disk Drive
  • RAM Random Access Memory
  • the control unit 61 includes a central processing unit (CPU) that constitutes a computer, and a program memory.
  • the control unit 61 causes the CPU to execute a program stored in the program memory so that a monitoring video acquisition control unit 611 and a concentration degree determination unit 612 are control functions necessary for implementing the first embodiment.
  • the operation mode switching control part 613 is implement
  • the monitoring video acquisition control unit 611 takes in digital data (monitoring video data) about the driver video signal output from the driver camera 7 from the input / output interface unit 62.
  • the monitoring video acquisition control unit 611 stores the captured monitoring video data in the monitoring video storage area 631 of the storage unit 63 at a predetermined sampling rate.
  • the monitoring image acquisition control unit 611 takes in digital data (steering angle data) about the steering angle detection signal output from the steering sensor 8 from the input / output interface unit 62.
  • the monitoring image acquisition control unit 611 determines whether or not the driver camera 7 is in the shielding state based on whether or not the steering angle represented by the captured steering angle data is included in the shielding angle width.
  • FIG. 3 is a schematic diagram showing an example of an installation position of the driver camera 7 shown in FIG.
  • the driver camera 7 images the driver from the back side of the steering wheel 4 as viewed from the driver. For this reason, depending on the steering angle of the steering wheel 4, the photographing range is blocked by the support members 41-1 to 41-3 of the steering wheel 4.
  • the shielding angle width represents the width of the steering angle at which the photographing range of the driver camera 7 is shielded by the support members 41-1 to 41-3.
  • the shielding angle width is set in advance based on, for example, the shape of the steering wheel 4 mounted on the vehicle 1.
  • the monitoring image acquisition control unit 611 determines that the steering angle is included in the shielding angle width
  • the monitoring image acquisition control unit 611 attaches a shielding flag to the monitoring image data sampled while the steering angle is included in the shielding angle width. And stored in the surveillance video storage area 631.
  • the shielding flag is a flag representing monitoring video data sampled while the steering angle is included within the shielding angle width.
  • the concentration degree determination unit 612 reads the monitoring video data stored during this time interval from the monitoring video storage area 631 at a preset time interval.
  • the concentration degree determination unit 612 recognizes the movement of the driver and / or the expression of the driver based on the read monitoring video data.
  • the degree-of-concentration determination unit 612 collates the recognized driver movement and / or driver's facial expression with a predetermined criterion for determining the driver's movement and facial expression, and determines the driver's degree of concentration with respect to driving. judge.
  • the concentration level determination unit 612 outputs the determination result of the concentration level to the operation mode switching control unit 613. Further, the concentration degree determination unit 612 stores the determination result of the concentration degree in the concentration degree storage area 632 with a time stamp.
  • the degree of concentration is an index indicating how suitable the driver is for driving.
  • the driver's movements are detected as forward gaze and gaze in a predetermined direction for viewing the rearview mirror or side mirror.
  • a state in which the detected driver's motion is almost occupied by forward gaze and gaze in a predetermined direction is considered that the driver is sufficiently concentrated on driving.
  • the driver is driving, he / she should not be looking closely ahead, talking, sleepy, carrying food in his mouth, or should not be mobile
  • the driver is not concentrated on driving.
  • the driver's facial expression for example, the eye open state of the driver's eyes, the frequency of blinking, or eye movements are detected.
  • the degree of concentration is defined as “high”.
  • 20% or more of the driver's movement is forward gaze or gaze in a predetermined direction, and the driver's movement includes operation of a mobile phone or smartphone and a state of a call.
  • the degree of concentration is defined as “low”.
  • the concentration degree determination unit 612 determines the movement of the driver recognized from the monitoring video data stored in the monitoring video storage area 631 with reference to the determination criterion.
  • the determination criterion is an example, and the designer can appropriately set it appropriately.
  • the determination criterion may be set such that a score is determined in advance for each representative operation, and the total is multiplied by the ratio of each operation to the total time. At this time, the degree of concentration is digitized based on the driver's time-series movement.
  • the concentration level is not limited to two stages.
  • the degree of concentration may be evaluated in three stages by defining the degree of concentration “medium” between the degree of concentration “high” and the degree of concentration “low”.
  • the concentration degree determination unit 612 reads the concentration degree obtained in the immediately preceding determination from the concentration degree storage area 632 when the number of monitoring video data necessary for the concentration degree is not acquired due to the shielding of the driver camera 7 by the steering wheel 4. .
  • the concentration level determination unit 612 outputs the read concentration level to the operation mode switching control unit 613 as the current concentration level.
  • the concentration degree determination unit 612 stores the read concentration degree in the concentration degree storage area 632 with a time stamp.
  • the degree of concentration is described as an example of an index representing the driver's state, but the present invention is not limited to this.
  • the operation mode switching control unit 613 controls the automatic operation control device 5 to switch the setting between the manual operation mode and the automatic operation mode based on the concentration degree determined by the concentration degree determination unit 612.
  • the operation mode switching control unit 613 stores information indicating the switching control in the history storage area 633 with a time stamp.
  • FIG. 4 is a flowchart showing a procedure when the monitoring video acquisition control unit 611 of the control unit 61 provided in the operation mode switching control device 6 shown in FIG. 2 stores the monitoring video data.
  • the driver camera 7 and the steering sensor 8 are activated.
  • the driver camera 7 continuously captures a predetermined range including the driver's face.
  • the driver camera 7 outputs the captured monitoring video to the operation mode switching control device 6 as a video signal.
  • the driver camera 7 may intermittently image the driver at a cycle shorter than the time interval for determining the driver state.
  • the driver camera 7 or the input / output interface unit 62 may encode the video signal according to a predetermined encoding method. By doing so, it is possible to reduce the information amount of the monitoring video data and save the storage capacity of the monitoring video storage area 631.
  • the steering sensor 8 detects the steering angle of the steering wheel 4.
  • the steering sensor 8 outputs the detected steering angle to the operation mode switching control device 6 as a steering angle detection signal.
  • the monitoring video acquisition control unit 611 receives the steering angle data output from the steering sensor 8 and converted by the input / output interface unit 62 (step S41). Further, the monitoring video acquisition control unit 611 receives the monitoring video data output from the driver camera 7 and converted by the input / output interface unit 62 (step S42).
  • the monitoring image acquisition control unit 611 determines whether or not the steering angle represented by the received steering angle data is included in the preset shielding angle width at a predetermined sampling rate (step S43). When the steering angle is not included in the shielding angle width (No in step S43), the monitoring video acquisition control unit 611 samples the received monitoring video data at the sampling rate and stores the sampled monitoring video data in the monitoring video storage. It memorize
  • the monitoring video acquisition control unit 611 When the monitoring video data is stored in the monitoring video storage area 631 in step S44 or the monitoring video data to which the shielding flag is attached is stored in the monitoring video storage area 631 in step S45, the monitoring video acquisition control unit 611 operates. It is determined whether or not has been completed (step S46). When the operation is finished (Yes in step S46), the monitoring video acquisition control unit 611 ends the process. On the other hand, when the driving does not end (No in step S46), the monitoring video acquisition control unit 611 shifts the process to step S41, and repeats the operations in steps S41 to S45 until the driving of the vehicle 1 ends.
  • FIG. 5 is a flowchart illustrating an example of a procedure when the concentration degree determination unit 612 of the control unit 61 provided in the operation mode switching control device 6 illustrated in FIG. 2 determines the concentration degree of the driver.
  • the concentration degree determination unit 612 starts timing.
  • the concentration degree determination unit 612 determines whether or not a certain time has elapsed since the start of time measurement (step S51).
  • the concentration degree determination unit 612 stores the preset number of monitoring video data stored in the monitoring video storage area 631 during the predetermined time in the monitoring video storage area 631.
  • Step S52 The concentration degree determination unit 612 recognizes the movement of the driver and / or the expression of the driver based on the read monitoring video data.
  • the degree-of-concentration determination unit 612 reads a preset number of monitoring video data each time the monitoring video acquisition control unit 611 newly samples the monitoring video data, and moves the driver based on the read monitoring video data. Etc. may be recognized.
  • the degree-of-concentration determination unit 612 determines whether or not there is data in which the movement of the driver and / or the driver's facial expression cannot be recognized among the read monitoring video data, that is, data with a shielding flag (see FIG. Step S53). When there is data to which the shielding flag is attached (Yes in step S53), the concentration determination unit 612 determines whether there is the number of samples necessary for determining the concentration of the data in which the driver's movement or the like can be recognized. It is determined whether or not (step S54). When the number of samples necessary to determine the concentration level exists (Yes in step S54), the concentration level determination unit 612 determines the concentration level based on data that can recognize the movement of the driver and the like (step S55). ).
  • the concentration level determination unit 612 outputs information indicating the determined concentration level to the operation mode switching control unit 613.
  • the concentration level determination unit 612 stores information indicating the determined concentration level in the concentration level storage area 632 in association with information indicating the determination timing, for example, time stamp information (step S56).
  • the concentration level determination unit 612 refers to the attached time stamp with the concentration level determined immediately before the current time. To read from the concentration storage area 632.
  • the concentration level determination unit 612 employs the read concentration level as the current concentration level (step S57).
  • the concentration degree determination unit 612 outputs information representing the adopted concentration degree to the operation mode switching control unit 613.
  • the concentration degree determination unit 612 stores the information indicating the adopted concentration degree in the concentration degree storage area 632 in association with information indicating the determination timing, for example, time stamp information (step S58).
  • step S54 it is not essential to determine whether or not the number of samples necessary for determining the degree of concentration exists in step S54. For example, when it is determined in step S53 that there is data with a shielding flag, the concentration degree determination unit 612 may shift the process to step S57.
  • step S56 or step S58 when the information indicating the degree of concentration is stored in the monitoring video storage area 631, the concentration degree determination unit 612 determines whether or not the driving is finished (step S59). When the driving is finished (Yes in step S59), the concentration degree determination unit 612 ends the process. On the other hand, when the driving does not end (No in step S59), the concentration degree determination unit 612 shifts the process to step S51, and repeats the operations in steps S51 to S58 until the driving of the vehicle 1 ends.
  • the operation mode switching control unit 613 controls the automatic operation control device 5 to switch the setting between the manual operation mode and the automatic operation mode based on the concentration degree determined by the concentration degree determination unit 612. For example, when switching from the “automatic operation mode” to the “manual operation mode”, the operation mode switching control unit 613 informs the driver that the operation mode is to be switched and the period until the switching using an alarm generator (not shown). Notice.
  • the driving mode switching control unit 613 determines whether or not the driver's concentration level exceeds a preset concentration level during the switching period. The driving mode switching control unit 613 determines that the driver is ready to switch to the “manual driving mode” when the driver's concentration exceeds the preset concentration during the switching period. If it is determined that the driver is ready during the switching period, the driving mode switching control unit 613 switches the “automatic driving mode” to the “manual driving mode” after the switching period has elapsed. Is output to the automatic operation control device 5.
  • the operation mode switching control unit 613 stores information indicating the switching control in the history storage area 633 of the storage unit 63 in association with information indicating the determination timing, for example, time stamp information.
  • the determination using the degree of concentration is not limited to the above.
  • the countdown may be started based on the degree of concentration during the switching period for switching the operation mode.
  • the driving mode switching control unit 613 notifies the driver that the driving mode is to be switched, and then whether or not the driver is ready based on the concentration level output from the concentration level determination unit 612. Determine whether.
  • the operation mode switching control unit 613 determines that the driver is ready and starts counting down the period until the operation mode is switched.
  • the control unit 61 is configured such that the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the steering angle detected by the steering sensor 8. It is determined whether or not.
  • the control unit 61 sets the concentration level at the time when the shooting range of the driver camera 7 is shielded at the timing immediately before the shielding. The determined degree of concentration is adopted. For this reason, even when the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4, for example, when it is desired to use the degree of concentration in determining whether to switch from automatic driving to manual driving, etc. It becomes possible to carry out the determination without any problem.
  • the monitoring image acquisition control unit 611 is acquired when the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the steering angle detected by the steering sensor 8. Discriminate surveillance video.
  • the concentration level determination unit 612 determines the concentration level determined at the timing immediately before the shooting range of the driver camera 7 is blocked. 7 is adopted as the degree of concentration while the shooting range is shielded. Therefore, the monitoring video acquisition control unit 611 can efficiently sort the monitoring video acquired when the shooting range of the driver camera 7 is blocked.
  • the case where the monitoring video acquired by the monitoring video acquisition control unit 611 when the imaging range of the driver camera 7 is shielded by the support member of the steering wheel 4 has been described as an example.
  • the steering angle acquired by the steering sensor 8 may be used.
  • the concentration degree determination unit 612 is based on the steering angle acquired by the steering sensor 8, and the monitoring video is in a shielding state where the shooting range of the driver camera 7 is blocked by the steering wheel 4. You may determine with what was acquired.
  • the concentration degree determination unit 612 receives the steering angle data output from the steering sensor 8 and converted by the input / output interface unit 62.
  • the concentration degree determination unit 612 determines whether or not the steering angle represented by the received steering angle data is included in a preset shielding angle width at a predetermined sampling rate.
  • the concentration degree determination unit 612 causes the storage unit 63 to store timing information indicating the timing determined to be in the shielding state.
  • the concentration degree determination unit 612 reads the monitoring video data stored in the monitoring video storage area 631 from the monitoring video storage area 631 for a predetermined time.
  • the degree-of-concentration determination unit 612 determines the monitoring video data acquired in the shielding state from the read monitoring video data by matching the stored timing information with the read monitoring video data.
  • the concentration level determination unit 612 determines immediately before the current time. The degree of concentration is read from the concentration degree storage area 632. Then, the concentration level determination unit 612 employs the read concentration level as the current concentration level.
  • the concentration level determination unit 612 adopts the concentration level determined immediately before the current time as the current concentration level. May be.
  • the concentration degree determination unit 612 can select the monitoring video data acquired when the shooting range of the driver camera 7 is shielded by the steering wheel 4, and the processing of the monitoring video acquisition control unit 611 can be reduced. It becomes.
  • the operation mode switching control unit 613 determines that the shooting range of the driver camera 7 is blocked by the steering wheel 4 based on the steering angle acquired by the steering sensor 8. Good. At this time, the operation mode switching control unit 613 operates as follows, for example. In the following operation, the concentration level determination unit 612 reads the monitoring video data stored in the monitoring video storage area 631 every time a predetermined time elapses, and determines the concentration level based on the read monitoring video data. The concentration level determination unit 612 stores information indicating the determined concentration level in the concentration level storage area 632.
  • the operation mode switching control unit 613 receives the steering angle data output from the steering sensor 8 and converted by the input / output interface unit 62. The operation mode switching control unit 613 determines whether or not the steering angle represented by the received steering angle data is included in the preset shielding angle width at a predetermined sampling rate. When the steering angle is included in the shielding angle width, the operation mode switching control unit 613 causes the storage unit 63 to store timing information indicating the timing determined to be in the shielding state.
  • the operation mode switching control unit 613 determines whether or not the operation mode can be switched, the operation mode switching control unit 613 reads, from the concentration level storage area 632, information indicating the concentration level acquired by the concentration level determination unit 612 immediately before the determination time point. Based on the timing information stored in the storage unit 63, the operation mode switching control unit 613 determines whether or not the read concentration level is the concentration level acquired in the shielding state. When the read concentration degree is the concentration degree acquired in the shielding state, the operation mode switching control unit 613 reads information indicating the concentration degree acquired immediately before entering the shielding state from the concentration degree storage area 632. The operation mode switching control unit 613 determines whether or not the operation mode can be switched based on the degree of concentration represented by the newly read information.
  • the operation mode switching control unit 613 switches the operation mode using the degree of concentration acquired immediately before entering the shielding state. It becomes possible. For this reason, it is possible to reduce processing in the monitoring video acquisition control unit 611 and the concentration degree determination unit 612.
  • the concentration determined in the past when the photographing range of the driver camera 7 is shielded by the support member of the steering wheel 4 is used.
  • the case where it is adopted has been described as an example.
  • the second embodiment a case will be described in which the timing at which the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 is predicted and handled.
  • FIG. 6 is a block diagram showing an example of a functional configuration of the operation mode switching control device 6B according to the second embodiment of the present invention.
  • the operation mode switching control device 6B includes a control unit 61B, an input / output interface unit 62, and a storage unit 63.
  • the control unit 61 by causing the CPU to execute a program stored in the program memory, a monitoring video acquisition control unit 611B and a concentration degree determination unit 612B, which are control functions necessary for implementing the second embodiment, are provided. Then, the operation mode switching control unit 613 and the shielding prediction unit 614 are realized.
  • the shielding prediction unit 614 takes in digital data (steering angle data) about the steering angle detection signal output from the steering sensor 8 from the input / output interface unit 62.
  • the shielding prediction unit 614 calculates the steering angular velocity based on the change over time of the acquired steering angle data.
  • the shielding prediction unit 614 predicts a period during which the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the relative position between the driver camera 7 and the steering wheel 4 and the calculated steering angular velocity.
  • the shielding prediction unit 614 outputs an operation signal to the monitoring video acquisition control unit 611B and the concentration degree determination unit 612B when a time that is a predetermined period before the predicted start time of the shielding period comes.
  • the monitoring video acquisition control unit 611B takes in digital data (monitoring video data) about the driver video signal output from the driver camera 7 from the input / output interface unit 62.
  • the monitoring video acquisition control unit 611B stores the captured monitoring video data in the monitoring video storage area 631 of the storage unit 63 at a predetermined sampling rate.
  • the monitoring image acquisition control unit 611B synchronizes with the timing at which the operation signal is received, and stores the captured monitoring image data in the monitoring image storage area 631 of the storage unit 63.
  • the concentration determination unit 612B reads the monitoring video data stored during the time interval from the monitoring video storage area 631 at a preset time interval.
  • the concentration degree determination unit 612B recognizes the movement of the driver and / or the facial expression of the driver based on the read monitoring video data.
  • the concentration degree determination unit 612B determines the degree of concentration of the driver with respect to driving based on the recognized movement of the driver.
  • the concentration determination unit 612B receives the operation signal transmitted from the shielding prediction unit 614, the monitoring video data stored during a predetermined period from the time including the received timing is received from the monitoring video storage area 631. Read. The concentration level determination unit 612B determines the concentration level based on the read monitoring video data. The concentration level determination unit 612B outputs the determination result of the concentration level to the operation mode switching control unit 613. In addition, the concentration level determination unit 612B stores the determination result of the concentration level in the concentration level storage area 632 with a time stamp.
  • FIG. 7 is a flowchart illustrating an example of a procedure when the shielding prediction unit 614 of the control unit 61B provided in the operation mode switching control device 6B illustrated in FIG. 6 outputs an operation signal.
  • the steering sensor 8 when driving is started, the steering sensor 8 is activated.
  • the steering sensor 8 detects the steering angle of the steering wheel 4.
  • the steering sensor 8 outputs the detected steering angle as a steering angle detection signal to the operation mode switching control device 6B.
  • the shielding prediction unit 614 receives the steering angle data for a predetermined period output from the steering sensor 8 and converted by the input / output interface unit 62 (step S71).
  • the shielding prediction unit 614 calculates the steering angular velocity based on the change over time of the received steering angle data (step S72).
  • the shielding prediction unit 614 predicts a period during which the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4 based on the relative position between the driver camera 7 and the steering wheel 4 and the calculated steering angular velocity ( Step S73).
  • the shielding prediction unit 614 determines whether or not the current time is a time that is a time period (for example, s seconds) prior to the predicted start time of the shielding period (step S74). When it is a time before a predetermined period from the start time of the shielding period (Yes in step S74), the shielding prediction unit 614 outputs an operation signal to the monitoring video acquisition control unit 611B and the concentration degree determination unit 612B ( Step S75).
  • a time period for example, s seconds
  • the shielding prediction unit 614 has finished driving the vehicle 1 It is determined whether or not (step S76).
  • the shielding prediction unit 614 ends the process when the driving is finished (Yes in Step S76).
  • the shielding prediction unit 614 shifts the process to step S71, and repeats the operations in steps S71 to S45 until the driving of the vehicle 1 is finished.
  • FIG. 8 is a flowchart showing an example of a procedure when the monitoring video acquisition control unit 611B of the control unit 61B shown in FIG. 6 stores the monitoring video data.
  • the driver camera 7 is activated.
  • the driver camera 7 continuously captures a predetermined range including the driver's face.
  • the driver camera 7 outputs the captured monitoring video as a video signal to the operation mode switching control device 6B.
  • the monitoring video acquisition control unit 611B receives the monitoring video data output from the driver camera 7 and converted by the input / output interface unit 62 (step S81).
  • the monitoring video acquisition control unit 611B determines whether or not the operation signal transmitted by the shielding prediction unit 614 has been received (step S82).
  • the monitoring video acquisition control unit 611B samples the received monitoring video data at a timing at which the operation signal is received, that is, at a timing different from a preset sampling rate.
  • the sampled monitoring video data is stored in the monitoring video storage area 631 (step S83).
  • the monitoring video acquisition control unit 611B samples the received monitoring video data at a preset sampling rate, and the sampled monitoring video data is stored in the monitoring video storage area. 631 (step S84).
  • the monitoring video acquisition control unit 611B determines whether or not the operation is finished (step S85). When the driving is finished (Yes in step S85), the monitoring video acquisition control unit 611B ends the process. On the other hand, when the driving does not end (No in step S85), the monitoring video acquisition control unit 611B shifts the process to step S81, and repeats the operations in steps S81 to S84 until the driving of the vehicle 1 ends.
  • FIG. 9 is a flowchart showing an example of a procedure when the concentration determination unit 612B of the control unit 61B shown in FIG. 6 determines the concentration of the driver.
  • the concentration degree determination unit 612B starts measuring time. After starting the time measurement, the concentration degree determination unit 612B determines whether or not the operation signal transmitted by the shielding prediction unit 614 has been received (step S91). When the operation signal is received (Yes in step S91), the concentration determination unit 612B receives the monitoring video data stored in the monitoring video storage area 631 before a predetermined period from the time including the timing at which the operation signal is received. Reading from the monitoring video storage area 631 (step S92).
  • the concentration degree determination unit 612B recognizes the movement of the driver and / or the facial expression of the driver based on the read monitoring video data.
  • the concentration level determination unit 612B determines the concentration level based on the movement of the driver recognized in the monitoring video data (step S93).
  • Concentration determination unit 612B outputs information indicating the determined concentration to operation mode switching control unit 613.
  • the concentration level determination unit 612B stores information indicating the determined concentration level in the concentration level storage area 632 in association with information indicating determination timing, for example, time stamp information (step S94).
  • step S91 when the operation signal is not received, the concentration degree determination unit 612B determines whether or not a predetermined time has elapsed since the start of time measurement (step S95).
  • the concentration determination unit 612B reads the monitoring video data stored in the monitoring video storage area 631 during the predetermined time from the monitoring video storage area 631 (step S96). The process proceeds to step S93.
  • the concentration determination unit 612B may read a preset number of monitoring video data each time the monitoring video acquisition control unit 611B newly samples the monitoring video data, and may shift the process to step S93.
  • step S93 when information indicating the degree of concentration is stored in the monitoring video storage area 631, the concentration degree determination unit 612B determines whether or not the driving is finished (step S97). When the driving is finished (Yes in step S97), the concentration degree determination unit 612B ends the process. On the other hand, when the driving does not end (No in step S97), the concentration degree determination unit 612B shifts the process to step S91, and repeats the operations in steps S91 to S96 until the driving of the vehicle 1 ends.
  • the control unit 61B has the shooting range of the driver camera 7 shielded by the support member of the steering wheel 4 based on the steering angle detected by the steering sensor 8. Predict the period. Then, the control unit 61B determines the concentration of the driver in the predicted period based on the monitoring video data acquired before the shielding period. For this reason, at the timing when the shooting range of the driver camera 7 is predicted to be shielded by the support member of the steering wheel 4, for example, when it is desired to use the degree of concentration in determining whether to switch from automatic driving to manual driving. However, it is possible to determine whether or not switching is possible using the degree of concentration acquired immediately before it is predicted to be in the shielding state.
  • the shielding prediction unit 614 outputs an operation signal when a time point that is a predetermined time period before the predicted start time of the shielding period is reached.
  • the monitoring video acquisition control unit 611B stores the monitoring video data captured at the timing when the operation signal is received.
  • the concentration level determination unit 612B determines the concentration level based on the monitoring video data acquired by the monitoring video acquisition control unit 611B at the timing at which the operation signal is received, and this concentration level is predicted as a shielding. The degree of concentration in the period is set.
  • the concentration determination unit 612B determines the concentration based on the monitoring video data acquired by the monitoring image acquisition control unit 611B at this timing when the operation signal is received.
  • the case where this concentration level is set as the concentration level in the predicted shielding period has been described as an example.
  • the concentration degree determination unit 612B may set the concentration degree determined immediately before receiving the operation signal in accordance with a preset period as the concentration degree in the predicted shielding period. As a result, the degree of concentration at the timing when the operation signal is received can be quickly calculated.
  • the driving mode switching control part 613 acquires the information showing a concentration degree when switching a driving mode similarly to 1st Embodiment, and a driver
  • the driving mode switching control unit 613 acquires information indicating the concentration level immediately before the shooting range of the driver camera 7 is shielded by the support member of the steering wheel 4, and the driver's concentration level is set in advance based on the acquired information. You may make it determine whether it exceeds the concentration degree to be performed.
  • the operation mode switching control unit 613 when the operation mode switching control unit 613 receives an operation signal from the shielding prediction unit 614, the operation mode switching control unit 613 acquires information indicating the degree of concentration at the timing when the operation signal is received.
  • the information acquired here may be information indicating the degree of concentration determined at a preset timing, or information indicating the degree of concentration determined at the timing when the operation signal is received in the concentration degree determination unit 612. It may be.
  • the driving mode switching control unit 613 determines whether or not the driver's concentration level exceeds a preset concentration level based on the acquired information.
  • the operation mode switching control unit 613 stores the determination result in the storage unit 63.
  • the operation mode switching control unit 613 uses the determination result stored in the storage unit 63 to switch the operation mode. Judgment is made. As a result, the determination result can be stored in preparation for an operation mode switching instruction given when the driver camera 7 is shielded by the steering wheel 4.
  • the vehicle type, the function of the automatic operation control device, the control function and control procedure of the operation mode switching control device, and the control contents can be variously modified and implemented without departing from the gist of the present invention.
  • the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage.
  • various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.
  • a driving mode switching control device for switching and controlling a driving mode of a vehicle, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
  • a memory for storing monitoring video data representing the monitoring video shot by the monitoring camera;
  • the at least one hardware processor comprises: Based on the monitoring video data stored in the memory, determine the concentration of the driver, Based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel, it is determined whether or not the imaging range of the monitoring camera is in a shielding state that is shielded by a support member of the steering wheel;
  • An operation mode switching control device configured to set the concentration degree determined immediately before entering the shielding state as the concentration degree in the shielding state in the shielding state.
  • a driving mode switching control device for switching and controlling a driving mode of a vehicle, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver,
  • a memory for storing monitoring video data representing the monitoring video shot by the monitoring camera;
  • At least one hardware processor connected to the memory, the at least one hardware processor comprising: Based on the monitoring video data stored in the memory, determine the concentration of the driver, Based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel, predicting a shielding period in which the imaging range of the monitoring camera is shielded by a support member of the steering wheel;
  • An operation mode switching control device configured to determine the degree of concentration in the predicted shielding period based on monitoring video data acquired before the shielding period.
  • the process is A determination process for determining whether or not the imaging range of the monitoring camera is in a shielded state that is shielded by a support member of the steering wheel based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel;
  • An operation mode switching control method comprising: a determination process in which the concentration degree determined immediately before entering the shielding state is the concentration degree in the shielding state in the shielding state.
  • a driving mode switching control method executed by a device for switching control of a vehicle driving mode, which is used in a vehicle in which a monitoring camera is installed on the back side of a steering wheel as viewed from a driver, Using at least one hardware processor to perform a step of determining the concentration of the driver based on a monitoring image captured by the monitoring camera;
  • the process is A prediction process for predicting a shielding period in which the imaging range of the monitoring camera is shielded by a support member of the steering wheel based on a steering angle detected by a steering sensor that detects a steering angle of the steering wheel;
  • An operation mode switching control method comprising: a determination process for determining the degree of concentration in the predicted shielding period based on a monitoring video acquired before the shielding period is reached.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Mechanical Engineering (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Automation & Control Theory (AREA)
  • Mathematical Physics (AREA)
  • Transportation (AREA)
  • Multimedia (AREA)
  • Traffic Control Systems (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)

Abstract

La présente invention concerne la contre-mesure suivante, dans le but de permettre de déterminer une condition d'un conducteur même si la plage de capture d'une caméra de surveillance est bloquée par un élément de support d'un volant de direction. L'invention concerne une partie de commande qui est dans un dispositif de commande de commutation de mode de conduite, ledit dispositif de commande de commutation de mode de conduite étant utilisé dans un véhicule dans lequel une caméra de surveillance est positionnée sur un côté arrière d'un volant de direction lorsqu'elle est vue par un conducteur et fournissant une commande de commutation d'un mode de conduite du véhicule, ladite partie de commande déterminant le niveau de concentration du conducteur sur la base d'une vidéo de surveillance qui est capturée par la caméra de surveillance. Sur la base d'un angle de direction qui est détecté par un capteur de direction, ledit capteur de direction détectant l'angle de direction d'un volant de direction, la partie de commande détermine s'il existe une condition bloquée dans laquelle la plage de capture de la caméra de surveillance est bloquée par un élément de support du volant de direction et s'il est déterminé qu'il y a une condition bloquée, traite le niveau de concentration qui a été déterminé immédiatement avant la condition bloquée en tant que niveau de concentration pendant la condition bloquée.
PCT/JP2017/036705 2017-03-09 2017-10-10 Dispositif, procédé et programme de commande de commutation de mode de conduite WO2018163492A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017045212A JP2018149821A (ja) 2017-03-09 2017-03-09 運転モード切替制御装置、方法、及びプログラム
JP2017-045212 2017-03-09

Publications (2)

Publication Number Publication Date
WO2018163492A1 WO2018163492A1 (fr) 2018-09-13
WO2018163492A9 true WO2018163492A9 (fr) 2018-11-15

Family

ID=63448514

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/036705 WO2018163492A1 (fr) 2017-03-09 2017-10-10 Dispositif, procédé et programme de commande de commutation de mode de conduite

Country Status (2)

Country Link
JP (1) JP2018149821A (fr)
WO (1) WO2018163492A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7346969B2 (ja) * 2019-07-22 2023-09-20 トヨタ自動車株式会社 運転支援装置
CN113014913B (zh) * 2019-12-19 2022-09-06 合肥君正科技有限公司 一种车内监控相机分段模式遮挡检测的方法
CN114179831B (zh) * 2021-12-29 2023-02-17 吉林大学 一种基于驾驶员分心判断的人机转向切换控制方法
US12195007B2 (en) * 2022-11-28 2025-01-14 GM Global Technology Operations LLC Systems and methods for improving driver monitoring systems

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002254956A (ja) * 2001-03-02 2002-09-11 Toyota Industries Corp 覚醒状態低下検知装置
JP6350145B2 (ja) * 2014-09-09 2018-07-04 株式会社デンソーアイティーラボラトリ 顔向き検出装置及び車両用警告システム
JP2016151815A (ja) * 2015-02-16 2016-08-22 株式会社デンソー 運転支援装置

Also Published As

Publication number Publication date
JP2018149821A (ja) 2018-09-27
WO2018163492A1 (fr) 2018-09-13

Similar Documents

Publication Publication Date Title
US11027746B2 (en) Drive mode switching control device, method and program
CN108140319B (zh) 停车辅助方法及停车辅助装置
RU2656933C2 (ru) Способ и устройство для предупреждения о встречном транспортном средстве
JP4206928B2 (ja) 衝突可能性判定装置
CN109844842B (zh) 驾驶模式切换控制装置、系统、方法以及存储介质
WO2018163492A1 (fr) Dispositif, procédé et programme de commande de commutation de mode de conduite
US20120296522A1 (en) Driving support system
JP5585194B2 (ja) 事故状況記録システム
JP2015164027A (ja) 電子機器、電子機器の制御方法及び電子機器の制御プログラム
CN109843690B (zh) 驾驶模式切换控制装置、系统以及方法
CN110546699B (zh) 推荐驾驶输出装置、推荐驾驶输出方法和推荐驾驶输出系统
US20200017123A1 (en) Drive mode switch controller, method, and program
US11557206B2 (en) Information provision system, server, and mobile terminal
JP6528213B2 (ja) 車両制御装置、車両制御方法、および車両制御プログラム
JP7073972B2 (ja) 情報処理システム、プログラム、及び制御方法
CN111469762B (zh) 显示系统、行驶控制装置、显示控制方法以及存储介质
JP2013109639A (ja) 画像処理装置、サーバ、携帯端末装置、及び画像処理方法
JP7202195B2 (ja) 表示システム、走行制御装置、表示制御方法およびプログラム
JP2018149822A (ja) 運転モード切替制御装置、方法及びプログラム
JP2010238037A (ja) 信号予測装置
CN109661688B (zh) 影像输出系统
WO2018163493A1 (fr) Dispositif, procédé et programme de surveillance
WO2018163454A1 (fr) Dispositif, procédé et programme de commande de commutation de mode de conduite
JP4830457B2 (ja) ナビゲーション装置、ナビゲーション方法、ナビゲーションシステム及び渋滞情報案内プログラム
JP2019028481A (ja) 車載器および運転支援装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17899803

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17899803

Country of ref document: EP

Kind code of ref document: A1

点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载