WO2010139351A1

WO2010139351A1 - Vision system and method for a motor vehicle

Info

Publication number: WO2010139351A1
Application number: PCT/EP2009/004075
Authority: WO
Inventors: Anders Bergmark
Original assignee: Autoliv Development Ab
Priority date: 2009-06-05
Filing date: 2009-06-05
Publication date: 2010-12-09

Abstract

A vision system (10) for a motor vehicle comprises imaging means (11) for recording images of a region in front of the motor vehicle, object identifying means (15) for processing image data provided by the imaging means (11) in order to identify objects in the imaged scene, and object tracking means (16) for tracking the position of objects identified by said object identifying means (15), characterized in that said vision system (10) comprises noise level calculation means (24) adapted to calculate an estimated noise level of recorded image data, and to input said estimated noise level into said object tracking means (16) for use in said object tracking.

Description

Vision system and method for a motor vehicle

The invention relates to a vision system for a motor vehicle, comprising an imaging means for recording images of a region in front of the motor vehicle, object identifying means for processing image data provided by the imaging means in order to identify objects in the imaged scene, and object tracking means for tracking the position of objects identified by said object identifying means. Furthermore, the invention relates to a corresponding vision method.

Vision systems of this kind are generally known, for example from US 7 263 209.

Vision systems for motor vehicles must work under a large variety of conditions ranging from clear day or indoors, to heavy snowfall or rain even at night. The reliability of object detection is affected by the surrounding conditions, for example if there is water on a window in front of the camera, and therefore the information available in the image is also affected. This can have a negative influence on the reliabil- ity and/or speed of the tracking means. Furthermore, the tracking means usually operates under the assumption of an average noise level in the recorded images. An actual noise level being higher than the assumed average noise level will result in noisy filter outputs. On the other hand, an actual noise level being lower than the assumed average noise level will result in the filter response being relatively slow.

The object of the invention is to provide a vision system and method with a fast and reliable tracking means providing high quality outputs under different surrounding conditions.

The invention solves this object with the features of the in- dependent claims. By calculating an estimated noise level of the recorded image data, situations can be determined where the image quality in the recorded images is low. Such conditions for example include darkness, overexposure by sunlight and/or rain, snow or fog. Using the calculated noise level in the tracking software, the tracking means can be optimally adjusted to the current surrounding conditions, providing tracking outputs with low noise under different external conditions, and avoiding a tracking means which is unnecessarily slow if the external conditions are good.

Preferably the object tracking means comprises a tracking filter. Preferably this may be a recursive filter, in particular a recursive time domain filter like a Kalman filter which has the possibility to specify the expected measurement noise. This parameter is conventionally set to a constant value corresponding to an average expected noise level independent of the actual external conditions. The invention allows using the calculated (variable) information content of the recorded images, reflecting the actual surrounding condition, to be used as the expected measurement noise control parameter of the Kalman filter, or the tracking filter in general.

Preferably the noise level is calculated using image diagnostics of the recorded image data which allows a reliable esti- mation of the information content in the recorded image data. This may comprise for example a frequency spectrum analysis, e.g. a Fourier transform of the image data and/or an edge strength analysis, e.g. the application of a Sobel operator filter to the image data. Information from a rain sensor and/or information indicating windscreen wiper activation may also be used to estimate the noise level.

In the following the invention shall be illustrated on the basis of preferred embodiments with reference to the accompanying drawings, wherein:

Fig. 1 shows a vision system for a motor vehicle.

The vision system 10 is mounted in a motor vehicle and comprises an imaging means 11 for recording images of a region in front of the motor vehicle. Preferably the imaging means 11 comprises one or more optical and/or infrared cameras 12a, 12b, where infrared covers near IR with wavelengths below 5 microns and/or far IR with wavelengths beyond 5 microns . Preferably the imaging means 11 comprises stereo cameras 12a, 12b; alternatively a mono camera can be used.

The cameras 12a, 12b are coupled to an image pre-processor 13 which may be realized by a dedicated hardware circuit, and is adapted to calibrate the cameras 12a, 12b, control the capture and digitizing of the images, warp them into alignment, merge left/right images into single images, and create multi-resolu- tion disparity images, which per se is known in the art.

The pre-processed image data is then provided to an electronic processing means 14 where further image processing is carried out by means of a corresponding software. In particular, the processing means 14 comprises an object detection means 15 adapted to identify and classify possible objects in front of the motor vehicle, such as pedestrians, bicyclists or large animals, a tracking means 16 adapted to track the position of object candidates in the recorded images identified by the object detection means 15 over time, and a decision means 17 adapted to activate or control vehicle safety means 18, 19, 20, ... depending on the result of the processing in the object detection and tracking means 15, 16. The vehicle safety means 18, 19, 20, ... may comprise a warning means 18 adapted to provide a collision warning to the driver by suitable optical, acoustical and/or haptical warning signals; display means 19 for displaying information relating to an identified object; one or more restraint systems 20 such as occupant airbags or safety belt tensioners,- pedestrian airbags, hood lifters and the like; dynamic vehicle control systems such as brakes.

The object tracking means 16 comprises a tracking filter, e.g. a Kalman filter, which allows to specify an expected measurement noise as a control parameter.

The electronic processing means 14 is preferably programmed or programmable and may comprise a microprocessor or micro- controller. The image pre-processor 13 and the electronic processing means 14 are preferably realised in an on-board electronic control unit (ECU) and may be connected to the cameras 12a, 12b, the safety means 18, 19, 20, ... and other devices like a vehicle rain sensor 21 or a means 22 indicating windscreen wiper activation via a vehicle data bus. All steps from imaging, image pre-processing, image processing, to activation or control of safety means 18, 19, 20, ... are automatically and continuously performed during driving in real time.

The processing means 14 comprises a noise level calculation means 24, preferably realized by software, which is adapted to calculate an estimated noise level of the recorded images, and to input said estimated noise level into the object tracking means 16 for use in the object tracking process performed therein. In particular, said estimated noise level is specified as the expected measurement noise control parameter of the Kalman filter. In this manner, the object tracking means 16 can be adapted to changing conditions of the vehicle surrounding environment, resulting in a fast and reliable object tracking process.

In a preferred embodiment the estimated noise level is calcu- lated using image diagnostics of the recorded image data, which is indicated in Fig. 1 by the solid input arrow to the noise level calculation means 24. Several calculation methods are possible. In one embodiment, an analysis of the image frequency contents in a detected region-of-interest , e.g. using a 2D discrete Fourier transformation, can be employed, where a large high frequency content indicates a clear, sharp view. Similarly, an analysis of the image frequency contents in the whole image, also e.g. using a 2D discrete Fourier transformation, can be employed. Alternatively an edge strength analy- sis, e.g. using a Sobel operator filter, can be employed, where strong edges indicate a clear view. A clear, sharp view indicates a high information content in the image resulting in low noise in the detected object.

In an embodiment not shown in the Figure an independent imaging means or other dedicated sensing means may be used for obtaining information allowing to calculate the estimated noise level of the surrounding vehicle environment .

In addition to, or instead of, image diagnostics of recorded image data, information from other sources may be used for calculating the estimated noise level. For example signals from a vehicle rain sensor 21 may be used, where detected rain usually indicates a reduced image quality. Information 22 indicating windscreen wiper activation, providing another indicator for a reduced image quality, may also be used.

Claims

Claims :

1. A vision system (10) for a motor vehicle, comprising imaging means (11) for recording images of a region in front

5 of the motor vehicle, object identifying means (15) for processing image data provided by said imaging means (11) in order to identify objects in the imaged scene, and object tracking means (16) for tracking the position of objects identified by said object identifying means (15) ,o characterized in that said vision system (10) comprises noise level calculation means (24) adapted to calculate an estimated noise level of recorded image data, and to input said estimated noise level into said object tracking means (16) for use in said object tracking. 5

2. The safety system according to claim 1, wherein said object tracking means (16) comprises a tracking filter.

3. The safety system according to claim 2, wherein said 0 tracking filter is a Kalman filter.

4. The safety system according to claim 2 or 3, wherein said object tracking means (16) is adapted to use said estimated noise level as a control parameter for said tracking5 filter.

5. The safety system according to any one of the preceding claims, wherein said noise level calculation means (24) is adapted to perform said noise level calculation using im-o age diagnostics of recorded image data.

6. The safety system according to claim 5, wherein said noise level calculation means (24) is adapted to perform a fre- quency spectrum analysis of recorded image data.

7. The safety system according to claim 5 or 6, wherein said noise level calculation means (24) is adapted to perform a Fourier transform of recorded image data.

8. The safety system according to any one of claims 5 to 7, wherein said noise level calculation means (24) is adapted to perform an edge strength analysis of recorded image data.

9. The safety system according to any one of claims 5 to 8 , wherein said noise level calculation means (24) is adapted to apply a Sobel operator filter to recorded image data.

10. The safety system according to any one of the preceding claims, wherein said noise level calculation means (24) is adapted to use information from a vehicle rain sensor (21) in said noise level calculation.

11. The safety system according to any one of the preceding claims, wherein said noise level calculation means (24) is adapted to use information (22) indicating windscreen wiper activation in said noise level calculation.

12. A vision method for a motor vehicle, comprising recording images of a region in front of the motor vehicle, automatically processing said recorded image data in order to identify objects in the imaged scene, and tracking the po- sition of identified objects, characterized by calculating an estimated noise level of the recorded image data, and using said estimated noise level in said object tracking.