WO2017017359A1

WO2017017359A1 - Method and system for displaying a direction sign of the road scene for a motor vehicle

Info

Publication number: WO2017017359A1
Application number: PCT/FR2016/051911
Authority: WO
Inventors: Francois Laurent; Adrien CARMONE
Original assignee: Renault S.A.S
Priority date: 2015-07-24
Filing date: 2016-07-22
Publication date: 2017-02-02
Also published as: EP3326110A1; FR3039309A1; FR3039309B1

Abstract

A method for displaying a direction sign of the road scene for a motor vehicle provided with a camera, a screen and a first calculation means (10a) capable of recognising speed limit signs, the camera being attached to the motor vehicle in such a way as to acquire images of the road scene located in front of the vehicle, the screen being disposed so as to be visible to the driver during use of the vehicle. The method comprises the following steps: acquiring, without digital zoom, an image of the scene observed with the camera, detecting if at least one road sign is present on the image of the road scene observed by the first calculation means (10a), and, if this is the case, determining if at least one direction sign is present among the road signs detected by means of the first calculation means (10a), and, if this is the case, determining the direction sign to be displayed and carrying out character recognition on the direction sign to be displayed, determining the readability of the characters of the direction sign to be displayed on the screen of the vehicle, and, if the direction sign is considered to be readable, scaling the image of the direction sign to be displayed and processing the scaled image, and maintaining the display on the screen of the direction sign to be displayed for as long as the direction sign remains present in the acquired image.

Description

Method and system for displaying a road sign of the road scene for a motor vehicle.

The technical field of the invention is the processing and display of images acquired by an on-board camera, and more particularly the application of such a treatment to traffic assistance systems.

When a driver of a motor vehicle arrives on a road interchange, it is interesting for him to identify the path he must take to position himself correctly in his direction and avoid hasty maneuvers at the last moment.

This is usually offered by navigation systems when a route is defined. On the other hand, when the route in the navigation system has not been defined, the cartography is no longer up to date, only bypass / deviation zones are proposed for reasons of track work, that routes "bis Are proposed or that the GPS signal is lost during taxi (ex: tunnel), it is interesting for the driver to have a visual aid on the screen present in the dashboard of the vehicle.

From the state of the art, document KR101478258 discloses a method of lane detection on a multi-lane road. However, the direction signs are not detected or displayed.

In addition, it is known from the state of the art, methods of displaying the speed limit signs in a multimedia screen of a motor vehicle. These speed limit signs are located on the road scene. However, these methods do not allow to display the direction signs.

There is a need for a method and system for displaying road scene direction panel images detected on an image acquired by an on-board camera. The subject of the invention is a method of displaying a direction sign of the road scene for a motor vehicle equipped with a camera, a screen and a first calculation means able to realize a recognition of the panels. of speed, the camera being fixed on the motor vehicle so as to acquire images of the road scene in front of the vehicle, the screen being arranged so as to be visible to the driver during the use of the vehicle.

The method comprises the following steps:

we acquire, without digital zoom, an image of the scene observed with the camera,

detecting if at least one traffic sign is present on the image of the road scene observed by the first calculation means,

if this is the case, it is determined whether at least one direction sign is present among the traffic signs detected via the first calculation means,

if this is the case, determine the direction sign to be displayed and proceed to a character recognition on the direction panel to be displayed,

the readability of the characters of the direction sign to be displayed on the vehicle screen is determined,

if the direction sign is considered readable, the image of the direction sign to be displayed is scaled and the scaled image is processed, and

the display on the display of the direction panel is displayed, as long as the direction sign remains on the acquired image.

To determine if at least one direction sign is present among the detected traffic signs, it is possible to determine the shape of the panels detected by recognition of the speed panels and the determined shapes can be compared to a database of steering panel shapes. .

It can be determined whether a road sign is a direction sign by determining the presence of an arrow of direction by comparison with a database of direction arrows on direction signs.

When the road includes several traffic lanes, the sign to be displayed can be determined by performing the following steps:

the presence of tracks according to the signaling lines present on the road is determined by means of an AFIL (Lane Departure Warning System) unintentional crossing warning method; line maintenance system carried out by a calculation system fitted to the vehicle,

the detected direction panels are associated with a track according to the respective positions of the panels and tracks,

The route is determined in the vehicle, and the panel is displayed as a panel to display the steering panel among the detected direction signs associated with the lane in which the vehicle is traveling.

When the road includes a single lane, it can be determined that the panel to be displayed is the direction sign detected.

To determine the legibility of the directional panel to be displayed, it is possible to determine the number of pixels of the characters whose presence has been detected on the cropped image of the panel to be displayed when recognizing the speed signs.

it can be determined whether the determined size of the characters is greater than or equal to a predetermined minimum character size value according to the Johnson Rosell criterion with a pixel number s corresponding to 50% recognition of an empirically determined character. , and

if this is the case, we can confirm the possibility of a mid scale without degradation of the liibility on the screen according to ergonomic rules.

Another object of the invention is a system for displaying a roadside direction sign for a motor vehicle. equipped with a camera, a screen and a first calculation means adapted to perform a recognition of the speed panels, the camera being fixed on the motor vehicle so as to acquire images of the road scene in front of the vehicle , the screen being arranged so as to be visible to the driver during the use of the vehicle.

The system comprises the first calculation means capable of detecting the presence of signaling panels on the image received from the camera, a second calculation means able to determine if at least one direction sign is present among the detected signaling panels, and a fourth calculation means adapted to determine the readability of the characters of the direction panel to be displayed on the vehicle screen. The fourth calculation means is furthermore capable of scaling the image of the directional panel to be displayed as well as scaled image processing, and to maintain the display on the screen. the display of the direction panel to be displayed, as long as the steering panel remains present on the acquired image, the first calculation means being connected at the output to the second calculation means itself connected at the output to the fourth calculation means, the fourth calculation means being outputted to the screen.

The second calculation means may be able to determine if a traffic sign is a direction sign comparing the shape of the traffic sign with a database of the shapes of the direction signs.

The second calculation means may be able to determine if a traffic sign is a direction sign by determining the presence of a direction arrow by comparison with a database of direction arrows on steering panels.

The system may comprise a third calculation means able to carry out an unintentional line crossing warning or a line maintenance assistance connected between the second calculation means and the fourth calculation means, the third calculation means being able to determine whether the road includes one or more lanes, and capable of determining the direction sign to be displayed in function of the respective positions of the panels, tracks and the vehicle.

In order to determine the readability of the characters of the direction panel to be displayed on the vehicle screen, the fourth calculation means may be able to determine the size in number of pixels of the characters whose presence has been detected on a cropped image of the vehicle. panel to be displayed from the first calculation means, to determine whether the determined size of the characters is greater than or equal to a predetermined minimum character size value according to the Johnson Rosell criterion with a pixel number value corresponding to 50% of recognition of an empirically determined character and if so, to confirm the possibility of scaling without degradation of readability on the screen according to ergonomic rules,

Other objects, features and advantages of the invention will become apparent on reading the following description, given solely by way of nonlimiting example and with reference to the appended drawings in which:

FIG. 1 illustrates the main steps of a method of displaying a direction sign of the road scene for a motor vehicle according to the invention,

FIG. 2 illustrates the main elements of a display system of a roadside direction sign for a motor vehicle according to a first embodiment of the invention, and

- Figure 3 illustrates the main elements of a display system of a road sign of the road scene for a motor vehicle according to a second embodiment of the invention.

The Johnson Rosell Criterion was originally defined for military thermal observation applications in which detection, identification and recognition thresholds were to be determined. This criterion is employed in the present invention to determine the driver's ability to detect, identify and recognize on a display screen the image of characters present on a direction sign.

Johnson Rosell's criterion is to substitute for the object to be detected a target of the same size containing one or more pairs of lines in its height and width. The number of line pairs in the target is associated with a detection, recognition, and identification rate. These rates represent the probabilities of detection, recognition or identification of the object by the user. In other words, the implementation of the Johnson Rosell criterion in the invention makes it possible to determine whether an acquired image can be displayed or not, while knowing that it will be readable for the driver.

When applied to determining the readability of characters on a direction panel, the number of pairs of lines N corresponds to a number of pairs of lines of pixels on the screen displaying the final image while the object to be identified corresponds to the shape of a letter on the direction sign. The probability curve of Johnson's criterion is constructed according to the following formula:

E

( NOT

Probability (N) = (Eq 1)

With

N: the number of line pairs

N50: the number of line pairs with a probability of identification. The N50 value must be determined empirically and is dependent on the screen and the camera, as well as the position of the camera relative to the user.

This curve therefore makes it possible to determine the number N of lines or pixels on the screen necessary for the user to be able to identify a letter with a probability of success Probability (N) given and a number N50 of lines or pixels on the screen necessary for the user to be able to identify a letter with a 50% probability of success.

Knowing the number of N pixels required for identification, it can be compared to the number of pixels displayed to determine whether the user can identify a letter or not.

On the sensor of the camera, the letter must at least make the same number of pixels as on the screen (a pixel of the screen corresponds to a pixel of the camera) for such a conclusion to be valid. In the opposite case, a ratio of pixels between the camera and the screen must be introduced.

Taking into account the elements exposed above, one can also define the following probabilities by extension of the equation (Eq.1) to different situations:

On the road scene with the acquisition of a direction panel by the camera, it can detect the presence of information inside this panel up to a probability of detection. In this case, an N50 value relative to the detection is considered.

Among the information detected inside the panel, it is possible to recognize a character of another type of information (arrow, symbol, etc.) up to a probability of recognition. In this case, an N50 value relative to the recognition is considered.

On panels containing recognized characters, these characters can be identified up to a probability of identification. In this case, an N 50 value relative to the identification is considered.

In each of these situations, a minimum characteristic size of detection, recognition and identification to compare with the size of the corresponding object on the image from the camera or displayed on the screen.

We will now describe the evaluation of the readability of a character.

The reasoning presented below is made with respect to the horizontal direction of a character and must be reproduced in the same way for the vertical size.

It is posited that a pixel of the camera corresponds to a pixel of the screen.

It is also stated that the number of line pairs of the camera is equal to the number of pixel pairs.

The following quantities are also defined:

Pe_h: Horizontal size of the screen in pixels

Pe_v: Vertical size of the screen in pixels

PL h: Horizontal size of the letter in pixels

PL v: Vertical size of the letter in pixels

A probability table identifying a letter is then defined according to the number of pairs of lines acquired. Such a table can be determined empirically.

From this table, a threshold of number of line pairs is determined which makes it possible to exceed a chosen identification probability threshold. For example, such a table can indicate that a 95% probability of good identification corresponds to a number of pairs of pixels equal to at least 12, ie 24 pixels in height.

In other words, in this example, when the size of the letter is greater than 24 pixels, the character is considered identifiable in the horizontal direction.

The same steps are then repeated to determine the minimum vertical identification size.

The character is then considered readable when its dimensions satisfy the minimum identification sizes in both directions. The method of displaying a road scene steering panel for a motor vehicle will now be described in connection with FIG.

During a first step 1, it acquires, without digital zoom, an image of the scene observed with a camera attached to the motor vehicle so as to acquire images of the road scene in front of the vehicle.

During a second step 2, it is detected whether at least one traffic sign is present on the observed scene via a recognition of speed panels known from the state of the prior art. Indeed, this type of method is able to recognize the traffic signs, and among these panels, to recognize those of the speed panel type. Such a method can therefore be used to detect the presence of a traffic sign. If this is the case, the process continues with a third step.

During the third step 3, it is determined whether at least one direction sign is present among the detected traffic signs. For this purpose, the detected panels are classified according to their type by determining the shape of the panels detected by recognizing the speed panels and by comparing the determined shapes to a database of panel shapes. Alternatively, it is also determined by comparison with a database whether the panel comprises a direction arrow. If this is the case, the panel is a direction sign. If at least one detected panel is a direction panel, the method continues with a fourth step.

During a fourth optional step 4 according to the embodiments, it is determined whether the road comprises one or more lanes through an AFIL line crossing warning method (in English "Lane"). departure warning System ") or Lane Keeping Assist (known in French as" Lane Keeping Assist ") known from the state of the art. When the road includes multiple traffic lanes and several directional signs are detected on the central portion of the road above each lane, the directional sign to be displayed is determined by performing the following steps:

the presence of tracks is determined according to the signaling lines present on the road,

associating the detected directional signs with a track according to the respective positions of the panels and tracks,

the route in which the vehicle is traveling is determined

-Site sit as a panel to display the direction sign among the detected direction signs associated with the lane in which the vehicle is traveling.

When the road comprises a single traffic lane, it is determined that the panel to be displayed is the direction sign detected.

The presence of characters is implicit when a direction sign is detected.

During a fifth step 5, the liability of the directional panel to be displayed on the screen is determined. To achieve this, the following steps are applied:

the number of pixels of the characters whose presence has been detected on the cropped image of the panel to be displayed is determined by the method of recognizing the speed panels,

determining whether the determined size of the characters is greater than or equal to a predetermined minimum character size value based on Johnson Rosell's criterion with a pixel number N50 value of 50% of recognition of an empirically determined character ,

if this is the case, we confirm the possibility of a mid scale without deterioration of liibilit y on the screen by applying ergonomic rules specific to the manufacturer.

In the course of a possible sixth step 6, the image of the directional panel to be displayed is displayed at the scale determined in the fifth step. During a possible seventh step 7, an image processing (saturation, contrast) of the direction panel to be displayed and scaled is performed.

In all cases, the method continues with the display on the screen of the direction panel to be displayed, the panel display being maintained as long as the panel remains present on the acquired image.

Alternatively, the method displays at least one other type of road signs, for example an indication panel of fixed speed control radars.

Figure 2 illustrates the main elements of a roadside sign display system for a motor vehicle. It can be seen that the display system comprises a calculator 10 comprising calculation means (10a, 10b, 10c, 10d), input connected to a camera 11 so as to acquire images of the road scene situated in front of the vehicle, the computer being connected at the output to a screen 12 disposed in the passenger compartment for displaying information to the driver during the use of the vehicle.

The computer 10 comprises a first calculation means 10a able to analyze the image received from the camera 11 in order to detect the presence of traffic signs. A calculation means 10a may be a means for recognizing the speed limit signs. Indeed, such a means is able to determine the presence of road signs and then among these panels, those indicating a speed limit. It is thus possible to use the intermediate result of such a means to have road signs of the scene.

The first calculation means 10a is outputted to a second calculation means 10b capable of detecting the presence of direction panels among the detected panels.

For this, the calculation means 10b is able to determine the shape of the panels detected by known means of the state of the art and to compare the determined forms to a database of forms of direction signs. As a variant, the calculation means 10b is also able to determine whether a signaling panel is a direction panel by determining the presence of a direction arrow by comparing it with a database of the direction arrows present on the steering arrows. direction signs. The second calculation means 10b is connected at the output to a fourth calculation means l Od able to evaluate the liability of the characters detected which will be described later.

According to a second embodiment illustrated in FIG. 3, the system comprises a third computing means 10c connected at input to the second calculation means I Od and connected at the output to the fourth calculation means I Od able to evaluate the accessibility. detected characters.

The third calculation means 10c is able to determine the position of said panels with respect to the different traffic lanes on the road and to detect the characters present on said panels. The third calculation means 10c may be an AFIL (Lane departure warning System) line crossing warning system or a line maintenance system (in the English language). "Lane Keeping Assi st") known from the state of the art.

In order to determine the position of said panels relative to the different traffic lanes on the road, the calculation means 10c is able to determine whether the road comprises one or more traffic lanes. When the road comprises several traffic lanes and several directional panels are detected on the central part of the road above each lane, the calculating means 10c is able to determine the directional sign to be displayed by determining the presence of lanes according to the signaling lines present on the road, by associating the detected directional signs with a lane according to the respective positions of the panels and the lanes, by determining the lane in which the vehicle is traveling, by choosing panel to display the direction sign among the panels detected direction associated with the lane in which the vehicle is traveling.

When the road comprises a single traffic lane, the calculating means 10c is able to determine that the panel to be displayed is the direction panel detected.

Furthermore, the presence of characters is implicit when a direction sign is detected.

Whatever the embodiment, the calculation means l 0 is capable of determining the number of pixels of the characters whose presence has been detected on the cropped image of the panel to be displayed received from the first calculation means 10 a, determining whether the determined size of the characters is greater than or equal to a predetermined minimum character size value based on Johnson Rosell's criterion with a pixel number s corresponding to 50% recognition of an empirically determined character, and if this is the case, to confirm the possibility of scaling without impairing the accessibility on the screen according to ergonomic rules specific to the manufacturer.

The present invention can be applied to any type of panel and ensures a comprehensible display by the driver of the image captured by the camera.

Claims

1. A method of displaying a direction sign of the road scene for a motor vehicle equipped with a camera (11), a screen (12) and a first calculation means (10a) adapted to perform recognition of speed signs,

the camera (11) being fixed on the motor vehicle so as to acquire images of the road scene in front of the vehicle,

the screen (12) being arranged so as to be visible to the driver during the use of the vehicle,

characterized by the fact that it comprises the following steps: one acquires, without digital zoom, an image of the scene observed with the camera (11),

detecting whether at least one traffic sign is present on the image of the road scene observed by the first calculating means (10a),

if this is the case, it is determined whether at least one direction sign is present among the traffic signs detected via the first calculation means (10a),

2. Method according to claim 1, wherein, to determine if at least one direction panel is present among the detected signaling panels, the shape of the panels detected by recognition of the speed panels is determined and compares the determined forms to a database of steering panel shapes.

Method according to any one of the preceding claims, wherein it is determined whether a signaling sign is a direction sign by determining the presence of a direction arrow by comparison with a steering arrows database. present on direction signs.

4. A method according to any one of the preceding claims, wherein, when the road comprises several traffic lanes, the panel to be displayed is determined by carrying out the following steps:

the presence of tracks is determined according to the signaling lines present on the road by means of an unintentional line crossing warning or line maintenance assay procedure carried out by a calculation system. equipping the vehicle,

The route in which the vehicle is traveling is determined, and it is selected as the panel to display the steering panel among the detected direction signs associated with the lane in which the vehicle is traveling.

5. A method according to any one of the preceding claims, wherein, when the road comprises a single traffic lane, it is determined that the panel to be displayed is the detected direction panel.

A method according to any one of the preceding claims, wherein, to determine the liability of the directional panel to be displayed, the pixel number size s of the characters whose presence is detected on the cropped image of the panel to display the recognition of the speed panels, it is determined whether the determined size of the characters is greater than or equal to a predetermined minimum size value of character according to Johnson Rosell's criterion with a number of pixels corresponding to 50% recognition of an empirically determined character, and

if this is the case, it confirms the possibility of scaling without degradation of the readability on the screen according to ergonomic rules.

7. A display system of a road scene direction panel for a motor vehicle equipped with a camera (11), a screen (12) and a first calculation means (10a) adapted to perform recognition of speed signs,

the screen (12) being arranged so as to be visible to the driver during the use of the vehicle, the system being characterized by the fact that it comprises

the first calculation means (10a) being able to detect the presence of signaling panels on the image received from the camera (H),

a second calculating means (10b) able to determine if at least one direction sign is present among the detected traffic signs,

fourth calculation means (10d) capable of determining the readability of the characters of the direction panel to be displayed on the screen of the vehicle,

the fourth calculating means (10d) being further capable of scaling the image of the directional panel to be displayed as well as scaling the image, and maintaining the image display on the screen (12) of the direction panel to be displayed, as long as the direction sign remains on the acquired image,

the first calculating means (10a) being connected at the output to the second calculating means (10b) itself connected at the output to the fourth calculating means (10d), the fourth calculating means (10d) being connected at the output to the screen (12).

The system of claim 7, wherein the second calculating means (10b) is adapted to determine whether a signaling panel is a direction panel by comparing the shape of the signaling panel with a database of the forms of the signals. direction signs.

9. System according to any one of claims 7 or 8, wherein the second calculating means (10b) is able to determine if a signaling panel is a direction panel by determining the presence of a direction arrow by compared with a database of direction arrows on direction signs.

10. A system according to any one of claims 7 to 9, comprising a third calculation means (10c) capable of realizing an unintentional crossing warning line or a line maintenance assistance connected between the second means of calculation (10b) and the fourth calculating means (10d), the third calculation means (10c) is able to determine whether the road comprises one or more traffic lanes, and able to determine the directional panel to display according to respective positions of the panels, tracks and the vehicle.

1 1. A system according to any one of claims 7 to 10, wherein for determining the legibility of the characters of the direction panel to be displayed on the vehicle screen, the fourth calculating means (10d) is able to determine the number size. pixel s characters whose presence has been detected on a cropped image of the panel to be displayed i ssue the first calculation means (10a), to determine if the determined size of the characters is greater than or equal to a minimum value of a predetermined size of character according to Johnson Rosell 's criterion with a pixel number s corresponding to 50% recognition of an empirically determined character and if so, to confirm the possibility of setting. scale without degradation of readability on the screen according to ergonomic rules.