DE102017211081A1 - Apparatus and method for finding a change in a scene - Google Patents

Abstract

The invention relates to a device (1) for detecting changes in a scene (100). A first determination device (10) determines, based on three images (P1, P2, P3) to which recording times (T1, T2, T3) are assigned and which have different perspectives, data of a trifocal senor. The recording times (T1, T2) of two pictures (P1, P2) belong to a first time interval (I1) and the recording time (T3) of a third picture (P3) belongs to a second time interval (I2). A time interval (dt1) between the acquisition times (T1, T2) of the two images (P1, P2) of the first time interval (I1) is less than a time interval (dt2) between the first (I1) and the second time interval (I2) , A second determination device (20) determines data of a pixel-by-pixel assignment for the two images (P1, P2) of the first time interval (I1). An image processing device (30) provides two comparison images (C1, C2) showing the scene (100) from the same perspective, starting from the three images (P1, P2, P3), the data of the trifocal sensor, and the data of the pixel-by-pixel assignment. A comparison device (40) evaluates the comparison images (C1, C2) as to whether there is a difference. Furthermore, the invention relates to a corresponding method.

Description

The invention relates to a device for finding at least one change in a scene. Furthermore, the invention relates to a method for finding at least one change in a scene.

Currently, image-based change detection requires that the images that depict a scene before and after an event have been taken from the same perspective as possible. However, this can only be guaranteed in exceptional cases.

If images show a clear difference in their perspective, depending on the nature of the scene, surfaces are only visible in one image (occlusions) and parallax effects occur. Concerning the occlusion, it is unclear which image regions are comparable for a change detection. With respect to parallax effects, geometric estimates or detailed models of the scene are required to compare the corresponding image regions.

Some approaches are briefly outlined below.

D-change detection

One possibility is described in the description of the geometry of the two-image case, that is, the case of two images of an object or a scene, by means of homography.

Homography as a geometric model of the two-image case allows pixel-by-pixel mapping between two images without the need for complex dense depth estimation techniques.

However, the pixel-by-pixel assignment on the basis of homography is correct only in three special cases over all image regions, namely with identical recording position, infinitely large distance to the scene or a completely flat scene. For most applications, none of these special cases apply. Correspondingly, as a rule, correction methods are used which include the epipolar geometry and can largely compensate for errors due to parallax effects. The problem of occlusion is conceptually insoluble with this approach.

The most commonly used method for pixel-wise mapping for change detection is the generation of 2D image mosaics (stitching), as described by Szeliski (2006). Before and after pictures of a scene are combined to form a large picture without considering the geometry of the scene, and then compared.

However, such a global approach has significant conceptual weaknesses by ignoring the geometry of the scene, and therefore leads to acceptable results only in exceptional cases.

Conceptually similar is the generation of true ortho images instead of image mosaics. In this case, using absolute orientation and dense depth estimation or an existing elevation model, a composite image is created that takes into account the geometry of a scene. The overall picture is processed as if it were completely taken from nadir view. With a correspondingly accurate procedure, the true ortho images can be compared for a change detection.

The problem is that the reliability and accuracy of the dense depth estimation is generally not the same for all image regions. When comparing for change detection, a plausibility check must be included accordingly. In addition, the determination of the absolute orientation and the dense depth estimation is computationally intensive, so that a real-time capable realization with corresponding reliability and accuracy hardly seems possible.

Use of 2.5D and 3D reference models

The use of 2.5D (elevation models) or full 3D surface reference models of the scene allows dealing with the problem of parallax effects as well as occlusions.

The images that depict a scene before and after an event are registered against the reference model. In this way, it can be determined directly on the basis of the visual rays, which surfaces are visible in which image and what distance they had when shooting from the camera.

The disadvantage is that the demands on the level of detail and the recording time of the reference models are very high and can be guaranteed only for a few applications.

volume estimation

Image-based, Euclidean 2.5D or full 3D surface reconstruction techniques allow the modeling of a scene so that the input or output of a scene is enhanced Decrease in volume can be used as an indicator of change detection.

However, the requirements for image registration, dense depth estimation, and surface reconstruction for the sensors used, as well as accuracy and performance, may be high to very high. In addition, a precise knowledge or estimation of the intrinsic camera parameters is mandatory.

In principle, all surfaces reconstructed by such methods have a noise, so that the changes sought must stand out clearly in order to be reliably recognized.

Possible applications of change detection include the monitoring of vulnerable infrastructure for civil security and defense. In this case, for example, images of manned or unmanned aerial vehicles can be used. Another application is, for example, damage assessment for the insurance industry based on satellite and aerial imagery.

The invention is therefore based on the object of proposing a device and a method for finding changes in a scenery, which are as simple as possible.

The invention achieves the object by a device for finding at least one change in a scene.

The device has a first determination device, a second determination device, an image processing device and a comparison device. The first determination device is designed to determine data from a trifocal sensor based on at least three images of the scene. The three pictures are assigned recording times. The recording times of two pictures of the three pictures belong to a first time interval. The two images of the first time interval each have different perspectives relative to the scenery. The recording time of a third image of the three images belongs to a second time interval. The third image of the second time interval has a different perspective relative to the scene than the two images of the first time interval. A time interval between the acquisition times of the two images of the first time interval is smaller than a time interval between the first time interval and the second time interval. The second determination device is designed in such a way, at least for the two images of the first time interval, to determine data of a pixel-by-pixel association between the two images. The image processing apparatus is configured to provide at least two comparison images based on the at least three images and on the determined data of the trifocal sensor and the determined data of the pixel-by-pixel assignment. The at least two comparison images have substantially the same perspective on the scene and are assigned to different time intervals. The comparison device is designed in such a way to evaluate the two comparison images as to whether at least one difference exists between the two comparison images, and to generate a comparison result on the basis of the evaluation.

The device evaluates at least three images of the scenery, each showing the scene from different perspectives. Two of the images come from a first time interval and are relatively close to each other in terms of their recording times or have even been recorded at the same time. The remaining third image comes from a second time interval. The two time intervals are farther apart than the two images of the first time interval. For example, an event takes place between the two time intervals that leads or could lead to a change in the scene. Whether there was a change is determined by the device.

For finding the change or the changes, the data of a trifocal sensor are determined for the three images, and data of a pixel-by-pixel assignment is determined for the two images, which are simultaneous or temporally adjacent. With the data then two comparison images are determined with the same perspective and from the two different time intervals. From the two comparative images with suitable perspectives but different time intervals, changes can then be recognized.

In one embodiment, it is provided that the device has a correction device. In this case, the correction device is configured to recognize distortion errors in input images and to provide distortion-free images and / or correction data. In this refinement, lens correction errors are thus corrected by the correction device or at least data are determined which allow the further steps to take into account the effects of lens aberrations.

An embodiment consists in that the first determination device is designed in such a way to use the pixel information with regard to at least one image for the determination of the data of the trifocal sensor.

Alternatively or additionally, there is an embodiment in that the first determination device is configured in such a way for determining the Trifocal Values Data should be used for at least one image condition of at least one image.

In one embodiment, it is provided that at least one recording condition relates to a position of a recording device used for recording an image relative to the scenery or absolutely.

Alternatively or additionally, the at least one recording condition relates to an orientation of a recording device used to record an image relative to the scenery or absolutely.

Alternatively or additionally, the at least one recording condition relates to intrinsic parameters of a recording device used to record an image.

When determining the data of the trifocal sensor, recording conditions are used - or more specifically: data describing these recording conditions. Particular consideration is given to the recording device -. As the camera - for the generation of images.

Depending on the design or the availability of the data, the recording conditions therefore relate to the position of the recording device relative to the scenery and / or to the orientation of the recording device relative to the scenery. Thus, it is relevant where the recording device is relative to the scenery or absolutely and / or how the recording device - or in particular its optics - relative to the scenery or absolutely aligned.

In this case, the intrinsic parameters of the recording device itself are to be understood as recording conditions. The influence of the internal properties of the recording device is thus also taken into account as a supplement or alternatively for the determination of the data of the trifocal senor.

An embodiment is that the images of the first time interval are before images and the image of the second time interval is a post-image.

In one embodiment, it is provided that the images of the first time interval are after-images and the image of the second time interval is a before-image.

The terms before and after pictures refer to the larger time interval between the two time intervals and / or before and after an event. Furthermore, depending on the embodiment, the before images belong to the first time interval or to the second time interval. Accordingly, the after-pictures belong to the second time interval or to the first time interval.

One embodiment is that the recording times of the images of the first time interval are almost or exactly the same. In this embodiment, the time interval between the two images of the first time interval is thus shortened as possible.

Furthermore, the invention achieves the object by a method for finding at least one change in a scene.

• • Dass mindestens drei Bilder bereitgestellt werden, die Aufnahmezeitpunkten zugeordnet sind.
• • Dass ausgehend von den mindestens drei Bildern Daten eines Trifokaltensors oder einer äquivalenten Beschreibung der Geometrie des Dreibildfalls ermittelt werden.
• • Dass mindestens für die zwei Bilder des ersten Zeitintervalls Daten einer pixelweisen Zuordnung zwischen den zwei Bildern ermittelt werden.
• • Dass ausgehend von den mindestens drei Bildern sowie ausgehend von den ermittelten Daten des Trifokaltensors bzw. der äquivalenten Beschreibung und den ermittelten Daten der pixelweisen Zuordnung mindestens zwei Vergleichsbilder, die im Wesentlichen die gleiche Perspektive auf die Szenerie aufweisen und unterschiedlichen Zeitintervallen zugeordnet sind, bereitgestellt werden.
• • Dass die zwei Vergleichsbilder dahingehend ausgewertet werden, ob mindestens ein Unterschied zwischen den zwei Vergleichsbildern besteht.
• • Dass ausgehend von der Auswertung der zwei Vergleichsbilder ein Vergleichsergebnis erzeugt wird.

The method comprises at least the following steps:

• • That at least three images are assigned, which are assigned to recording times.
• • That, on the basis of the at least three images, data of a trifocal sensor or an equivalent description of the geometry of the triple-fall case is determined.
• • That at least for the two images of the first time interval, data of a pixel-wise association between the two images are determined.
• • That at least two comparison images, which have substantially the same perspective on the scene and are assigned to different time intervals, are provided on the basis of the at least three images as well as on the basis of the determined data of the trifocal tensor or the equivalent description and the determined data ,
• • That the two comparison images are evaluated to determine if there is at least one difference between the two comparison images.
• • That a comparison result is generated starting from the evaluation of the two comparison images.

• • Die Aufnahmezeitpunkte von zwei Bildern der drei Bilder gehören zu einem ersten Zeitintervall und der Aufnahmezeitpunkt eines dritten Bildes der drei Bilder gehört zu einem zweiten Zeitintervall.
• • Die zwei Bilder des ersten Zeitintervalls weisen unterschiedliche Perspektiven relativ zur Szenerie auf.
• • Das dritte Bild des zweiten Zeitintervalls weist eine andere Perspektive relativ zu der Szenerie als die zwei Bilder des ersten Zeitintervalls auf.
• • Ein zeitlicher Abstand zwischen den Aufnahmezeitpunkten der zwei Bilder des ersten Zeitintervalls ist kleiner als ein zeitlicher Abstand zwischen dem ersten Zeitintervall und dem zweiten Zeitintervall.

The three images are provided in this way, that is, for example, appropriately selected or generated, that the following conditions are met:

• • The recording times of two pictures of the three pictures belong to a first time interval and the recording time of a third picture of the three pictures belongs to a second time interval.
• • The two images of the first time interval have different perspectives relative to the scenery.
• The third image of the second time interval has a different perspective relative to the scene than the two images of the first time interval.
• • A time interval between the recording times of the two images of the first time interval is smaller than a time interval between the first time interval and the second time interval.

The above embodiments of the device can also be realized by steps of embodiments of the method, so that the explanations apply accordingly. Therefore, a repeat is omitted here.

Finally, the invention relates to a computer program with a program code for carrying out the aforementioned method according to one of the embodiments.

So far presented 2D-change methods have basically been limited to the geometry of the two-image case. This procedure has a decisive disadvantage for the change detection: Even when geometrical plausibility is taken into account by the epipolar geometry and dense depth estimation, it is indistinguishable when performing the change detection whether a surface is visible in only one image or if there is a change in the scene. The use of three images and the determination of the geometry of the three-frame case enables geometrically correct image transformation, even without a Euclidean reconstruction.

The solution described below uses the following conditions:

The invention is summarized again in other words.

Step 1: Determination of thin point correspondences

The reliable determination of point correspondences (homologous points) is also possible with recording configurations for which a pixel-wise assignment is not meaningful (Stewart 1999), (Mikolajczyk & Schmid 2004).

This is part of the determination of the data of the Trifokaltensors.

Methods for the automatic determination and assignment of point correspondences in images are described, for example, by (Förstner & Gülch 1987), (Harris & Stephens 1988), (Lowe 2004) or (Rublee et al., 2011).

Describe methods for compensating misallocations (Fischler & Bolles 1981).

In one embodiment, methods for the local enhancement of image coordinates are used, cf. (Green 1985) or (Bethmann & Luhmann 2010).

In one embodiment, methods for robust estimation of a geometric model are also used, cf. (Torr 1998) or (Stewart 1999).

Step 2: Determination of the geometry of the triple fall

Based on point correspondences (homologous points), the geometry of the three-frame case can be reliably determined (Hartley & Zisserman 2004).

This step is also part of determining the data of the Trifocal tensor.

• - sind folgende Ausgestaltungen vorgesehen, die Alternativen oder Ergänzungen zueinander darstellen:
  • In einer Ausgestaltung erfolgt die Bestimmung des unkalibrierten Dreibildfalls durch Sechs-Punkt-Korrespondenz und optional durch eine kleinste Quadrate-Optimierung, vgl. (Quan 1994), (Torr & Zisserman 1997) oder (Schaffalitzki et al. 2000).

For the determination of the geometry of the three-frame case - ie the processing of the three pictures

• the following configurations are provided, which represent alternatives or additions to one another:
  • In one embodiment, the determination of the uncalibrated three-frame case is effected by six-point correspondence and optionally by a least squares optimization, cf. (Quan 1994), (Torr & Zisserman 1997) or (Schaffalitzki et al., 2000).

In one embodiment, the determination of the calibrated three-frame case is based on two calibrated two-image cases, cf. (Carlson & Weinshall 1998, Nister 2005). In addition, in one embodiment, a least squares optimization takes place, cf. (Schaffalitzki et al., 2000).

In one embodiment, projection matrices are used which are present as a result of a relative or absolute orientation of a multiple image case.

In this case, in one embodiment, three projection matrices are selected and the Trifokaltensor is determined directly. For details see (Hartley & Zisserman 2004). In one embodiment, a least-squares optimization optionally takes place, cf. (Schaffalitzki et al., 2000).

In a further embodiment projection matrices are used which are given by suitable sensors and preferably their GNSS data (GNSS from global navigation satellite system, global navigation satellite system). The data of the trifocal senor are also determined directly from this, cf. (Hartley & Zisserman 2004). Optionally, a least squares optimization is performed, cf. (Schaffalitzky et al., 2000).

In one embodiment, an extension to the determination and corresponding use of a quadrifocal senors takes place, cf. (Triggs 1995, Hartley 1998) as long as there are four or more pictures showing the scene.

Step 3: Determination of dense point correspondences (pixel-wise assignment)

For an image triplet, a pair of images is acquired in such a way that it is as suitable as possible for a pixel-by-pixel assignment. The acquisition of this image pair is not bound to the recording time of the third image. Both monocular and stereo recording configurations are possible. The pixel-by-pixel assignment for the image pair, which depicts the scene at a virtually identical point in time, takes place in one embodiment as a global solution z. In the form of an optical flow, with or without the inclusion of geometric plausibility, or by non-global depth estimation, cf. (Lucas & Kanade 1981), (Brox et al., 2004) or (Hirschmüller, 2008). For the latter approach, planar and polar rectification of the images is equally conceivable (Pollefeys et al., 2004).

Step 4: Align the perspective between before and after pictures.

In this step, using the determined Trifokaltensors and the pixel-by-pixel assignment for the image pair, which images the scene at a virtually identical time, the transformation of one or more perspectives of the three images is made, see. (Avidan & Shashua 1997), (Connor & Reid 2002).

In one embodiment, a relative or absolute orientation in conjunction with an existing, calculated or determined by RGB-D cameras coarse height model is used for a rough transformation of the perspectives of the images.

The images for which a change detection is to be performed are available without distortion, are prepared without distortion or the distortion is modeled (Buchdahl 1958 ), (Hartley & Zisserman 2004 ), (Barreto & Daniilidis 2005 ), (Brito et al., 2013) or, analogously to (Brito et al., 2013), the directory parameters are determined directly with the trifocal senor (radial trifocal senor).

Consider the case with a before-picture and two after-pictures, ie an image belonging to the second time interval and two pictures belonging to the first time interval. The time interval between the two subsequent pictures is negligible. The following explanations apply mutatis mutandis to the reverse constellation that there are two before-pictures at the time t0 and one after-picture at the time t1. Also, the explanations apply to the case where additional before images and / or after images exist. This is z. Example, the case when the images of a video sequence or video sequences come from.

An existing reference image (before image) is used, this recording time is referred to as t0 in the following.

Use or capture of two suitable, current images (after images) for which a pixel-wise assignment is meaningfully possible, these include in particular stereo recording configurations. The recording times must either be identical, which is the case with stereo recording configurations, or it must be assumed that there is no relevant change in the scene between the recording times. This recording time is referred to below as t1.

• • Es wird eine 2D-Änderungsdetektion ermöglicht, die nicht von Referenzmodellen abhängig ist. Es ist auch keine Veränderung des Volumens erforderlich, um eine Identifizierung zu erlauben.
• • Die Bestimmung der Geometrie des Dreibildfalls erfordert für diese Anwendung keine euklidische Rekonstruktion, sondern lediglich eine projektive. Daher können alle Verfahrensschritte unkalibriert, d. h. ohne Kenntnis der intrinsischen Kameraparameter durchgeführt werden.
• • Alle Verfahrensschritte sind echtzeitfähig.
• • Die pixelweise Zuordnung erfolgt für zwei Bilder, die aufgrund ihrer zeitlichen Nähe eine unveränderte Szenerie zeigen. Daher kann davon ausgegangen werden, dass die Zuordnung nicht durch eine Änderung der Szenerie beeinträchtigt wird. Etwaige Fehler im Disparitätsbild - also Fehler, die bei der pixelweisen Zuordnung auftreten - können ggf. dem Verfahren oder der Aufnahmekonfiguration zugerechnet und somit nicht mehr fälschlich als Änderungen gedeutet werden. Für die betroffenen Bildregionen wäre somit bekannt, dass eine Aussage über etwaige Änderungen nicht möglich ist.
• • Verdeckungen führen bei der Transformation der Perspektive zu leeren Bildregionen, so dass feststellbar ist, wo Information fehlt und eine Aussage über etwaige Änderungen nicht möglich ist.
• • Ein großer Vorteil besteht darin, dass die Erfindung auf Grundlage von Consumer-Kameras, rein bildbasiert und ohne Expertenwissen umsetzbar ist, sodass auch bei sehr geringen Hardwarekosten zuverlässige Ergebnisse erzielt werden können.

Some advantages of the invention are the following:

• • It allows 2D change detection that does not depend on reference models. Also, no change in volume is required to allow identification.
• • The determination of the geometry of the three-dimensional case does not require a Euclidean reconstruction for this application, but only a projective one. Therefore, all process steps can be performed uncalibrated, ie without knowledge of the intrinsic camera parameters.
• • All process steps are real-time capable.
• • The pixel-by-pixel assignment is done for two images which, due to their temporal proximity, show an unchanged scene. Therefore, it can be assumed that the assignment is not affected by a change in the scenery. Any errors in the disparity image - that is, errors that occur in the pixel-wise assignment - can possibly be attributed to the procedure or the recording configuration and thus no longer be interpreted as changes wrongly. For the affected image regions would thus be known that a Statement about any changes is not possible.
• • When the perspective is transformed, obscurations lead to empty image regions, so that it is possible to determine where information is missing and if it is not possible to make any statements about any changes.
• • A big advantage is that the invention can be implemented on the basis of consumer cameras, purely image-based and without expert knowledge, so that reliable results can be achieved even at very low hardware costs.

• 1 eine schematische Darstellung einer photographischen Aufnahme einer Szenerie,
• 2 eine schematische Darstellung der zeitlichen Verteilung von drei Bildern einer Szenerie,
• 3 eine schematische Darstellung einer Ausgestaltung einer Vorrichtung zur Identifizierung einer Veränderung in einer Szenerie,
• 4 eine schematische Darstellung einer ersten Variante der Ermittlung von Daten eines Trifokaltensors (a)) und Daten einer pixelweisen Zuordnung (b)),
• 5 eine schematische Darstellung einer zweiten Variante der Ermittlung von Daten eines Trifokaltensors (a)) und Daten einer pixelweisen Zuordnung (b)),
• 6 eine schematische Darstellung einer dritten Variante der Ermittlung von Daten eines Trifokaltensors (a) oder b)) und Daten einer pixelweisen Zuordnung (c)),
• 7 eine schematische Darstellung mehrerer Varianten einer unidirektionalen, einfachen Transformation zwischen zwei Bildern,
• 8 eine schematische Darstellung mehrerer Varianten einer unidirektionalen, doppelten Transformation zwischen drei Bildern,
• 9 eine schematische Darstellung mehrerer Varianten einer bidirektionalen, einfachen Transformation zwischen zwei Bildern,
• 10 eine schematische Darstellung mehrerer Varianten einer bidirektionalen, doppelten Transformation zwischen drei Bildern und
• 11 eine beispielhafte Realisierung des Verfahrens als Ablaufdiagramm.

In particular, there are a variety of ways to design and further develop the device and the corresponding method. Reference is made on the one hand to the claims, on the other hand to the following description of embodiments in conjunction with the drawings. Show it:

• 1 a schematic representation of a photograph of a scene,
• 2 a schematic representation of the temporal distribution of three images of a scene,
• 3 a schematic representation of an embodiment of a device for identifying a change in a scenery,
• 4 a schematic representation of a first variant of the determination of data of a Trifokaltensors (a)) and data of a pixel-by-pixel assignment (b)),
• 5 a schematic representation of a second variant of the determination of data of a Trifokaltensors (a)) and data of a pixel-by-pixel assignment (b)),
• 6 a schematic representation of a third variant of the determination of data of a Trifokaltensors (a) or b)) and data of a pixel-by-pixel assignment (c)),
• 7 a schematic representation of several variants of a unidirectional, simple transformation between two images,
• 8th a schematic representation of several variants of a unidirectional, double transformation between three images,
• 9 a schematic representation of several variants of a bidirectional, simple transformation between two images,
• 10 a schematic representation of several variants of a bidirectional, double transformation between three images and
• 11 an exemplary implementation of the method as a flowchart.

In the 1 is represented as from a scenery 100 with a recording device 200 , which is a camera here by way of example, pictures are taken.

The temporal assignment of three pictures of such a scene 100 is in the 2 shown.

The three pictures are with it P1 . P2 and P3 referred to at the recording times T1 . T2 and T3 been recorded. Two pictures P1 and P2 belong to a first time interval I1 and the third picture P3 belongs to a second time interval 12 ,

The embodiment shown is two before-pictures and one after-picture. However, this is only an example because the method or apparatus described herein is also applicable to a before-picture and two-after-picture. The same applies to the case that even more before-pictures or after-pictures are available.

Between the two pictures P1 and P2 the first time interval I1 there is a time gap here dt1 which is smaller than the distance dt2 between the two time intervals I1 and I2 , In one embodiment, the time interval dt1 between the pictures P1 and P2 almost zero, so the pictures P1 and P2 that is, they were essentially recorded at the same time. In the pictures P1 and P2 It is assumed that there is no change in the scenery 100 between recording times T1 and T2 has set. Therefore, the time interval dt1 preferably as small as possible. The three pictures P1 . P2 and P3 They generally differ from each other in that they are the scenery 100 from different perspectives.

The third picture P3 , here also the picture of the second time interval I2 and a after-image is at another time T3 as the before pictures P1 and P2 added: T1 and T2 , The perspective of the third picture P3 is different to the perspectives of the other two pictures P1 and P2 ,

For identifying a change of scenery 100 thus at least three images are used. Two pictures - here P1 and P2 - show the scenery 100 from two different perspectives and have different recording times T1 and T2 , The third picture P3 also shows the scene, but has a significantly different recording time T3 , The time interval dt2 between the time intervals I1 . I2 in which the three pictures P1 . P2 and P3 lie, is arbitrary, but is at least greater than the time interval dt1 between the pictures P1 . P2 the first time interval I1 , Also arbitrary is whether the two pictures P1 . P2 time before or after the recording time T3 of the third picture P3 lie. Thus, the first time interval I1 also the second time interval I2 consequences.

The 3 schematically shows an embodiment of a device 1 for processing three input pictures P1 ' P2 ' and P3 'as well as the corresponding identification of changes in the scenery 100 based on the pictures.

The three input pictures P1 ' P2 ' and P3 'Be in the illustrated embodiment, first of a correction device 50 fed, which detects possible distortion errors and corrected appropriately. In the illustrated embodiment, the correction device gives 50 corrected pictures P1 . P2 . P3 out. In an alternative embodiment, the correction device determines 50 Correction data used in further steps.

The three pictures P1 . P2 . P3 will be sent to the first investigator 10 transmitted from the pictures P1 . P2 . P3 Data of a Trifokaltensors determined.

A tri-fold sorcerer is a tensor, which has geometrical relationships between the images of the three images P1 . P2 . P3 describes. In one variant, projection matrices are used for this purpose, the respective recording z. B. describe with a camera.

The two pictures P1 and P2 that at the same time interval I1 also become the second investigative device 20 fed. Between the two pictures P1 and P2 there is only a small difference in time dt1 , In one embodiment, the two images P1 and P2 recorded at the same time. Both pictures P1 and P2 but show the scenery 100 from different perspectives.

The second investigative device 20 determines data of a pixel-wise assignment between the two images P1 and P2 , It is thus determined which pixels of the two images P1 and P2 belong to each other.

Finally, the three pictures P1 . P2 . P3 as well as the data of the Trifokaltensors and the data of the pixel-by-pixel assignment of the image processing device 30 fed.

The image processing device 30 is designed in such a way that it starts from the three perspectively unadjusted images P1 . P2 . P3 as well as the data regarding the Trifokaltensor and the pixelwise assignment at least two comparison images C1 and C2 that creates the scenery 100 show from the same perspective.

The comparison pictures C1 . C2 come from the two different time intervals I1 and I2 , In one embodiment is a comparison image C1 equal to a perspective not aligned image - z. For example, the picture P1 from the first time interval I1 - and the other comparative image results from another perspective not aligned image - corresponding to the image P3 from the second time interval I2 - with changed perspective. Alternatively, both time intervals become I1 and I2 Transformed images into a common perspective.

The two comparison pictures C1 and C2 are then a comparison device 40 fed to the two comparison images C1 and C2 compares with each other and generates a comparison result that provides information about changes in the scenery 100 gives.

As already noted, three images are required, showing the scene from different perspectives, where two images have been taken close to each other in time, or even at the same time, and the third image has a different recording time. The two images recorded temporally close to one another are preferably generated or selected in such a way that their recording conditions are as well suited for pixel-by-pixel assignment.

It has also been mentioned that the two pictures can be temporally in front of or behind the third picture. Therefore, the following statements are to be understood accordingly in temporal reversal.

Furthermore, it is assumed in the following embodiments that two images have the same recording time. However, the corresponding embodiments also apply to the case of a time interval, wherein the distance dt1 between the pictures smaller than the distance dt2 between the time intervals I1 and I2 is.

The 4 shows three pictures here with P1 . P2 and P3 are designated and to which the recording times t0 and t1 assigned. Thus, two pictures are: P2 and P3 been recorded at the same time. So these two pictures belong P2 and P3 also at the first time interval I1 in the example shown, the second time interval I2 follows. There are thus two after-pictures P2 and P3 and a before-picture P3 in front.

The 4 a) illustrates the determination of the data of - in this case in particular uncalibrated - Trifokaltensors, wherein at least six point correspondences on the three images P1 . P2 . P3 be determined.

The 4 b) shows the determination of the pixel-by-pixel assignment for the two after-pictures P2 . P3 , that is, for the images that have the same time interval I1 come from and look at the perspective differ from each other. The strokes indicate that significantly more pixels are detected than in the determination of the trifocal scores - 4 a) - the case is.

The same procedure applies to the case of two before-pictures and one after-picture.

The 5 shows the case that there are n total pictures: P1 . P2 to Pn. These are ( n-1 ) Before pictures: P1 to Pn-1 and a after-picture: Pn. where n is a natural number greater than three.

The before pictures are in the first time interval I1 been recorded and even have the same recording time here t0 , The after picture pn with the recording time t1 lies in the second time interval I2 ,

In the 5 a) is suggested as the uncalibrated trifocal sensor by six point correspondences for three of the n images P1 . pn - 1 and pn is determined. At least one picture is the after-picture pn ,

The 5 b) clarifies that a pixel-by-pixel association between all n-1 ) is determined at the same time images.

The 6 shows the case that total ( n + m ) Images are present, where n and m are natural numbers.

Thereby lie ( n-1 ) Pictures before, at a time t0 been recorded. So these are before pictures. There are also ( m + 1 ) Pictures at another time t1 been recorded. They are here after pictures.

Three images are used to determine the trifocal senor. Two pictures are taken from a common and - here designated - first time interval I1 and the third image comes from a second time interval I2 ,

This results in two options that in the 6 a) and b) are shown.

In the 6 a) is the case shown that the first time interval I1 on the after pictures at the time t1 refers. There are thus two pictures Pj and Pk where n <= j <k and k <= n + m. The single image Pi with 1 <= i <n is a before image from the time interval I2 ,

Conversely, the shows 6 b) that the two pictures pi and Pj Before pictures and therefore the first time interval I1 are assigned. The third picture Pk is a picture after.

For the determination of the pixel-wise assignment - indicated in the 6 c) - on the one hand all ( n-1 ) Before-pictures and become on the other all ( m + 1 ) After-pictures evaluated separately. There is thus a pixel-by-pixel assignment for the before state and a pixel-by-pixel assignment for the after state. The data of both assignments become the image processing device 30 (see. 3 ).

The following illustrations relate to some exemplary transformations between the individual images by the image processing device 30 (see. 3 ). The transformations refer to the adaptation of the perspectives of the images. At least two comparison images will be made C1 . C2 (see. 3 ) with the same perspective on the scenery 100 (see. 1 ) generated.

In the 7 Three examples of a unidirectional, simple transformation are presented. In each case, one image - from which the arrow starts - is adapted with respect to the perspective to another image - on which the arrow ends -.

There are three pictures each: P1 . P2 and P3 , in total at two different recording times: t0 and t1 been recorded. Two pictures have been taken at the same time: t0 or t1 , However, the links shown also apply to the case in which two images between the recording times a time interval dt1 (see. 2 ) consists. Or in other words: two images belong to the first time interval I1 and have a certain - small - or no time interval and a picture belongs to a second time interval I2 whose time interval dt2 at the first time interval I1 greater than the possibly existing distance dt1 between the recording times of the images of the first time interval I1 is. This relationship is given in particular when the images of the first time interval I1 have identical recording times, so so dt1 = 0 applies.

In the 7 a) is a before picture P1 with the recording time t0 and are two after pictures P2 and P3 with a recording time t1 given. There - for the description - the two timely or simultaneous pictures P2 and P3 the first time interval I1 is assigned, here goes the second time interval I2 with the temporally spaced image P1 the first time interval I1 ahead.

In the case shown, the picture becomes P1 on the picture P3 pictured in terms of perspective. In one alternative case, the adjustment of the perspective of the image P1 to the perspective of the picture P2 ,

Accordingly, in the 7 b) case shown when two before pictures P1 and P2 - in the first time interval I1 - with the common recording time t0 and a picture after P3 - in the second time interval I2 - given is. Here, too, becomes the single picture P3 an image of the other two - and here at the same time - pictures P1 or P2 customized. Alternatively, here is the picture P3 to the picture P1 be adjusted.

Conversely, a picture can also be P1 the two pictures of the first time interval I1 to the single picture of the other time interval I2 be adjusted. An example of such a before-after transformation shows the 7c) ,

Alternatively, the picture P2 to the picture P3 be adjusted. The same applies in the case that a before-picture and two after-pictures are present.

In the 8th Two exemplary variants for a unidirectional, double transformation are shown. The illustrated examples can be applied analogously to other constellations as in the previous figures.

The 8 a) shows the case with three pictures P1 . P2 and P3 , taking a picture P1 a recording time t0 and the other two pictures P2 and P3 another recording time t1 assigned. For the transformation thereby the perspective of the picture becomes P1 the second time interval I2 to the perspectives of the other two pictures P2 and P3 the first time interval I1 customized.

The same applies in the event that two pictures with time t0 - that is, two before pictures - and one picture with time t1 - that is, a post-picture. In this case, the individual image is also adapted to the two images, which preceded it in time.

The 8 b) shows that two pictures P1 and P2 to a picture P3 be adjusted. The two pictures P1 and P2 have the common recording time t0 and belong to the first time interval I2 , The perspectives of these pictures P1 and P2 become the perspective of the temporal image P3 the second time interval I2 customized.

The same applies to the case with a before-picture and two after-pictures.

In the 9 Two possible variants of a bidirectional, simple transformation are shown.

In the 9 a) are two before pictures P1 and P2 which the common recording time t0 is assigned, and an after-image P3 with the recording time t1 available.

The statements concerning a common recording time apply - as already mentioned and generally valid - even for slightly different recording times, insofar as the corresponding recording times are still an assignment to the first time interval I1 allow.

In the 9 a) becomes a bidirectional transformation between the images P1 and P3 performed. Alternatively, there is a transformation between the pictures P2 and P3 instead of.

The 9 b) shows the case with a before picture P1 and two after pictures P2 and P3 , Here is a transformation between the pictures P1 and P2 realized. In an alternative embodiment, a transformation between the images P1 and P3 performed.

In the 10 Two cases of bidirectional, double transformation are shown. Each transformation in perspective takes place between two images taken at different times. For the double transformation, there is a transformation between the image of the second time interval I2 and one image each of the two images of the first time interval I1 performed.

In the 10 a) are the two pictures P2 . P3 the first time interval I1 After pictures, here also a common recording time t1 assigned. Between these two pictures P2 and P3 and the single picture P1 the second time interval I2 , which is the before picture here, a bidirectional transformation is made in each case.

In the 10 b) the alternative case is shown that two before-pictures P1 and P2 and a picture after P3 given are. Therefore, in each case a bidirectional transformation between the images P1 and P3 as well as between the pictures P2 and P3 realized.

The aforementioned transformations each lead to at least two comparison images which have the same perspective.

An exemplary procedure of the method for identifying a change in a scene based on at least three images is described in US Pat 11 shown.

The exemplary course begins in the step 500 with the question whether three pictures of the scene under consideration are available without distortion.

In the positive case (Y branch), step follows 505 which represents the beginning of further steps.

In the negative case (N-branch) is in step 501 a distortion model is selected and the correction parameters are determined with respect to the distortion.

In step 502 It is crucial whether the original images should be used for further action.

If this is not the case (N-branch), then in step 503 calculated distortion-free images, with which then the step 505 follows.

Should the original images be used (Y branch after step 502 ), so in the step 504 provided the data for corrected image coordinates and used in the following steps. The first step is also here step 505 ,

In step 505 It is relevant whether the associated projection matrices are available for the three images, ie whether the necessary data are available to determine the matrices from these. The data result, for example, by the use of an inertial navigation system or by the photogrammetric determination of the relative or absolute orientation. The projection matrices describe the respective image - also called projection - of the scene on the camera image during recording. This refers to the position and / or orientation of the camera relative to the cameras with each other or absolutely to the scenery. Alternatively or additionally, this refers to intrinsic camera parameters, e.g. Camera constant, main point, shear, or scale difference.

If the projection matrices are present (Y branch after step 505 ), so in the step 506 immediately determines the data of the Trifokaltensors for the three images. Optionally, a minimum squares optimization is performed on the trifocal senor.

If the projection matrices are not available (N branch after step 505 ), so is in the step 507 the question of whether at least the intrinsic camera parameters are present.

If that is the case, then in step 508 the Trifokaltensor determined over two calibrated two cases. This means that two essential matrices are determined using the five-point algorithm. The essential matrix describes the geometry of the two-image case. Since the intrinsic camera parameters are known, a Euclidean reconstruction of the camera positions is possible. As a result, the two two-image cases can be combined to form a three-image case and the Trifokaltensor can be determined from this. This procedure is z. In Carlsson & Weinshall 1998 described.

If the intrinsic camera parameters are not available, find in step 509 an uncalibrated determination of the Trifokaltensors instead. This shows a great advantage of the method that even without the knowledge of the recording conditions, processing is possible.

In step 510 the determination of the pixel-by-pixel assignment for the image pair, which originates from the same - or here first - time interval and which has been recorded with different perspectives, takes place. For the pixel-wise assignment, it is thus advantageous if the recording times of the two images have the same or only a small time interval, so that no changes in the scenery are to be expected.

In step 511 the perspectives of at least two pictures are aligned with each other. Either the corrected unsigned images of the step 503 or the original images and the correction data of the step 504 used. For the approximation of the perspective, the data of the Trifocalsensor will also be used, depending on the availability of the data on the recording situation in one of the steps 506 . 508 or 509 have been determined. Moreover, the data becomes the pixel-by-pixel assignment of the step 510 applied.

Overall, result from the step 511 two comparison images that have the same perspective and are based in particular on images from different time intervals. This results in a before-image and a after-image as comparison images, where both comparison images have the same perspective.

In step 512 For the two comparison images, a change detection is performed.

Although some aspects have been described in the context of a device, it will be understood that these aspects also constitute a description of the corresponding method, so that a block or a component of a device is also to be understood as a corresponding method step or as a feature of a method step. Similarly, aspects described in connection with or as a method step also represent a description of a corresponding block or detail or feature of a corresponding device. Some or all of the method steps may be performed by a Hardware apparatus (or using a hardware apparatus), such as a microprocessor, a programmable computer or an electronic circuit are performed. In some embodiments, some or more of the most important method steps may be performed by such an apparatus.

Depending on particular implementation requirements, embodiments of the invention may be implemented in hardware or in software, or at least partially in hardware, or at least partially in software. The implementation may be performed using a digital storage medium such as a floppy disk, a DVD, a BluRay disc, a CD, a ROM, a PROM, an EPROM, an EEPROM or FLASH memory, a hard disk, or other magnetic or optical Memory are stored on the electronically readable control signals are stored, which can cooperate with a programmable computer system or cooperate such that the respective method is performed. Therefore, the digital storage medium can be computer readable.

Thus, some embodiments according to the invention include a data carrier having electronically readable control signals capable of interacting with a programmable computer system such that one of the methods described herein is performed.

In general, embodiments of the present invention may be implemented as a computer program product having a program code, wherein the program code is operable to perform one of the methods when the computer program product runs on a computer.

The program code can also be stored, for example, on a machine-readable carrier.

Other embodiments include the computer program for performing any of the methods described herein, wherein the computer program is stored on a machine-readable medium. In other words, an embodiment of the method according to the invention is thus a computer program which has a program code for performing one of the methods described herein when the computer program runs on a computer.

A further embodiment of the inventive method is thus a data carrier (or a digital storage medium or a computer-readable medium) on which the computer program is recorded for carrying out one of the methods described herein. The data carrier or the digital storage medium or the computer-readable medium are typically tangible and / or non-volatile.

A further embodiment of the method according to the invention is thus a data stream or a sequence of signals, which represent the computer program for performing one of the methods described herein. The data stream or the sequence of signals may be configured, for example, to be transferred via a data communication connection, for example via the Internet.

Another embodiment includes a processing device, such as a computer or a programmable logic device, that is configured or adapted to perform one of the methods described herein.

Another embodiment includes a computer on which the computer program is installed to perform one of the methods described herein.

Another embodiment according to the invention comprises a device or system adapted to transmit a computer program for performing at least one of the methods described herein to a receiver. The transmission can be done for example electronically or optically. The receiver may be, for example, a computer, a mobile device, a storage device or a similar device. For example, the device or system may include a file server for transmitting the computer program to the recipient.

In some embodiments, a programmable logic device (eg, a field programmable gate array, an FPGA) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, in some embodiments, the methods are performed by any hardware device. This may be a universal hardware such as a computer processor (CPU) or hardware specific to the process, such as an ASIC or, for example, a microprocessor, e.g. In the form of an ARM architecture.

The embodiments described above are merely illustrative of the principles of the present invention. It will be understood that modifications and variations of the arrangements and details described herein will be apparent to others of ordinary skill in the art. Therefore, it is intended that the invention be limited only by the scope of the appended claims and not by the specific details presented in the description and explanation of the embodiments herein.

references

• [1] Szeliski, R .: "Image Alignment and Stitching: A Tutorial", Now: 2006, Foundations and Trends in Computer Graphics and Vision.
• [2] Stewart, C .: "Robust Parameter Estimation in Computer Vision", SIAM Review 1999; 41 (3): 513-537 ,
• [3] Mikolajczyk, K. & Schmid, C .: "Scale & Affine Invariant Interest Point Detectors", International Journal of Computer Vision 2004; 60 (1): 63-86.
• [4] Hartley, RI, Zisserman, A .: "Multiple View Geometry in Computer Vision (2 ed.)", Cambridge University Press, New York, NY, USA, 2004. Pollefeys, M., Van Gool, L. Vergauwen , M., Verbiest, F., Cornelis, K., Tops, J., Koch, R .: "Visual Modeling with a Handheld Camera", International Journal of Computer Vision 2004; 59 (3): 207-232.
• [5] Lucas, BD, Kanade, T .: "Iterative Image Registration Technique with Application to Stereo Vision", International Joint Conferences on Artificial Intelligence 1981: 675-679 ,
• [6] Brox, T., Bruhn, A. Papenberg, N., Weickert, J .: "High Accuracy Optical Flow Estimation based on a Theory for Warping," European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, Springer, 2004 ; 3024: 25-36 ,
• [7] Hirschmüller H .: "Stereo Processing by Semiglobal Matching and Mutual Information", IEEE Transactions on Pattern Analysis and Machine Intelligence 2008; 30 (2): 328-341 ,
• [8th] Avidan, S., Shashua, A .: Novel View Synthesis in Tensor Space, in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1997: 1034-1040 ,
• [9] Buchdahl, HA: "Optical Aberration Coefficients. III. The Computation of the Tertiary Coefficients, "Journal of the Optical Society of America 1958; 48: 747-756 ,
• [10] Barreto, JP, Daniilidis, K .: "Fundamental Matrix for Cameras with Radial Distortion." In Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, IEEE Computer Society, Washington, DC, USA, 2005, pp. 625-632 ,
• [11] Brito, JH, Angst, R .; Köser, K; Pollefeys, M .: "Radial Distortion Self-Calibration", 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, 2013, p. 1368-1375 ,
• [12] Förstner, W., Gülch, E .: "A Fast Operator for Detection and Precise Location of Distinct Points, Corners and Centers of Circular Features", in Proceedings of the ISPRS Conference on Fast Processing of Photogrammetric Data, Interlaken, 1987, pp. 281-305 ,
• [13] Harris, C., Stephens, M .: "A Combined Corner and Edge Detector", Proceedings of the Fourth Alvey Vision Conference, 1988, p. 147-151 ,
• [14] Lowe, DG: "Distinctive Image Features from Scale Invariant Keypoints" in the International Journal of Computer Vision; 2004, 60 (2): 91-110 ,
• [15] Rublee, E., Rabaud, V., Konolige, K., Bradski, G .: "ORB: An Efficient Alternative to SIFT or SURF", in Proceedings of the 2011 International Conference on Computer Vision (ICCV'11), IEEE Society , Washington, DC, USA, 2011, pp. 2564-2571 ,
• [16] Fischler, MA, Bolles, RC: "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," in Journal of Communications of the ACM; 1981, 24 (6): 381-395 ,
• [17] Green. A .: "Adaptive Least Squares Correlation: A Powerful Image Matching Technique," South African Journal of Photogrammetry, Remote Sensing, and Cartography, 1985; 14: 175-187 ,
• [18] Bethmann, F., Luhmann, T .: Least-Squares Matching with Advanced Geometric Transformation Models, International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XXXVIII, Part 5 Commission V Symposium, Newcastle upon Tyne, UK , 2010 ,
• [19] Torr, PHS: "Geometric Motion Segmentation and Model Selection", Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 1998; 356 (1740): 1321-1340 ,
• [20] Quan L .: "Invariants of 6 points from 3 Uncalibrated Images", in: Eklundh JO. (EDS) Computer Vision - ECCV '94. ECCV 1994. Lecture Notes in Computer Science, vol 801. Springer, Berlin, Heidelberg
• [21] Torr, PHS, Zisserman, A .: "Robust Parameterization and Computation of the Trifocal Tensor", Image and Vision Computing 1997; 15: 591-605 ,
• [22] Schaffalitzky, F., Zisserman, A. Hartley, RI, Torr, PHS: Computer Vision - ECCV 2000: 6th European Conference on Computer Vision Dublin, Ireland, June 26 - July 1, 2000, Proceedings, Part I Springer, Berlin, Heidelberg, 2000: 632-648 ,
• [23] Carlsson, S., Weinshall, D .: "Dual Computation of Projective Shapes and Camera Positions from Multiple Images", International Journal of Computer Vision 2004; 27 (3): 227-241 ,
• [24] Nister, D .: "An Efficient Solution to the Five-Point Relative Pose Problem," IEEE Transactions on Pattern Analysis and Machine Intelligence 2004; 26 (6): 756-777 ,
• [25] Triggs, B .: The Geometry of Projective Reconstruction I: Matching Constraints and the Joint Image. Circulated in 1995. Accepted subject to revision to IJCV in 1995, but never completed. 1995 ,
• [26] Hartley, RI: "Computation of the Quadrifocal Tensor", in: Burkhardt H., Neumann B. (eds) Computer Vision - ECCV'98. ECCV 1998. Lecture Notes in Computer Science, volume 1406. Springer, Berlin, Heidelberg ,
• [27] Connor, K., Reid, I .: "Novel View Specification and Synthesis", Proceeding of 13th British Machine Vision Conference, Cardiff, September 2002: 22.1-22.10 ,

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited non-patent literature

• Stewart, C .: "Robust Parameter Estimation in Computer Vision", SIAM Review 1999; 41 (3): 513-537 [0153]
• Lucas, B.D., Kanade, T .: "Iterative Image Registration Technique with Application to Stereo Vision", International Joint Conferences on Artificial Intelligence 1981: 675-679 [0153]
• Brox, T., Bruhn, A. Papenberg, N., Weickert, J .: "High Accuracy Optical Flow Estimation based on a Theory for Warping," European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, Springer, 2004 ; 3024: 25-36 [0153]
• Hirschmüller H .: "Stereo Processing by Semiglobal Matching and Mutual Information", IEEE Transactions on Pattern Analysis and Machine Intelligence 2008; 30 (2): 328-341 [0153]
• Avidan, S., Shashua, A .: "Novel View Synthesis in Tensor Space," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1997: 1034-1040 [0153]
• Buchdahl, H.A .: "Optical Aberration Coefficients. III. The Computation of the Tertiary Coefficients, "Journal of the Optical Society of America 1958; 48: 747-756 [0153]
• Barreto, JP, Daniilidis, K .: "Fundamental Matrix for Cameras with Radial Distortion." In Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, IEEE Computer Society, Washington, DC, USA, 2005, pp. 625-632 [0153]
• Brito, J.H., Angst, R .; Köser, K; Pollefeys, M .: "Radial Distortion Self-Calibration", 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, 2013, p. 1368-1375 [0153]
• Förstner, W., Gülch, E .: "A Fast Operator for Detection and Precise Location of Distinct Points, Corners and Centers of Circular Features", in Proceedings of the ISPRS Conference on Fast Processing of Photogrammetric Data, Interlaken, 1987, pp. 281-305 [0153]
• Harris, C., Stephens, M .: "A Combined Corner and Edge Detector", Proceedings of the Fourth Alvey Vision Conference, 1988, p. 147-151 [0153]
• Lowe, D.G .: "Distinctive Image Features from Scale Invariant Keypoints" in the International Journal of Computer Vision; 2004, 60 (2): 91-110 [0153]
• Rublee, E., Rabaud, V., Konolige, K., Bradski, G .: "ORB: An Efficient Alternative to SIFT or SURF", in Proceedings of the 2011 International Conference on Computer Vision (ICCV'11), IEEE Society , Washington, DC, USA, 2011, pp. 2564-2571 [0153]
• Fischler, M.A., Bolles, R.C .: "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," in Journal of Communications of the ACM; 1981, 24 (6): 381-395 [0153]
• Green. A .: "Adaptive Least Squares Correlation: A Powerful Image Matching Technique," South African Journal of Photogrammetry, Remote Sensing, and Cartography, 1985; 14: 175-187 [0153]
• Bethmann, F., Luhmann, T .: Least-Squares Matching with Advanced Geometric Transformation Models, International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XXXVIII, Part 5 Commission V Symposium, Newcastle upon Tyne, UK , 2010 [0153]
• Torr, P.H. S .: "Geometric Motion Segmentation and Model Selection", Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 1998; 356 (1740): 1321-1340 [0153]
• Quan L .: "Invariants of 6 points from 3 Uncalibrated Images", in: Eklundh JO. (EDS) Computer Vision - ECCV '94. ECCV 1994. Lecture Notes in Computer Science, vol 801. Springer, Berlin, Heidelberg [0153]
• Torr, P.H.S., Zisserman, A .: "Robust Parameterization and Computation of the Trifocal Tensor", Image and Vision Computing 1997; 15: 591-605 [0153]
• Schaffalitzky, F., Zisserman, A. Hartley, RI, Torr, PHS: Computer Vision - ECCV 2000: 6th European Conference on Computer Vision Dublin, Ireland, June 26 - July 1, 2000, Proceedings, Part I Springer, Berlin, Heidelberg, 2000: 632-648 [0153]
• Carlsson, S., Weinshall, D .: "Dual Computation of Projective Shapes and Camera Positions from Multiple Images", International Journal of Computer Vision 2004; 27 (3): 227-241 [0153]
• Nister, D .: "An Efficient Solution to the Five-Point Relative Pose Problem," IEEE Transactions on Pattern Analysis and Machine Intelligence 2004; 26 (6): 756-777 [0153]
• Triggs, B .: The Geometry of Projective Reconstruction I: Matching Constraints and the Joint Image. Circulated in 1995. Accepted subject to revision to IJCV in 1995, but never completed. 1995 [0153]
• Hartley, R.I .: "Computation of the Quadrifocal Tensor", in: Burkhardt H., Neumann B. (eds) Computer Vision - ECCV'98. ECCV 1998. Lecture Notes in Computer Science, vol. 1406. Springer, Berlin, Heidelberg [0153]
• Connor, K., Reid, I .: "Novel View Specification and Synthesis", Proceeding of 13th British Machine Vision Conference, Cardiff, September 2002: 22.1-22.10 [0153]

Claims (9)

Device (1) for finding at least one change in a scene (100), the device (1) comprising a first determining device (10), a second determining device (20), an image processing device (30) and a comparison device (40), wherein the first determining device (10) is designed in such a way to determine data of a trifocal sensor from at least three images (P1, P2, P3) of the scene (100), wherein recording times (T1, T2, T3) are assigned to the three pictures (P1, P2, P3), the recording times (T1, T2) of two pictures (P1, P2) of the three pictures (P1, P2, P3) belonging to a first time interval (I1), wherein the two images (P1, P2) of the first time interval (I1) each have different perspectives relative to the scene (100), wherein the recording time (T3) of a third image (P3) of the three images (P1, P2, P3) belongs to a second time interval (I2), wherein the third image (P3) of the second time interval (I2) has a different perspective relative to the scene (100) than the two images (P1, P2) of the first time interval (I1), wherein a time interval (dt1) between the recording times (T1, T2) of the two images (P1, P2) of the first time interval (I1) is less than a time interval (dt2) between the first time interval (I1) and the second time interval (I2 ), wherein the second determination device (20) is designed in such a way to determine data of a pixel-wise association between the two images (P1, P2) for at least the two images (P1, P2) of the first time interval (I1) wherein the image processing device (30) is configured to provide at least two comparison images (C1, C2) starting from the at least three images (P1, P2, P3) and from the determined data of the trifocal sensor and the determined data of the pixel-by-pixel assignment, wherein the at least two comparison images (C1, C2) have substantially the same perspective on the scene (100) and are assigned to different time intervals (I1, I2), and wherein the comparison device (40) is configured to evaluate the two comparison images (C1, C2) as to whether there is at least one difference between the two comparison images (C1, C2) and to generate a comparison result based on the evaluation. Device (1) according to Claim 1 wherein the device (1) comprises a correction device (50), and wherein the correction device (50) is designed to recognize distortion errors on input images (P1 ', P2', P3 ') and non-distortion images (P1, P2, P3) and / or provide correction data. Device (1) according to Claim 1 or 2 in which the first determining device (10) is designed to use data relating to at least one recording condition of at least one image (P1, P2, P3) for the determination of the data of the trifocal sensor. Device (1) according to Claim 3 in which the at least one recording condition relates to a position of a recording device (200) used to record an image (P1, P2, P3) and / or to an orientation of a recording device (P1, P2, P3) used for recording an image (P1, P2, P3). 200) relative to the scene (100) and / or to intrinsic parameters of a capture device (200) used to capture an image (P1, P2, P3). Device (1) according to one of Claims 1 to 4 wherein the images (P1, P2) of the first time interval (I1) are before images and the image (P3) of the second time interval (I2) is a post-image. Device (1) according to one of Claims 1 to 4 wherein the images (P1, P2) of the first time interval (I1) are after images and the image (P3) of the second time interval (I2) is a before image. Device (1) according to one of Claims 1 to 6 , wherein the recording times (T1, T2) of the images (P1, P2) of the first time interval (I1) are almost or exactly the same. A method for finding at least one change in a scene (100), wherein at least three pictures (P1, P2, P3) are assigned, which are associated with recording times (T1, T2, T3), wherein the recording times (T1, T2) of two pictures (P1, P2) of the three images (P1, P2, P3) belong to a first time interval (I1) and the recording time (t2) of a third image (P3) of the three images (P1, P2, P3) at a second time interval ( I2), the two images (P1, P2) of the first time interval (I1) having different perspectives relative to the scene (100), the third image (P3) of the second time interval (I2) having a different perspective relative to the scene (I2). 100) as the two images (P1, P2) of the first time interval (I1), wherein a time interval (dt1) between the recording times (T1, T2) of the two images (P1, P2) of the first time interval (I1) is less than a time interval (dt2) between the first time interval (I1) and the second time interval (I2), starting from the at least three images (P1, P2, P3) Data of a Trifokaltensors or an equivalent description of the geometry of the three-figure case are determined, wherein at least for the two images (P1, P2) of the first time interval (I1) data of a pixel-wise association between the two images (P1, P2) are determined, starting from the at least three images (P1, P2, P3) as well as on the basis of the determined data of the Trifokaltensors or the equivalent description and the determined data of the pixel-wise assignment at least two comparison images (C1, C2), the substantially the same perspective on the scenery ( 100) and assigned to different time intervals (I1, I2), wherein the two comparison images (C1, C2) are evaluated as to whether there is at least one difference between the two comparison images (C1, C2), and based on the Evaluation of the two comparison images (C1, C2) a comparison result is generated. Computer program with a program code for carrying out the method according to Claim 8 ,

Images