DE102019117680A1 - AUTOMATED SEGMENTATION PROCESS FOR QUALITY INSPECTION - Google Patents

Abstract

A method for measuring defects in components in a quality inspection process is presented. The method includes receiving a digital image of a component to be examined, receiving a first reference image of the component to be examined, and determining a second reference image from a combination of the received digital image and the first reference image. In addition, the method has an activation of a trained machine learning-based classifier system, which was trained with training data to create a model, the model serving as the basis for semantic segmentation of voxels of the received image into defect classes, and classification by the activated classifier system of Voxels of the received digital image, both voxels of the received digital image and voxels of the second reference image serving as input data for the classifier system.

Description

Field of invention

The invention relates to an automated segmentation of defects in image files in test processes in which similar parts are tested (inline and at-line) and in particular to a computer-implemented method for measuring defects in components in a quality test process, a corresponding system and a corresponding computer program product.

Technical background

In order to keep the quality of manufactured components at a consistently high level, permanent quality controls in the ongoing production process are essential. In the meantime, optical systems are often used for this purpose, which can identify defects on or in the manufactured components using image recognition processes. Microscope systems are also increasingly being used for quality assurance measures of this kind, in which small and very small defects in components can be identified by means of magnification. This is often done using image segmentation.

A fully automatic segmentation of defects is particularly necessary in inline processes and at-line in order to determine the dimensions of a defect such as its volume, its circumference, its diameter or a local frequency fully automatically in a subsequent step without user interaction. This is the only way to carry out an inexpensive and complete inspection of components for quality assurance.

The automatic segmentation is accomplished in different ways. In test processes that often have to react to process changes and have to deal with a large variance in defect characteristics, a model based on a training set for a machine learning model is generally preferred, which consists of components of the same type as the test object. One approach consists in forming a model in which predominantly error-free parts are selected as the training set and the mathematical mean and / or the standard deviation of image brightness values are determined from these pixel-by-pixel or voxel-wise. However, this assumes that the training set consists of images / volume data of the same component. The main disadvantage of this method is that, in addition to defects, scan artifacts and slight displacements of elements of the test object (e.g. walls), which are caused by manufacturing tolerances, are segmented and thus lead to incorrect results. Contrast effects also make this method less robust.

A second approach to creating a model is to use both faulty sample parts and faultless sample parts in the training set for a machine learning model. Defects in the defective pattern parts are provided with a pixel / voxel-wise annotation / marking of the defective points. In principle, it does not have to be assumed that the training set consists of images / volume data of the same component, but it is merely assumed that locally similar defects with similar characteristics are present. Based on the training set, a static classification method is trained in which only the image brightness information of the test object is used to segment the defects. A major disadvantage of this method is that a sufficiently large (annotated) training set must be available in order to achieve a robust segmentation of the defects. This usually requires very long training times. Furthermore, this method tends to segment structures that are similar to defects in the local context, but are target structures, as defects.

Based on the disadvantages of the known methods, an underlying task for the concept presented here is to overcome the mentioned disadvantages of the known methods, and in particular also to present a method that is little dependent on the contrast of the defect in the image environment, nor that large amounts of training data are required, so that training times for creating a machine learning model can also be reduced.

Overview of the invention

This object is achieved by the method proposed here, the corresponding system and the associated computer program product in accordance with the independent claims. Further refinements are described by the respective dependent claims.

According to a first aspect of the present invention, a computer-implemented method for measuring defects in components in a quality inspection method is presented. The method can include receiving a digital image of a component to be examined, receiving a first reference image of the component to be examined, and determining a second reference image from a combination of the received digital image and the first reference image.

Furthermore, the method can have an activation of a trained machine learning-based classifier system, which was trained with training data to form a model, the model serving as the basis for semantic segmentation of voxels of the received image into defect classes, and classification by the activated classifier system of voxels of the received digital image. Both voxels of the received digital image and voxels of the second reference image can serve as input data for the classifier system.

According to a second aspect of the present invention, a system for measuring defects in components in a quality inspection method is presented. The system may have a first receiving unit that is adapted to receive a digital image of a component to be examined, a second receiving unit that is adapted to receive a first reference image of the component to be examined and a determination module that is adapted to determine a second reference image from a Combination of the received digital image and the first reference image.

In addition, the system can have a trained machine learning-based classifier system, which was trained with training data to create a model, the model serving as the basis for semantic segmentation of voxels of the received image into defect classes. In this case, the classifier system can be adapted for classification by the activated classifier system of voxels of the received digital image. Both voxels of the received digital image and voxels of the second reference image can serve as input data for the classifier system.

Additionally, embodiments may relate to a computer program product accessible from a computer-usable or computer-readable medium that includes program code for use by, by, or in connection with a computer or other instruction processing system, in the context of these Description, a computer-usable or computer-readable medium can be any device that is suitable for storing, communicating, forwarding or transporting the program code.

The computer-implemented method for measuring defects in components in a quality inspection process has several advantages and technical effects that can also apply to the associated system:

The proposed method or the corresponding system for measuring - and thus also for detecting - defects in components in a quality inspection method - in particular an inline or at-line quality inspection method - overcomes the disadvantages of the previously known methods based on discrete segmentation via a Difference signal and a segmentation of a recorded image of a test object by a classification system. In addition to the recorded digital image of the test item, the proposed concept uses further information for precise and robust segmentation of defects. These are created from the recorded digital image of the test item and a reference image. The data of the reference image can be generated in various ways (e.g. averaging, derivation from an abstract construction model, etc.). The additionally used information therefore consists, for example, of a comparatively easily derivable difference signal of the test object to a reference quantity. The classifier system therefore not only uses the image / volume brightness information that comes from the test item, but also an image / volume, for example, of the same size - or information about it - that describes the local deviations from the reference set. Such a differential signal is always available when similar test items of the same type are tested in large numbers (e.g.> 20). For this reason, the concept proposed here is excellently suited for segmenting defects in workpieces or components in inline and at-line inspection processes. This fully automatic segmentation of defects is particularly necessary in inline and at-line processes in order to be able to determine the dimensions of a defect such as volume, circumference, diameter or local clustering in a subsequent step fully automatically without user interaction. In this respect, the method proposed here makes a significant contribution to improving inline and at-line testing processes and thus a positive cost effect in production and quality assurance processes.

In principle, a large number of different classifier systems can be used. Examples are given elsewhere in this document. Particularly when using a Deep Neural Network (DNN) as a classifier system, the information channels of the input data (first reference image and second reference image) can be linked to one another and fed into the input layer of the DNN, so that the classifier system determines a weighting of the information based on the training set can.

As further advantages of the concept proposed here that should be particularly emphasized, a better generalization, a name higher robustness and a reduction in annotation effort. The proposed method achieves better segmentation accuracy with regard to defects that are only weakly represented in the training set. In addition, there are fewer false-positive segmentations, for example on imaging artifacts, so that the accuracy of the method is improved. In addition, defect-like target structures are segmented correctly with a higher probability.

In addition, the information in the channel, which represents the difference to the reference set, is relatively meaningful, so that the training for the classifier system is less complex. This also makes it possible to achieve comparatively precise and more robust results with less annotated data than without the additional channel. As a result, training times can be significantly reduced, which means that the proposed method can also be used more quickly for new components or test items, which at the same time helps to reduce required costs and set-up times.

In addition to the main application for inline and at-line quality inspection, in which many sample data are available for a reference quantity, it is also possible to provide the procedure for applications in the laboratory or in the measuring room, in which the reference quantity is used in all Usually from a few examples, in extreme cases from a single good example. Another possibility is to calculate the reference model via a mapping simulation, for example with the aid of a CAD model.

In the following, further embodiments of the inventive concept for the method, which can apply equally and correspondingly to the corresponding system, are presented:

According to an advantageous embodiment of the method, the second reference image can be determined by forming the difference between the brightness values of the received digital image and the first reference image by voxel. If necessary, weighting factors can also be used for the images or for partial areas of the images. The formation of the difference requires comparatively little computing resources and can be carried out quickly even with large quantities of voxels.

According to a possible embodiment of the method, the first reference image and the second reference image can be identical. This means that it would be possible to completely dispense with the formation of a difference, which in turn would accelerate the process. The additional interpretation service could then be taken over by the classifier system.

According to a further advantageous embodiment of the method, the received reference image can contain voxel-wise distribution parameter values - in particular based on the training data set - of brightness values and the second reference image can be normalized with regard to the distribution parameter values. In this way, elegantly comparable test results can be derived.

According to a further advantageous embodiment of the method, the spatial - in particular the number of surrounding pixels in a predetermined direction - or the temporal context - in particular a sliding window of the last N volumes - of the voxel can be included in the classification for the classification of a voxel. In this way, drift effects in particular can be eliminated.

According to a further developed embodiment of the method, the first reference image can be adapted over time, for example by means of a moving average. In this way, too, unavoidable drift effects can be proactively addressed.

Embodiments of the method can be implemented with regard to the classifier system, inter alia, by means of support vector machine, random forest, boosting, gradient boosting, Gaussian process, nearest neighbors, logistic regression, linear regression and neural network. Further, alternative classifier systems are possible.

According to a particularly advantageous embodiment of the method, the neural network can be a deep neural network (DNN). Such neural networks are particularly suitable for processing digital images. Fast and efficient processing of digital image information is possible, particularly in its embodiment as a convolutional neural network (CNN).

According to a further developed embodiment of the method, an input layer of the deep neural network can be expanded with respect to the set of input channels - ie the artificial neurons at the input of the DNN - which are required for processing the digital image, so that voxel data of the first and / or or the second reference image can be fed to additional input channels of the classifier system. The pixel information of the input data (recorded digital image, reference image (s)) can therefore be cascaded.

According to an optional embodiment of the method, image resolutions of the received digital image and of the first Reference image and / or the second reference image be different. This means that the method is also independent of recording systems for the received digital image and the reference image. Both could have been recorded with different recording systems. In addition, preprocessing of the reference image is possible in practically every respect.

According to a further embodiment of the method, the classifier system can have a first classifier system and a second classifier system. The first classifier system can work with a first number of parameter values - in particular, for example, weight parameters of the DNN or a lower resolution; and the second classifier system can work with a second number of parameter values - in particular, for example, a higher resolution - wherein the second number of parameter values is higher than the first number. This flexibility allows different requirements to be taken into account during different phases of the quality inspection process. In particular, a smaller number of parameter values is accompanied by faster processing.

According to a further embodiment shown above, the second classifier system can only be activated, for example, if the first classifier system has classified a coherent group - the size of which is subject to a threshold comparison so that only a minimum amount is taken into account - of voxels as belonging to a defect class. In simple terms, the second, more complex classifier system is only active when a signal is generated by the first, simpler and faster classifier system, which indicates that there is a suspicion of quality problems, whereupon a more detailed investigation is carried out using the second, more precise classifier system, in particular using a higher resolution to decide whether there is actually a quality problem with regard to the component.

According to possible embodiments of the method, the received digital image and elements of the training data set for the classifier system can be derived from an image recording method. The image recording method can be selected from the group consisting of an electron microscope, a fluorescence microscope, a light microscope, an optical coherence tomograph, an interferometer, a spectrometer, a surgical microscope and a computer tomograph. In principle, all common quality inspection approaches can be expanded by the method proposed here.

According to a further embodiment of the method, the first reference image is formed by averaging a plurality of sample components - in particular "good", ie defect-free or defect-reduced samples - of the component or is derived from a CAD model of the component, or is obtained by simulating the component Image acquisition process generated from one or more object models - for example from a CAD system - eg with voxel metadata “0”, “1” or “gray” - or derived from one or more object models based on rules.

It is also possible, in the method, to obtain the distribution parameter values by determining the standard deviation from mean brightness values above a mean value of a plurality of “good” sample components of the component. So there are a number of ways to get a reference image, some of which are only listed here as examples.

Overview of the figures

It should be noted that exemplary embodiments of the invention can be described with reference to different implementation categories. In particular, some exemplary embodiments are described in relation to a method, while other exemplary embodiments can be described in the context of corresponding devices. Regardless of this, it is possible for a person skilled in the art to recognize and combine possible combinations of the features of the method and possible combinations of features with the corresponding system from the description above and below - unless otherwise indicated - even if they belong to different claim categories.

Aspects already described above and additional aspects of the present invention emerge, inter alia, from the exemplary embodiments described and from the additional further specific configurations described by reference to the figures.

• 1 stellt ein Blockdiagramm eines Ausführungsbeispiels des erfinderischen computer-implementierten Verfahrens zur Vermessung von Defekten von Bauteilen in einem Qualitätsprüfungsverfahren dar.
• 2 stellt eine Prinzipskizze eines Vorgehens dar, das den hier vorgestellten Konzepten zugrunde liegt.
• 3 stellt ein Blockdiagramm eines Ausführungsbeispiels des Systems zur Vermessung von Defekten von Bauteilen in einem Qualitätsprüfungsverfahren dar.
• 4 stellt ein Blockdiagramm eines Computersystems dar, welches zusätzlich das System gemäß 3 ganz oder teilweise aufweisen kann.

Preferred embodiments of the present invention are described by way of example and with reference to the following figures:

• 1 shows a block diagram of an embodiment of the inventive computer-implemented method for measuring defects of components in a quality inspection method.
• 2 represents a basic sketch of a procedure on which the concepts presented here are based.
• 3 shows a block diagram of an embodiment of the system for measuring defects of components in a quality inspection process.
• 4th FIG. 11 is a block diagram of a computer system that additionally incorporates the system according to 3 may have wholly or partially.

Detailed description of the figures

In the context of this description, conventions, terms and / or expressions should be understood as follows:

The term “workpiece” describes a component or a test item which is going through a quality inspection process.

The term “defect” describes an anomaly on a surface or in the volume of the workpiece. Defects can be caused by pores, depressions, inhomogeneity, inclusions, etc. As a rule, they can be detected optically - on the surface or in the volume of the component.

The term "digital image" describes an image or the result of generating a quantity of data in the form of pixel or voxel data of a real existing object, here a component, for example in an in-line or at-line inspection process. digital image ”can be a signal of different dimensions (1-D, 2-D, 3-D, ..., nD) as long as sections (spatial points on the object) from different“ digital images ”can be clearly assigned to one another. In principle, the method would also work on audio spectra, as long as a 1: 1 assignment of spatial points to the signal can be established.

The term “first reference image” - in particular first digital reference image - describes a data record that describes an image that is used as a reference for the recorded digital image. The first reference image essentially corresponds to an ideal of an error-free test object.

The term “second reference image” - in particular a second digital reference image - describes a data record that is derived from a combination of the recorded digital image and the first reference image. Different mathematical methods for linking corresponding pixel or voxel data can be used for the combination. Examples are deviation from the mean value of the reference image, standard deviation, etc.

The term “classifying” describes the process of assigning a recorded image or parts thereof - here in particular individual pixels of the recorded image - to pixel classes. One pixel class can describe, for example, that the corresponding pixel belongs to an image segment which is free of defects, while another pixel class can describe that there is a defect on / in the test object. A probability value for this statement can also be specified. In the context of this text, the instrument for performing the classification is the classifier system.

The term "classifier system" - also referred to as a classifier or classification system in the context of machine learning - describes a machine learning-based system that is enabled through training with training data to input data - here in particular image data from recorded digital Images - assign features of the images to a certain class (e.g. defective / not defective).

It should also be noted here that a classifier typically classifies into a predetermined number of classes. This is usually done by determining a classification value of the input data for each class and a WTA (winner takes it all) filter selecting the class with the highest classification value as the classified class. In the case of classifiers, the deviation from a 100% classification value is often used as a quality parameter of the classification or as a probability for the correctness of the classification.

Examples of classifier systems that can be used for the subjects of the inventive concepts presented here are systems based on the following principles: Support Vector Machine, Random Forest, Boosting, Gradient Boosting, Gaussian Process, Nearest Neigbor, Logistic Regression, Linear Regression and Neural Network. Further algorithms are of course also possible.

The term “machine learning” (or machine learning) is a basic term or a basic function of artificial intelligence, whereby, for example, statistical methods are used to give computer systems the ability to “learn”. For example, certain behavior patterns are optimized within a specific task area. The methods used enable trained machine learning systems to analyze data without the need for explicit procedural programming. Typically, for example, an NN (Neural Network) or CNN (Convolutional Neural Network) is an example of a machine learning system to form a network of nodes that act as artificial neurons and artificial connections between the artificial ones Neurons - so-called links - whereby parameters - for example weight parameters for the connection - can be assigned to the artificial connections. During the training of the neural network, the weight parameter values of the connections are automatically adapted on the basis of input signals in order to generate a desired result. In supervised learning, the images - generally (input) data - supplied as input values (training data) are supplemented by desired output data (annotations) in order to generate a desired output value (desired class). In general terms, a mapping of input data to output data is learned.

The term “training the classification system” here means that, for example, a machine learning system (machine learning system) is adjusted by a plurality of sets of example data - ie reference data - in a neural network, for example, by partially repeated evaluation of the example data, so that after to assign unknown images or individual pixels or voxels to one or more classes with which the learning system has been trained. The sample data is typically annotated - i.e. provided with metadata - in order to generate desired results based on the input images; e.g. defect class, non-defect class.

In the context of this text, the term “semantic segmentation” describes a segmentation of parts of the data that describe a digital image into areas of the same classes of a digital image by the classifier system.

The term "convolutional neural network" (CNN) - as an example of a classifier / a classifier system - describes a class of artificial neural networks based on feed-forward techniques. They are often used for image analysis with images or their pixels as input data. The main component of convolutional neural networks are convolution layers (hence the name), which enable efficient evaluation through parameter sharing. Typically, each pixel of the recorded image is assigned as an input value to an artificial neuron in the neural network.

It should also be mentioned that deep neural networks consist of several layers of different functions - for example an input layer, an output layer and one or more layers in between, for example for convolution operations, the application of non-linear functions, a dimension reduction, normalization functions, etc. The functions can be "executed in software", or special hardware modules can take over the calculation of the respective function values. In addition, combinations of hardware and software elements are known.

The term “input channel” - in particular the input channel of the classifier system and, more precisely, a DNN or CNN - describes an input of the classifier system, for example in the form of an ANN (artificial neural network) in which an artificial neuron represents an input node. Typically, each artificial neuron in the input layer of the ANN is designed as an input node. In the concept presented here, not only pixel or voxel information of the recorded digital image can be fed to this input layer, but also pixel or voxel information of the second reference image, which is typically present as matrix data.

A detailed description of the figures is given below. It goes without saying that all details and instructions are shown schematically in the figures. First, a block diagram of an exemplary embodiment of the computer-implemented method according to the invention for measuring defects in components in a quality inspection method is shown. Further exemplary embodiments or exemplary embodiments for the corresponding system are described below:

1 FIG. 10 shows a block diagram of an embodiment of the inventive computer-implemented method 100 for the measurement - in particular also for the detection - of defects of components in a quality inspection process. The process 100 indicates a receiving 102 a digital image of a component to be examined. The component to be examined is typically a test object, ie a workpiece that is to be examined for quality features.

The procedure 100 also has a receiving 104 a first reference image of the component to be examined. The reference image is, for example, a calculated mean value of “good”, that is, fault-free components or those with acceptable faults. Alternatively, it can also be an idealized digital model of the component that is derived, for example, from a CAD model (Computer-Aided Design). It should also be noted that digital information for the pixels / voxels can consist of data relating to a number of channels both via the received digital image of the component to be examined and the reference image.

It also instructs the procedure 100 a determination 106 a second reference image from a combination of the received digital image and the first reference image as well as activating a trained machine learning-based classifier system that was trained with training data to create a model. The model serves as the basis for semantic segmentation of voxels of the received image in defect classes, whereby other neighboring voxels of the same class are combined into coherent image segments.

Finally instructs the procedure 100 a classification 108 by the activated classifier system of voxels of the received digital image, both voxels of the received digital image and voxels of the second reference image serving as input data for the classifier system.

It should not go unmentioned that the information or data of the digital images (regardless of whether they are received or determined) are typically represented in matrices. Data in matrix form can be elegantly processed using mathematical methods.

2 represents a schematic diagram 200 a procedure on which the concepts presented here are based. The image brightness values 202 from the measurement of the test piece, ie, a digital recording of the workpiece to be checked with regard to its manufacturing quality, are used twice. On the one hand there is a difference signal matrix 204 formed (corresponding to the second reference image), the operation input data a matrix of voxel information 206 of a reference workpiece and on the other hand the voxel data 202 of the image brightness values from the measurement of the test object.

A classifier 208 then gets both the difference signal matrix 204 as well as image brightness values 202 from the measurement of the test item presented as input data at the input layer. The classifier 206 then performs the segmentation of the input image (ie the image brightness values 202 from the measurement of the device under test), which results in segmentation results 210 arise. Optionally, the data that describe the reference image can also be used directly as input data for the classifier system (see dashed connection in 2 ).

In other words, the concept proposed here combines a direct segmentation of pixels / voxels via a difference signal and segmentation by means of a simple classification and makes use of the fact that the training data volume in inline and at-line inspection processes consists of images or volume data of the same component or workpiece.

3 Figure 10 is a block diagram of one embodiment of the system 300 for the measurement - in particular also for the detection - of defects of components in a quality inspection process. The system 300 has a first receiving unit 302 which is adapted to receive a digital image of a component to be examined, a second receiving unit 304 which is adapted to receive a first reference image of the component to be examined, and a determination module 306 that is adapted to determine a second reference image from a combination of the received digital image and the first reference image. In addition, the system 300 a trained machine learning based classifier system 308 on, which was trained with training data to create a model, the model serving as the basis for semantic segmentation of voxels of the received image into defect classes. Here is the activated classifier system 308 adapted to classify voxels of the received digital image. As input data for the classifier system 308 Both voxels of the received digital image and voxels of the second reference image are used.

4th shows a block diagram of a computer system which can have at least parts of the system for measuring defects in components in a quality inspection process. Embodiments of the concept proposed here can in principle be used with practically any type of computer, regardless of the platform used therein for storing and / or executing program code. 4th exemplifies a computer system 400 represents which is suitable for executing program code according to the method presented here. A computer system already present in a microscope system can also - if necessary with appropriate extensions - serve as a computer system for the implementation of the concept presented here.

The computer system 400 has a plurality of general purpose functions. The computer system can be a tablet computer, a laptop / notebook computer, another portable or mobile electronic device, a microprocessor system, a microprocessor-based system, a smartphone or a computer system with specially configured special functions. The computer system 400 can be set up to execute instructions that can be executed by the computer system - such as program modules, for example - which can be executed in order to implement functions of the concepts proposed here. The Program modules have routines, programs, objects, components, logic, data structures, etc. in order to implement certain tasks or certain abstract data types.

The components of the computer system can include: one or more processors or processing units 402 , a storage system 404 and a bus system 406 , which includes various system components including the storage system 504 with the processor 402 connects. Typically the computer system 400 a plurality of by the computer system 400 accessible volatile or non-volatile storage media. In the storage system 404 the data and / or instructions (commands) of the storage media can be stored in volatile form - such as in a RAM (random access memory) 408 - be saved to by the processor 402 to be executed. These data and instructions implement single or multiple functions or steps of the concept presented here. Other components of the storage system 404 can use a permanent memory (ROM) 410 and long-term storage 412 in which the program modules and data (reference symbols 516 ) as well as workflows can be saved.

The computer system has a number of dedicated devices (keyboard 418 , Mouse / pointing device (not shown), screen 420 , etc.). These dedicated devices can also be combined in a touch-sensitive display. A separately provided I / O controller 414 ensures a smooth exchange of data with external devices. A network adapter is used for communication via a local or global network (LAN, WAN, for example via the Internet) 522 to disposal. Other components of the computer system may be on the network adapter 400 via the bus system 506 can be accessed. It goes without saying that - although not shown - other devices can also be connected to the computer system 400 can be connected.

In addition, at least parts of the system 300 for the measurement of defects in components (cf. 3 ) to the bus system 306 be connected.

The description of the various exemplary embodiments of the present invention has been presented for better understanding, but does not serve to restrict the inventive idea directly to these exemplary embodiments. The person skilled in the art will find further modifications and variations himself. The terminology used here was chosen so as to best describe the basic principles of the exemplary embodiments and to make them easily accessible to the person skilled in the art.

The principle presented here can be embodied both as a system, as a method, combinations thereof and / or as a computer program product. The computer program product can have one (or more) computer-readable storage medium (s) which have computer-readable program instructions in order to cause a processor or a control system to execute various aspects of the present invention.

The media used are electronic, magnetic, optical, electromagnetic, infrared media or semiconductor systems as transmission media; For example, SSDs (solid state device / drive as solid-state storage), RAM (Random Access Memory) and / or ROM (Read-Only Memory), EEPROM (Electrically Eraseable ROM) or any combination thereof. Propagating electromagnetic waves, electromagnetic waves in waveguides or other transmission media (e.g. light pulses in optical cables) or electrical signals that are transmitted in wires can also be used as transmission media.

The computer-readable storage medium may be an embodied device that holds or stores instructions for use by an instruction execution device. The computer-readable program instructions that are described here can also be downloaded to a corresponding computer system, for example as a (smartphone) app from a service provider via a cable-based connection or a cellular network.

The computer-readable program instructions for carrying out operations of the invention described here can be machine-dependent or machine-independent instructions, microcode, firmware, status-defining data or any source code or object code, for example in C ++, Java or similar or in conventional procedural programming languages such as the programming language "C" or similar programming languages. The computer-readable program instructions can be completely executed by a computer system. In some exemplary embodiments, it can also be electronic circuits, such as, for example, programmable logic circuits, field-programmable gate arrays (FPGA) or programmable logic arrays (PLA), which execute the computer-readable program instructions by using status information of the computer-readable program instructions, to configure or individualize the electronic circuits according to aspects of the present invention.

In addition, the invention presented here is illustrated with reference to flow charts and / or block diagrams of methods, devices (systems) and computer program products according to exemplary embodiments of the invention. It should be noted that practically every block of the flowcharts and / or block diagrams can be designed as computer-readable program instructions.

The computer-readable program instructions can be made available to a general-purpose computer, a special purpose computer or some other programmable data processing system in order to manufacture a machine, so that the instructions which are executed by the processor or the computer or other programmable data processing device means to implement the functions or operations outlined in the flowchart and / or block diagrams. These computer-readable program instructions can accordingly also be stored on a computer-readable storage medium.

In this sense, each block in the illustrated flow diagram or the block diagrams can represent a module, a segment or portions of instructions which represent a plurality of executable instructions for implementing the specific logic function. In some exemplary embodiments, the functions that are shown in the individual blocks can be carried out in a different order - possibly also in parallel.

The illustrated structures, materials, processes, and equivalents of all means and / or steps with associated functions in the claims below are intended to apply any structure, material, or process as expressed by the claims.

List of reference symbols

100

Method for measuring defects in components

102

Process step to 100

104

Process step to 100

106

Process step to 100

108

Process step to 100

200

Procedure of the underlying concept presented here

202

Image data of the test item

204

Difference signal

206

Reference data

208

Classifier / classification system

210

Segmentation result

300

System for measuring defects in components

302

1. Receiver unit

304

2. Receiver unit

306

Determination module

308

Classifier / classification system

400

Computer system

402

processor

404

Storage system

406

Bus system

408

R.A.M.

410

ROME

412

Long-term storage

414

I / O controller

416

Program modules, potential data

418

keyboard

420

screen

422

Network adapter

Claims (16)

A computer-implemented method for measuring defects in components in a quality inspection method, the method having - Receiving a digital image of a component to be examined, - Receiving a first reference image of the component to be examined, - Determination of a second reference image from a combination of the received digital image and the first reference image, - Activation of a trained machine learning-based classifier system, which was trained with training data to create a model, the model serving as the basis for semantic segmentation of voxels of the received image into defect classes, and Classification by the activated classifier system of voxels of the received digital image, both voxels of the received digital image and voxels of the second reference image serving as input data for the classifier system. The procedure according to Claim 1 , in which the second reference image is formed by a voxel-wise subtraction of brightness values of the received digital image and first reference image is determined. The procedure according to Claim 1 , in which the first reference image and the second reference image are identical. The method according to one of the preceding claims, wherein the received reference image contains voxel-wise distribution parameter values of brightness values and the second reference image is normalized with regard to the distribution parameter values. Method according to one of the preceding claims, wherein, for the classification of a voxel, the spatial or temporal context of the voxel is included in the classification. Method according to one of the preceding claims, in which the first reference image is adapted over time. The method according to any one of the preceding claims, wherein the classifier system is selected from the group consisting of Support Vector Machine, Random Forest, Boosting, Gradient Boosting, Gaussian Process, Nearest Neigbor, Logistic Regression, Linear Regression and Neural Network. The procedure according to Claim 7 , where the neural network is a deep neural network. The procedure according to Claim 8 , wherein an input layer of the deep neural network is expanded compared to the amount of input channels that are required for processing the digital image, so that voxel data of the first and / or the second reference image are routed to additional input channels of the classifier system. The method according to any one of the preceding claims, wherein image resolutions of the received digital image and of the first reference image and / or second reference image are different. The method according to any one of the preceding claims, wherein the classifier system comprises a first classifier system and a second classifier system, wherein the first classifier system operates with a first number of parameter values, wherein the second classifier system operates with a second number of parameter values, the second number being greater than the first number. The procedure according to Claim 11 , the second classifier system being activated only if the first classifier system has classified a coherent group of voxels as belonging to a defect class. The method according to one of the preceding claims, in which the received digital image and elements of the training data set for the classifier system is derived from an image recording method, the image recording method being selected from the group consisting of an electron microscope, a fluorescence microscope, a light microscope, an optical coherence tomograph , an interferometer, a spectrometer, a surgical microscope and a computed tomograph. The method according to one of the preceding claims, in which the first reference image is formed by averaging a plurality of sample components of the component, or is derived from a CAD model of the component, or is generated from one or more object models by simulating the image recording process one or more object models is derived based on rules. A system for measuring defects in components in a quality inspection process, comprising the system - a first receiving unit which is adapted to receive a digital image of a component to be examined, - a second receiving unit which is adapted to receive a first reference image of the component to be examined, - a determination module which is adapted to determine a second reference image from a combination of the received digital image and the first reference image, a trained machine-learning-based classifier system which was trained with training data to form a model, the model serving as the basis for semantic segmentation of voxels of the received image into defect classes, and wherein the classifier system is adapted to classify voxels of the received digital image by the activated classifier system, both voxels of the received digital image and voxels of the second reference image serving as input data for the classifier system. A computer program product for measuring defects in components in a quality inspection process, the computer program product having a computer-readable storage medium which has program instructions stored thereon, the program instructions being provided by one or more computers or Control units are executable and causes the one or more computers or control units to implement the method according to one of the Claims 1 to 14th execute.

Images