For soldered connections on circuit boards, the legally mandatory switch to lead-free soldering has brought about higher defect rates. Quality control must therefore be improved. Systems for camera-supported automatic image analysis are used for this. However, the conventional procedures used up until now still have their limitations. When setting the control criteria, the user must tread a fine line between too high internal or too high external defect rates.

A solution now installed in Siemens Smart Infrastructure therefore now integrates a Deep Learning solution based on "artificial intelligence" (AI) that has enabled significant improvements.

"Our customer Siemens Smart Infrastructure manufactures smoke alarms for fire safety on automatic production lines, in many variants and in medium to large batches", states BSc FHO Lukas Vassalli, developer at the company Compar AG in Pfäffikon (Switzerland). The components used are placed on the circuit board with the help of placement machines and are then soldered from above. The EU-wide prohibition of leaded soldering alloys forces manufacturers to use lead-free soldering, which has poorer soldering properties, however. The consequence is increased reject and defect rates. Reliable automatic quality control systems are therefore all the more important. These are usually camera-supported image processing solutions that carry out In Order/Not in Order (IO/NIO) classifications with the help of suitable software packages based on image analyses. However, their selectivity has not always been satisfactory. Especially when used for critical safety functions, the control criteria must veer on the "safe" side, as fire detectors must guarantee the highest reliability. However, this causes increased reject rates with the corresponding cost disadvantages. In order to reduce these, Compar set itself the aim of using additional solutions with "artificial intelligence" in image analysis in the form of self-learning neural networks. This also involved integrating such devices into superordinate IT structures as part of "Industry 4.0" concepts.

Integration of AI

“The image processing specialist Cognex developed ready-to-use software packages in the form of plug-in modules for such tasks referred to as ViDi”, L. Vassalli adds. As a hardware condition, at least in the training phase, a high-performance image processor (Graphics Processing Unit, GPU) should be available on the computer used. A significant component of the software library is a neural network that is already partially pre-structured, so that the user can start induction quickly and easily. This is necessary before the first use and is put into practice by means of a certain number of images being entered into the network as “training material”. After that, it can analyse new images independently according to the required criteria. The pool of knowledge gathered during training is constantly extended and refined over the course of usage, which is also how the term “Deep Learning” came about. The application described here involves the assessment of soldered connections as well as identifying assembly errors.

“The overall system consists of a camera and lighting set up for the application, which records the circuit boards, as well as an Industry PC with a VISIONexpert® software”, says L. Vassalli. It is supplemented by the ViDi package, which functions as a “Black Box”. It analyses the transmitted images with the help of its neural network and delivers corresponding analyses in return.  This is carried out without delays within milliseconds on the production line. Before starting, the system was preconfigured by Compar with the help of images of provided sample components. During operation, the system can be trained with new products by the users themselves as required or else be trained further with already existing products. Owing to the high processing power, only a few minutes are necessary for such training phases. When training, you can either “feed” the system directly with photos or highlight fault areas through colour marking in advance in the supervisor mode. After a brief training, the customer is capable of carrying out such tasks themselves. This is an important precondition for the success of the project. In this case, around 50 images of fault-free parts sufficed as well as the same number of defected parts.

The ViDi processes

“The ViDi software consists of three modules (red, green and blue), of which in this case the modules “red” and “blue” are used”, L. Vassalli reveals. The “blue” module referred to as “locator” controls the circuit boards in terms of correct mounting. It identifies connection points and component positions, as well as imprints. Variances can be specified in advance. Then ViDi “red” takes over the classification into IO and NIO parts.

During training, various settings can be selected, e.g. instead of the two categories IO/NIO, exclusively specifying IO parts. In this case the AI will automatically classify as NIO everything that cannot be clearly identified as IO.

“An important property of ViDi analysis is the numerical evaluation of the classification of the respective result”, L. Vassalli adds. Although the system in principle classifies analysed images according to the criteria “IO” and “NIO”, it always states a percental value. This indicates to what percentage the software image is reliable, according to its evaluation. It ranges from 0 (100% IO) to 1 (= 0% IO or 100% NIO). The frequency distribution of these classifications is shown statistically in the form of diagrams with e.g. a green colour for IO and red colour for NIO results.

They take the form of two bar charts in green and red, which may partially overlap. A simpler representation results from the merging of the cumulated and normalised ranges of variation. Depending on the task and the analysis criteria, these can either partially overlap or form two clearly separate groups. If the training has run optimally, there is no overlapping between the cumulated frequency ranges. This indicates the good separation precision of the procedure. If this is not the case, you end up in the decision range between “false positives” and “false negatives” classifications. In such cases, the optimal setting of the so-called threshold value plays an important role. If this is set more on the safe side, you minimise for example the default risk of safety-relevant components for the customer. With the opposite strategy, you can reduce the internal reject occurrences.

Linking with VISIONexpert®

“What is particularly of interest for customers is the linking of the described ViDi possibilities with the image processing software VISIONexpert® that we have developed”, L. Vassalli states. As a main component, the Compar software initially takes over the external hardware handling, i.e. the linking of the numerous possible camera models and other peripherals. Another task is the image data management, as well as the transmission of image data to be analysed to ViDi. The results that are fed back are used internally, visualised and finally incorporated into the decision-making. Despite all the automatic features, there is always a human power of decision by setting the control criteria and decision specifications such as the threshold level.

For the analysis and evaluation of a sample, the results of the ViDi inspection are referred to along with the VISIONexpert® capabilities. Contrary to the ViDi plug-in, this software can measure for example down into the µm range with high precision and make decisions based on the results. Finally, VISIONexpert®  also handles the communication with the superordinate IT of the company. For Compar, which has been developing solutions for visual quality control for decades, the incorporation of the new AI-based tool is an important step in the further improvement of the product range, according to L. Vassalli.