How do Convolutional Neural Networks Work?
Trend

How do Convolutional Neural Networks Work?

Breakthroughs in deep learning in recent years have come from the development of Convolutional Neural Networks (CNNs or ConvNets). It is the main force in the development of the deep neural network field, and it can even be more accurate than humans in image recognition.
Published: Oct 06, 2022
How do Convolutional Neural Networks Work?

What is a Convolutional Neural Network?

Convolutional Neural Network is a feed-forward neural network whose artificial neurons can respond to surrounding units within a partial coverage area and has excellent performance for large-scale image processing. A convolutional neural network consists of one or more convolutional layers and a top fully connected layer, as well as associated weights and pooling layers. This structure enables convolutional neural networks to exploit the two-dimensional structure of the input data. Compared to other deep learning architectures, convolutional neural networks can give better results in image and speech recognition. This model can also be trained using the backpropagation algorithm. Compared to other deep, feed-forward neural networks, convolutional neural networks have fewer parameters to consider, making them an attractive deep learning architecture.

The Convolutional Neural Network is powerful in image recognition, and many image recognition models are also extended based on the CNN architecture. It is also worth mentioning that the CNN model is a deep learning model established by referring to the visual organization of the human brain. Learning CNN will help me learn other deep learning models.

Feature:

CNN compares parts of the image, which are called features. By comparing rough features at similar locations, CNNs are better at distinguishing whether images are the same or not, rather than comparing whole images. Each feature in an image is like a smaller image, that is, a smaller two-dimensional matrix and these features capture common elements in the image.

Convolution:

Whenever a CNN resolves a new image, without knowing where the above features are, the CNN compares anywhere in the image. To calculate how many matching features are in the whole image, we create a filtering mechanism here. The mathematical principle behind this mechanism is called convolution, which is where the name CNN comes from.

The basic principle of convolution is to calculate the degree of conformity between the feature and the image part, if the value of each pixel of the two is multiplied, and then the sum is divided by the number of pixels. If every pixel of the two images matches, sum these products and divide by the number of pixels to get 1. Conversely, if the two pixels are completely different, you will get -1. By repeating the above process and summarizing various possible features in the image, convolution can be completed. Based on the values and positions of each convolution, make a new 2D matrix. This is the original image filtered by the feature, which can tell us where to find the feature in the original image. The part with a value closer to 1 is more consistent with the feature, the closer the value is to -1, the greater the difference; as for the part with a value close to 0, there is almost no similarity at all. The next step is to apply the same method to different features, and convolutions in various parts of the image. Finally, we will get a set of filtered original images, each of which corresponds to a feature. Simply think of the entire convolution operation as a single processing step. In the operation of CNNs, this step is called a convolutional layer, which means that there are more layers to follow.

The operation principle of CNN is computationally intensive. While we can explain how a CNN works on just one piece of paper, the number of additions, multiplications, and divisions can increase quickly along the way. With so many factors affecting the number of computations, the problems that CNN's deal with can become complex with little effort, and it is no wonder that some chipmakers are designing and building specialized chips for the computational demands of CNNs.

Pooling:

Pooling is a method of compressing images and retaining important information. Its working principle can be understood with only a second degree in mathematics. Pooling will select different windows on the image, and select a maximum value within this window range. In practice, a square with a side length of two or three is an ideal setting with a two-pixel stride.

After the original image is pooled, the number of pixels it contains will be reduced to a quarter of the original, but because the pooled image contains the maximum value of each range in the original image, it still retains each range and each range. The degree of conformity of the characteristics. The pooled information is more focused on whether there are matching features in the image, rather than where these features exist in the image. Can help CNN to determine whether a feature is included in the image without having to be distracted by the location of the feature.

The function of the pooling layer is to pool one or some pictures into smaller pictures. We end up with an image with the same number of pixels, but with fewer pixels. Helps to improve the computationally expensive problem just mentioned. Reducing an 8-megapixel image to 2 megapixels beforehand can make subsequent work easier.

Linear rectifier unit:

An important step in the CNN is the Rectified Linear Unit (ReLU), which mathematically converts all negative numbers on the image to 0. This trick prevents CNNs from approaching 0 or infinity. The result after linear rectification will have the same number of pixels as the original image, except that all negative values will be replaced with zeros.

Deep learning:

After being filtered, rectified, and pooled, the original image will become a set of smaller images containing feature information. These images can then be filtered and compressed again, and their features will become more complex with each processing, and the images will become smaller. The final, lower-level processing layers contain simple features such as corners or light spots. Higher-order processing layers contain more complex features, such as shapes or patterns, and these higher-order features are usually well-recognized.

Fully connected layer:

Fully connected layers will collect the filtered pictures at a high-level, and convert this feature information into votes. In the traditional neural network architecture, the role of the fully connected layer is the main primary building block. When we input an image to this unit, it treats all pixel values as a one-dimensional list, rather than the previous two-dimensional matrix. Each value in the list determines whether the symbol in the picture is a circle or a cross. Since some values are better at discriminating forks and others are better at discriminating circles, these values will get more votes than others. The number of votes cast by all values for different options will be expressed in terms of weight or connection strength. So, every time CNN judges a new image, the image will go through many lower layers before reaching the fully connected layer. After voting, the option with the most votes will become the category for this image.

Like other layers, multiple fully-connected layers can be combined because their inputs (lists) and outputs (votes) are in similar forms. In practice, it is possible to combine multiple fully-connected layers, with several virtual, hidden voting options appearing on several of them. Whenever add a fully connected layer, the entire neural network can learn more complex feature combinations and make more accurate judgments.

Backpropagation:

The machine learning trick of backpropagation can help us decide the weights. To use backpropagation, need to prepare some pictures that already have the answer, and then must prepare an untrained CNN where the values of any pixels, features, weights, and fully connected layers are randomly determined. You can train this CNN with a labeled image.

After CNN processing, each image will eventually have a round of the election to determine the category. Compared with the previously marked positive solution, it is the identification error. By adjusting the features and weights, the error generated by the election is reduced. After each adjustment, these features and weights are fine-tuned a little higher or lower, the error is recalculated, and the adjustments that successfully reduced the error are retained. So, when we adjust each pixel in the convolutional layer and each weight in the fully connected layer, we can get a set of weights that are slightly better at judging the current image. Then repeat the above steps to identify more tagged images. During the training process, misjudgments in individual pictures will pass, but common features and weights in these pictures will remain. If there are enough labeled images, the values of these features and weights will eventually approach a steady state that is good at recognizing most images. But backpropagation is also a very computationally expensive step.

Hyperparameters:

  • How many features should be in each convolutional layer? How many pixels should be in each feature?
  • What is the window size in each pooling layer? How long should the interval be?
  • How many hidden neurons (options) should each additional fully connected layer have?

In addition to these issues, we need to consider many high-level structural issues, such as how many processing layers should be in a CNN and in what order. Some deep neural networks may include thousands of processing layers, and there are many design possibilities. With so many permutations, we can only test a small subset of the CNN settings. Therefore, the design of CNN usually evolves with the knowledge accumulated by the machine learning community, and occasionally there are some unexpected improvements in performance. In addition, many improvement techniques have been tested and found to be effective, such as using new processing layers or connecting different processing layers in more complex ways.

Published by Oct 06, 2022 Source :mcknote

Further reading

You might also be interested in ...

Headline
Trend
Grinding Robots and Human Machine Collaboration
The integration of robotics into grinding processes can greatly transform traditional manufacturing into dynamic environments where human workers and robots collaborate seamlessly. While robotics offers precision, consistency, and efficiency, skilled operators are essential for the efficient operation of advanced grinding machines. Training programs are important to provide hands-on education, certification, and expertise in setup, operation, and troubleshooting for optimal performance.
Headline
Trend
Keyless Digital Electronic Door Locks: The Evolution of Security
We've all had the experience of returning home with our hands full, juggling packages while fumbling for keys. However, there are innovative solutions that prevent this predicament by eliminating the need for traditional keys. Keyless digital electronic door locks utilize a variety of technologies to provide secure, flexible access control without the traditional key. Advanced technologies that use various forms of authentication, such as codes, biometrics, and smartphones, not only streamline your entry process but also enhance the security of your home.
Headline
Trend
Refining the Essence: Three Fundamental Pillars of Smart Industrial Manufacturing
The conventional manufacturing sector stands at a crossroads necessitating a shift towards intelligent transformation. By incorporating advanced production technologies, a new era of industrial development is inaugurated.
Headline
Trend
The Role of Artificial Intelligence in Autonomous Vehicles
Utilizing machine learning and neural networks, artificial intelligence (AI) plays a crucial role in enabling the autonomous operation of self-driving cars. These vehicles leverage a combination of sensors, cameras, radar, and AI to navigate between destinations without the need for human intervention. For a car to be considered fully autonomous, it should demonstrate the capability to independently navigate predetermined routes without human input, even on roads that have not been specifically modified for autonomous vehicle use.
Headline
Trend
Worldwide Bicycle and Electric Bicycle Market Overview
The global increase in environmental consciousness has resulted in a shift for bicycles from primarily sporting and recreational roles to becoming popular modes of commuting. Notably, the rising adoption of electric bicycles is driven by factors such as an aging population, contributing to a significant upsurge in the global production of electric bicycles in recent years.
Headline
Trend
Opportunities and Trends in the Application of 5G in Smart Grids
In recent years, developed nations have initiated comprehensive power grid upgrade initiatives. In line with its commitment to energy conservation and carbon reduction policies, Taiwan has advanced the implementation of Automated Metering Infrastructure (AMI) as part of its national energy-saving strategy. The plan encompasses the integration of 4G/5G and other communication industries. The noteworthy progress in the development and integration of smart grid applications with 5G communication technology represents a significant industrial advancement deserving of attention.
Headline
Trend
Confronting the Era of Digital Advancement, Facial Recognition Technology Has Enhanced
Recently, there has been widespread discussion about Artificial Intelligence, Machine Learning, Deep Learning, and Big Data. These technologies find application in various domains such as the financial industry, logistics, business analysis, unmanned vehicles, computer vision, natural language processing, and more, permeating every facet of daily life.
Headline
Trend
The Arrival of 5G Technology Marks a Shift in Business Transformation, Redefining Innovations in the Manufacturing Sector
5G is recognized as a key enabler of Industry 4.0. With its high network speed and low power consumption, 5G facilitates the connectivity of every sensor in the upcoming unmanned factory to the cloud. This connectivity allows for the extraction of data for analysis, ultimately fueling advancements in artificial intelligence.
Headline
Trend
How Can Humans Collaborate with Robots in a Work Environment?
The integration of collaborative robots into production has become a pivotal element in the manufacturing chain, enhancing overall production efficiency. These compact collaborative industrial robots are designed to operate in confined spaces, addressing challenges posed by limited working spaces.
Headline
Trend
Can 3D Printing Be Applied in the Die and Mold Industry?
As the utilization of 3D printing expands across the broader spectrum of industrial manufacturing, the significance of this technology extends beyond its role as a rapid prototyping tool. This article provides an overview of the applications of 3D printing in the fabrication of molds and dies for processes such as injection molding and die casting.
Headline
Trend
Tooling 4.0: Bridging Industry 4.0 with Mold Manufacturing for the Future
Are you familiar with the latest terminology related to Tooling 4.0? In this article, we'll offer an overview and examples that can help manufacturers understand and align with this evolving concept. Tooling 4.0 revolves around leveraging technology to transform 'inefficient' products into 'intelligent' ones.
Headline
Trend
Industry 4.0 Propels the Global Industrial Market Towards Automation
In the present day, conventional industries are blending Internet of Things technology to drive the evolution of Industry 4.0 and the advancement of smart manufacturing.
Agree