How do Convolutional Neural Networks Work?
Trend

How do Convolutional Neural Networks Work?

Breakthroughs in deep learning in recent years have come from the development of Convolutional Neural Networks (CNNs or ConvNets). It is the main force in the development of the deep neural network field, and it can even be more accurate than humans in image recognition.
Published: Oct 06, 2022
How do Convolutional Neural Networks Work?

What is a Convolutional Neural Network?

Convolutional Neural Network is a feed-forward neural network whose artificial neurons can respond to surrounding units within a partial coverage area and has excellent performance for large-scale image processing. A convolutional neural network consists of one or more convolutional layers and a top fully connected layer, as well as associated weights and pooling layers. This structure enables convolutional neural networks to exploit the two-dimensional structure of the input data. Compared to other deep learning architectures, convolutional neural networks can give better results in image and speech recognition. This model can also be trained using the backpropagation algorithm. Compared to other deep, feed-forward neural networks, convolutional neural networks have fewer parameters to consider, making them an attractive deep learning architecture.

The Convolutional Neural Network is powerful in image recognition, and many image recognition models are also extended based on the CNN architecture. It is also worth mentioning that the CNN model is a deep learning model established by referring to the visual organization of the human brain. Learning CNN will help me learn other deep learning models.

Feature:

CNN compares parts of the image, which are called features. By comparing rough features at similar locations, CNNs are better at distinguishing whether images are the same or not, rather than comparing whole images. Each feature in an image is like a smaller image, that is, a smaller two-dimensional matrix and these features capture common elements in the image.

Convolution:

Whenever a CNN resolves a new image, without knowing where the above features are, the CNN compares anywhere in the image. To calculate how many matching features are in the whole image, we create a filtering mechanism here. The mathematical principle behind this mechanism is called convolution, which is where the name CNN comes from.

The basic principle of convolution is to calculate the degree of conformity between the feature and the image part, if the value of each pixel of the two is multiplied, and then the sum is divided by the number of pixels. If every pixel of the two images matches, sum these products and divide by the number of pixels to get 1. Conversely, if the two pixels are completely different, you will get -1. By repeating the above process and summarizing various possible features in the image, convolution can be completed. Based on the values and positions of each convolution, make a new 2D matrix. This is the original image filtered by the feature, which can tell us where to find the feature in the original image. The part with a value closer to 1 is more consistent with the feature, the closer the value is to -1, the greater the difference; as for the part with a value close to 0, there is almost no similarity at all. The next step is to apply the same method to different features, and convolutions in various parts of the image. Finally, we will get a set of filtered original images, each of which corresponds to a feature. Simply think of the entire convolution operation as a single processing step. In the operation of CNNs, this step is called a convolutional layer, which means that there are more layers to follow.

The operation principle of CNN is computationally intensive. While we can explain how a CNN works on just one piece of paper, the number of additions, multiplications, and divisions can increase quickly along the way. With so many factors affecting the number of computations, the problems that CNN's deal with can become complex with little effort, and it is no wonder that some chipmakers are designing and building specialized chips for the computational demands of CNNs.

Pooling:

Pooling is a method of compressing images and retaining important information. Its working principle can be understood with only a second degree in mathematics. Pooling will select different windows on the image, and select a maximum value within this window range. In practice, a square with a side length of two or three is an ideal setting with a two-pixel stride.

After the original image is pooled, the number of pixels it contains will be reduced to a quarter of the original, but because the pooled image contains the maximum value of each range in the original image, it still retains each range and each range. The degree of conformity of the characteristics. The pooled information is more focused on whether there are matching features in the image, rather than where these features exist in the image. Can help CNN to determine whether a feature is included in the image without having to be distracted by the location of the feature.

The function of the pooling layer is to pool one or some pictures into smaller pictures. We end up with an image with the same number of pixels, but with fewer pixels. Helps to improve the computationally expensive problem just mentioned. Reducing an 8-megapixel image to 2 megapixels beforehand can make subsequent work easier.

Linear rectifier unit:

An important step in the CNN is the Rectified Linear Unit (ReLU), which mathematically converts all negative numbers on the image to 0. This trick prevents CNNs from approaching 0 or infinity. The result after linear rectification will have the same number of pixels as the original image, except that all negative values will be replaced with zeros.

Deep learning:

After being filtered, rectified, and pooled, the original image will become a set of smaller images containing feature information. These images can then be filtered and compressed again, and their features will become more complex with each processing, and the images will become smaller. The final, lower-level processing layers contain simple features such as corners or light spots. Higher-order processing layers contain more complex features, such as shapes or patterns, and these higher-order features are usually well-recognized.

Fully connected layer:

Fully connected layers will collect the filtered pictures at a high-level, and convert this feature information into votes. In the traditional neural network architecture, the role of the fully connected layer is the main primary building block. When we input an image to this unit, it treats all pixel values as a one-dimensional list, rather than the previous two-dimensional matrix. Each value in the list determines whether the symbol in the picture is a circle or a cross. Since some values are better at discriminating forks and others are better at discriminating circles, these values will get more votes than others. The number of votes cast by all values for different options will be expressed in terms of weight or connection strength. So, every time CNN judges a new image, the image will go through many lower layers before reaching the fully connected layer. After voting, the option with the most votes will become the category for this image.

Like other layers, multiple fully-connected layers can be combined because their inputs (lists) and outputs (votes) are in similar forms. In practice, it is possible to combine multiple fully-connected layers, with several virtual, hidden voting options appearing on several of them. Whenever add a fully connected layer, the entire neural network can learn more complex feature combinations and make more accurate judgments.

Backpropagation:

The machine learning trick of backpropagation can help us decide the weights. To use backpropagation, need to prepare some pictures that already have the answer, and then must prepare an untrained CNN where the values of any pixels, features, weights, and fully connected layers are randomly determined. You can train this CNN with a labeled image.

After CNN processing, each image will eventually have a round of the election to determine the category. Compared with the previously marked positive solution, it is the identification error. By adjusting the features and weights, the error generated by the election is reduced. After each adjustment, these features and weights are fine-tuned a little higher or lower, the error is recalculated, and the adjustments that successfully reduced the error are retained. So, when we adjust each pixel in the convolutional layer and each weight in the fully connected layer, we can get a set of weights that are slightly better at judging the current image. Then repeat the above steps to identify more tagged images. During the training process, misjudgments in individual pictures will pass, but common features and weights in these pictures will remain. If there are enough labeled images, the values of these features and weights will eventually approach a steady state that is good at recognizing most images. But backpropagation is also a very computationally expensive step.

Hyperparameters:

  • How many features should be in each convolutional layer? How many pixels should be in each feature?
  • What is the window size in each pooling layer? How long should the interval be?
  • How many hidden neurons (options) should each additional fully connected layer have?

In addition to these issues, we need to consider many high-level structural issues, such as how many processing layers should be in a CNN and in what order. Some deep neural networks may include thousands of processing layers, and there are many design possibilities. With so many permutations, we can only test a small subset of the CNN settings. Therefore, the design of CNN usually evolves with the knowledge accumulated by the machine learning community, and occasionally there are some unexpected improvements in performance. In addition, many improvement techniques have been tested and found to be effective, such as using new processing layers or connecting different processing layers in more complex ways.

Published by Oct 06, 2022 Source :mcknote

Further reading

You might also be interested in ...

Headline
Trend
Development of the Wireless Power Industry and Future Electrification
The wireless power industry is expected to grow exponentially. Wireless power has a major impact on almost all fields because it enables the Internet of Things to achieve and develop faster.
Headline
Trend
Status and Trends of the Biotechnology Pharmaceutical Industry
The growth of the overall global biotech pharmaceutical market is slowing down. Europe and the United States have entered a mature period, and global development in the overall biotech pharmaceutical industry market has slowed down in recent years.
Headline
Trend
Time to Start Digital Reinvention for Enterprises
New technologies bring forth the new. AI and the cloud are irreversible trends that are strengthening the competitiveness of enterprises. Enterprises should consider how to implement AI and the cloud in each application of the enterprises.
Headline
Trend
Low-Carbon Production and Global Climate Change
Countries are actively developing heavy industries, manufacturing, and infrastructure to drive market growth, but they often pay environmental costs accordingly. Approximately 70% of global greenhouse gas emissions come from infrastructure construction and operations, such as power plants, buildings, and transportation. What will be the effect of promoting society's emphasis on low-carbon manufacturing?
Headline
Trend
How Does AI Integrate with Manufacturing Industry?
Improving production efficiency is the foundation for the manufacturing industry to gain a firm foothold. The manufacturing industry achieves smart operations by introducing AI applications, automatically identifying abnormalities, or making adjustment suggestions, and assisting companies in achieving more accurate adjustments to machines and upgrading equipment. During the process, the traditional manufacturers are transformed into the smart manufacturers.
Headline
Trend
Growing Factors of the Global Cloud Server Market
The cloud server market is a cloud infrastructure service that allows service providers and end-users to use virtual networks to build architectures.
Headline
Trend
Face Recognition Technology Has Improved in Digital Age
Artificial Intelligence, Machine Learning, Deep Learning, and Big Data are topics that have been frequently discussed recently. These technologies are used in many fields, including the financial industry, logistics industry, business analysis, unmanned vehicles, computer vision, natural language processing, etc., and have spread in every corner of life.
Headline
Trend
Global Market Trend of Smart Lock Industry
In recent years, smart door lock technology has developed rapidly. As an important part of smart home access control, the smart lock market will grow substantially in the next few years.
Headline
Trend
The Developments of Optical Instrument and Equipment Manufacturing Industry
The optical industry is still the mainstream product sector in consumer electronics, and there is still a lot of room for innovation. Since smartphones are equipped with optical lenses, from the rear to the front, and from single-lens to multi-lens, hardware innovation not only facilitates consumers to take photos and videos but also directly drives the vigorous development of the optical industry.
Headline
Trend
The Developments of Optoelectronic Materials and Components Manufacturing
The optoelectronic material and component manufacturing industry is a branch of the electronic component manufacturing industry. It is a high-tech and capital-intensive industry. Components manufactured with optoelectronic technology or equipment with optoelectronic components as key components are all parts of the optoelectronics industry.
Headline
Trend
The Trends of Taiwan General Equipment and Machinery Manufacturing Industry
In recent years, Taiwan’s "mechanical transmission equipment industry", "pumps, compressors, stopcock and valve industry", "other general equipment and machinery", and other relevant industries have increased their production indexes, mainly because of the active demands for smart automation equipment which has driven the increase in overall output.
Headline
Trend
What Happen to the Market Demands of Medicine and Medical after Epidemic?
Due to the increases in demands for epidemic related drugs, screening reagents, and other products driven by the epidemic, the output values of drugs, medical supplies, and other related products have been increased.
Agree