How do Convolutional Neural Networks Work?
Trend

How do Convolutional Neural Networks Work?

Breakthroughs in deep learning in recent years have come from the development of Convolutional Neural Networks (CNNs or ConvNets). It is the main force in the development of the deep neural network field, and it can even be more accurate than humans in image recognition.
Published: Oct 06, 2022
How do Convolutional Neural Networks Work?

What is a Convolutional Neural Network?

Convolutional Neural Network is a feed-forward neural network whose artificial neurons can respond to surrounding units within a partial coverage area and has excellent performance for large-scale image processing. A convolutional neural network consists of one or more convolutional layers and a top fully connected layer, as well as associated weights and pooling layers. This structure enables convolutional neural networks to exploit the two-dimensional structure of the input data. Compared to other deep learning architectures, convolutional neural networks can give better results in image and speech recognition. This model can also be trained using the backpropagation algorithm. Compared to other deep, feed-forward neural networks, convolutional neural networks have fewer parameters to consider, making them an attractive deep learning architecture.

The Convolutional Neural Network is powerful in image recognition, and many image recognition models are also extended based on the CNN architecture. It is also worth mentioning that the CNN model is a deep learning model established by referring to the visual organization of the human brain. Learning CNN will help me learn other deep learning models.

Feature:

CNN compares parts of the image, which are called features. By comparing rough features at similar locations, CNNs are better at distinguishing whether images are the same or not, rather than comparing whole images. Each feature in an image is like a smaller image, that is, a smaller two-dimensional matrix and these features capture common elements in the image.

Convolution:

Whenever a CNN resolves a new image, without knowing where the above features are, the CNN compares anywhere in the image. To calculate how many matching features are in the whole image, we create a filtering mechanism here. The mathematical principle behind this mechanism is called convolution, which is where the name CNN comes from.

The basic principle of convolution is to calculate the degree of conformity between the feature and the image part, if the value of each pixel of the two is multiplied, and then the sum is divided by the number of pixels. If every pixel of the two images matches, sum these products and divide by the number of pixels to get 1. Conversely, if the two pixels are completely different, you will get -1. By repeating the above process and summarizing various possible features in the image, convolution can be completed. Based on the values and positions of each convolution, make a new 2D matrix. This is the original image filtered by the feature, which can tell us where to find the feature in the original image. The part with a value closer to 1 is more consistent with the feature, the closer the value is to -1, the greater the difference; as for the part with a value close to 0, there is almost no similarity at all. The next step is to apply the same method to different features, and convolutions in various parts of the image. Finally, we will get a set of filtered original images, each of which corresponds to a feature. Simply think of the entire convolution operation as a single processing step. In the operation of CNNs, this step is called a convolutional layer, which means that there are more layers to follow.

The operation principle of CNN is computationally intensive. While we can explain how a CNN works on just one piece of paper, the number of additions, multiplications, and divisions can increase quickly along the way. With so many factors affecting the number of computations, the problems that CNN's deal with can become complex with little effort, and it is no wonder that some chipmakers are designing and building specialized chips for the computational demands of CNNs.

Pooling:

Pooling is a method of compressing images and retaining important information. Its working principle can be understood with only a second degree in mathematics. Pooling will select different windows on the image, and select a maximum value within this window range. In practice, a square with a side length of two or three is an ideal setting with a two-pixel stride.

After the original image is pooled, the number of pixels it contains will be reduced to a quarter of the original, but because the pooled image contains the maximum value of each range in the original image, it still retains each range and each range. The degree of conformity of the characteristics. The pooled information is more focused on whether there are matching features in the image, rather than where these features exist in the image. Can help CNN to determine whether a feature is included in the image without having to be distracted by the location of the feature.

The function of the pooling layer is to pool one or some pictures into smaller pictures. We end up with an image with the same number of pixels, but with fewer pixels. Helps to improve the computationally expensive problem just mentioned. Reducing an 8-megapixel image to 2 megapixels beforehand can make subsequent work easier.

Linear rectifier unit:

An important step in the CNN is the Rectified Linear Unit (ReLU), which mathematically converts all negative numbers on the image to 0. This trick prevents CNNs from approaching 0 or infinity. The result after linear rectification will have the same number of pixels as the original image, except that all negative values will be replaced with zeros.

Deep learning:

After being filtered, rectified, and pooled, the original image will become a set of smaller images containing feature information. These images can then be filtered and compressed again, and their features will become more complex with each processing, and the images will become smaller. The final, lower-level processing layers contain simple features such as corners or light spots. Higher-order processing layers contain more complex features, such as shapes or patterns, and these higher-order features are usually well-recognized.

Fully connected layer:

Fully connected layers will collect the filtered pictures at a high-level, and convert this feature information into votes. In the traditional neural network architecture, the role of the fully connected layer is the main primary building block. When we input an image to this unit, it treats all pixel values as a one-dimensional list, rather than the previous two-dimensional matrix. Each value in the list determines whether the symbol in the picture is a circle or a cross. Since some values are better at discriminating forks and others are better at discriminating circles, these values will get more votes than others. The number of votes cast by all values for different options will be expressed in terms of weight or connection strength. So, every time CNN judges a new image, the image will go through many lower layers before reaching the fully connected layer. After voting, the option with the most votes will become the category for this image.

Like other layers, multiple fully-connected layers can be combined because their inputs (lists) and outputs (votes) are in similar forms. In practice, it is possible to combine multiple fully-connected layers, with several virtual, hidden voting options appearing on several of them. Whenever add a fully connected layer, the entire neural network can learn more complex feature combinations and make more accurate judgments.

Backpropagation:

The machine learning trick of backpropagation can help us decide the weights. To use backpropagation, need to prepare some pictures that already have the answer, and then must prepare an untrained CNN where the values of any pixels, features, weights, and fully connected layers are randomly determined. You can train this CNN with a labeled image.

After CNN processing, each image will eventually have a round of the election to determine the category. Compared with the previously marked positive solution, it is the identification error. By adjusting the features and weights, the error generated by the election is reduced. After each adjustment, these features and weights are fine-tuned a little higher or lower, the error is recalculated, and the adjustments that successfully reduced the error are retained. So, when we adjust each pixel in the convolutional layer and each weight in the fully connected layer, we can get a set of weights that are slightly better at judging the current image. Then repeat the above steps to identify more tagged images. During the training process, misjudgments in individual pictures will pass, but common features and weights in these pictures will remain. If there are enough labeled images, the values of these features and weights will eventually approach a steady state that is good at recognizing most images. But backpropagation is also a very computationally expensive step.

Hyperparameters:

  • How many features should be in each convolutional layer? How many pixels should be in each feature?
  • What is the window size in each pooling layer? How long should the interval be?
  • How many hidden neurons (options) should each additional fully connected layer have?

In addition to these issues, we need to consider many high-level structural issues, such as how many processing layers should be in a CNN and in what order. Some deep neural networks may include thousands of processing layers, and there are many design possibilities. With so many permutations, we can only test a small subset of the CNN settings. Therefore, the design of CNN usually evolves with the knowledge accumulated by the machine learning community, and occasionally there are some unexpected improvements in performance. In addition, many improvement techniques have been tested and found to be effective, such as using new processing layers or connecting different processing layers in more complex ways.

Published by Oct 06, 2022 Source :mcknote

Further reading

You might also be interested in ...

Headline
Trend
The Trend of Smart Logistics - Fast Logistics, Decentralized, Unmanned
The rising trend of e-commerce and retail online shopping has directly led to the improvement of distribution performance. In addition to the continuous expansion of the operating territory of local distribution companies, offline retail and mass stores have begun to carry out a series of measures to shorten delivery time and improve service quality.
Headline
Trend
How does E-commerce Create the Future of Warehousing?
E-commerce has taken the retail industry by storm and is expected to continue to grow in the coming years. At the same time, consumer spending habits change, as well as changes in the way suppliers and manufacturers respond to this growth. The rise of e-commerce in particular is changing the way warehouses are designed and operated.
Headline
Trend
Market Trends in Transportation Technology
Electric vehicles are only a small part of the future mobility trend. The future trend is not only electric vehicles, but also generates hidden business opportunities, and the opportunities are endless.
Headline
Trend
Why do We Need to Digitize Energy?
The demand for renewable energy is increasing day by day. Facing the instability of wind energy and solar energy, energy digitization can increase the productivity and availability of green electricity, such as using algorithms and blockchain technology to reduce consumption and improve flexibility.
Headline
Trend
What is the Computer-Aided Diagnosis? Promote the Development of Smart Medical Care
Refers to the use of imaging, medical image processing technology, and other possible physiological and biochemical means, combined with computer analysis and calculation, to assist radiologists in finding lesions and improve the accuracy of diagnosis.
Headline
Trend
What are The Green Power Generation? Understand The Types Of Green Energy Power Generation, Country Evaluation and Related Policies
After the first year of green power in 2020, 2021 will usher in a new peak period of green energy development. So far, in addition to Google, Apple, Amazon and other major technology companies that have adopted nearly 100% of their green energy power generation, many multinational companies have also stepped into the ranks of green energy investment. After all, how does the green energy industry flourish internationally? Follow the information compiled in this article to understand the types of green energy power generation, the application situation in various countries, and take the lead in grasping the trend of green energy power generation!
Headline
Trend
Advantages and Applications of Chatbots
The introduction of chatbots into conversational commerce has become a trend of transformation, but for small and medium-sized enterprises, the cost of introduction is high, there is a shortage of information talents, and the amount of member data is insufficient. What benefits can enterprise get from investing in the introduction system?
Headline
Trend
Packaging Industry Trends - What is Flexible Packaging and Why is It Important?
Why is flexible packaging and will continue to be a premium packaging solution?
Headline
Trend
Data-driven Development of Smart Manufacturing
How to make good use of digital technology to analyze, formulate strategies, and respond quickly at the core of enterprise operations has become the key to development.
Headline
Trend
Understand 5G at Once!
5G is the fifth generation of mobile data technology. Designed to increase speed, reduce latency, and improve wireless service resiliency. In theory, 5G technology can reach speeds as fast as 20 Gbps, while 4G's top speed is only 1 Gbps. 5G also has less latency, improving the performance of business applications and other digital experiences such as online gaming, video conferencing, and self-driving car performance.
Headline
Trend
Smart Manufacturing in the Era of the Industrial Internet of Things
To enhance competitiveness, manufacturers are driving a shift from a reactive problem-solving model to a more proactive device, process, product, and plant management model.
Headline
Trend
What is Cloud Service? Learn About Cloud Computing and Cloud Storage
As the global demand for cloud computing continues to double, cloud computing is not a whim, but a technology and various applications that have slowly developed from decades of information technology. With the rapid progress of the times and changes in the general environment, the public can enjoy cloud services at any time.
Agree