Deep Learning

Machine learning solves the problem of constructing a forecast based on the input description of the object under study, while the parameters of the forecasting function or model are not specified explicitly, but are determined automatically as a result of the training procedure (model training) on ​​the so-called training set – a labeled (in a supervised problem ) or unlabeled (in an unsupervised problem ) set of objects.

Automatic parameter tuning significantly simplifies and accelerates the construction of predictive models. It also enables the use of more complex models containing a large number of automatically tuned parameters, which improves forecast accuracy.

Deep learning is a subfield of machine learning that solves the same problems using more complex, multi-layered computations that automatically extract more informative features from the original data.

Before learning deep learning, make sure you understand the basic machine learning topics :

• Machine Learning Basics
• Data preparation
• Classifiers in general
• Regression quality assessment
• Evaluation of classification quality
• Overfitting and underfitting

In the following, it will be assumed that the reader is familiar with these sections.

An optional but useful chapter for interpreting the performance of deep learning models is the chapter on interpreting complex models.

The principle of deep 

Objects are represented in the form of a raw representation of the objects.

For example, in image processing, a low-level representation of an image would be:

• pixel intensity matrix
  $\mathrm{<span\; dir="auto">I</span>}<span\; dir="auto">\in </span>{\mathbb{<span\; dir="auto">R</span>}}^{\mathrm{<span\; dir="auto">H</span>}<span\; dir="auto">\times </span>\mathrm{<span\; dir="auto">W</span>}}$ 

  I∈RH × W(in case of black and white image

  $\mathrm{<span\; dir="auto">H</span>}<span\; dir="auto">\times </span>\mathrm{<span\; dir="auto">W</span>}$ 

  $H\times W$);

• intensity tensor
  $\mathrm{<span\; dir="auto">I</span>}<span\; dir="auto">\in </span>{\mathbb{<span\; dir="auto">R</span>}}^{\mathrm{<span\; dir="auto">3</span>}<span\; dir="auto">\times </span>\mathrm{<span\; dir="auto">H</span>}<span\; dir="auto">\times </span>\mathrm{<span\; dir="auto">W</span>}}$ 

  I∈R3 x H x W, representing the union of 3 matrices for the red, green, and blue channels (in the case of a color image

  $\mathrm{<span\; dir="auto">H</span>}<span\; dir="auto">\times </span>\mathrm{<span\; dir="auto">W</span>}$ 

  $H\times W$).

When processing sounds, the low-level representation will be a sequence of amplitudes (sound wave strengths) at each moment in time.

Applying a predictive model to a low-level representation is impractical – the dimensionality of the feature space is too large, so the model must be tuned on a small number of high-level and informative features (high-level representation).

The traditional approach in machine learning, called shallow learning, relies on humans to generate high-level features for a predictive model:

For images, for example, one can construct the distribution of colors across the red, green, and blue channels as features, calculating their means and standard deviations. For sounds, one can calculate the average sound wave strength, its standard deviation, the number and duration of pauses, etc.

The limitations of this approach are immediately clear:

• It is necessary to spend limited human resources on developing features and creating a procedure for their extraction (slow and time-consuming);
• In any case, these will be simple transformations (not efficient enough for the final task).

In deep learning, not only is the model configured, but also a sequence of transformations that generate features that the final model will use to make predictions:

Each transformation generates an intermediate representation of features that becomes more complex and informative with each subsequent one.

For example, in the case of images, first the boundaries will be extracted, then the corners, then the geometric shapes, and starting at a certain stage, complex objects will be extracted, such as the human eye, the wheel of a car, the window of a house, etc., based on which it will then be easy to solve the final problem (for example, classify what exactly is shown in the image).

Advantages of the approach:

• Extraction of informative features occurs automatically based on the data—just as predictive models are tuned in machine learning; there is no need to waste human resources on manual feature extraction.
• Features are selected faster, and these will be more complex and more suitable features for the final task, obtained as a result of multidimensional optimization.

Deep learning requires much more training data to apply , as it is now not only the model parameters that need to be tuned, but also the parameters of the intermediate feature transformations!

For images, at least tens of thousands of labeled examples are required. More complex models require training samples (called datasets ) containing several million training examples, such as ImageNet [1], [2].

The principle of deep learning has also been successfully applied to other areas, such as text, speech, and graph processing.

What previously studied classical machine learning approach is ideologically similar to deep learning?

To implement the principle of deep learning, neural networks are used, since a neural network is a sequence of nonlinear transformations that describe the sequential transformation of features and the construction of a forecast based on them.

Neural networks demonstrate excellent results and are often capable of solving a wide range of problems faster and better than the average person, not only in cases where you need to predict a number (regression) or category (classification), but also in more creative tasks where you need to generate an image, text, sound (for example, generating vocals from song lyrics), or a graph (describing the chemical compound of a substance or drug).

Strong and weak artificial 

Solving specific formalized problems using machine learning methods is called weak artificial intelligence (or applied AI, narrow AI).

There is also a hypothesis in the scientific community about artificial general intelligence (AGI ) [3], capable of solving any problem through self-learning and development. A major step toward creating artificial general intelligence has been the development of large language models, such as ChatGPT, capable of maintaining conversations and answering general questions.

There’s also the hypothesis of strong artificial intelligence (strong AI), capable of thinking and self-awareness as a separate individual (artificial consciousness). To what extent an artificially created system is theoretically capable of this is a major philosophical question.

The author of the book does not share this hypothesis. Most likely, a general artificial intelligence capable of accurately imitating the self-awareness of living people will be created.

More details about the types of artificial intelligence by the level of tasks solved can be found in [4].

Development of deep 

Deep learning received a boost in the 2010s with the advent of

• available computing power capable of performing large volumes of calculations (graphics accelerators, FPGA chips);
• Large training samples containing millions of labeled observations.

Deep learning is literally revolutionizing economics, politics, and social life. Deep neural networks enable faster and more efficient stock trading (see algorithmic trading [5] ), manage technological processes, recognize people in video surveillance systems, track and predict customer behavior based on their online behavior, movements, and financial transactions, generate realistic text, images, sounds, and videos virtually indistinguishable from the real thing, and even compile program code on demand. Neural networks are gradually displacing humans even in creative professions such as storytelling, graphic design, and music production.

Ethical 

The rapid development of artificial intelligence technologies brings not only opportunities but also challenges, which are widely discussed not only in the expert community but also among ordinary users.

The risks of deep learning are that its technologies

• Lead to the displacement of people from many professions;
• Allow the creation of fake news that is virtually indistinguishable from the real thing;
• Contribute to the fact that people’s immersion and involvement shift from the real world to the virtual one;
• Give very great power over society, and the technologies are concentrated in a narrow circle of large companies that have the data and equipment to implement and develop this science.

Deep learning students need to ask themselves how much their work is leading to positive change in society. Is it making society more free, expanding its possibilities, or, conversely, making it hostage to technology and the companies that control it?

Technologies exist for people, not people for technologies.

There are no clear and simple answers yet on how to address the challenges of new technologies so that society can reap their benefits without becoming hostage to them. This is a complex ethical issue that must be addressed jointly by governments, technology companies, and social movements. Interested readers can explore this issue in more detail in the book “The Coming Wave: AI, Power, and Our Future” by DeepMind and Inflection AI co-founder Mustafa Suleiman [6].

References

1. Wikipedia: ImageNet.
2. Official website of the ImageNet dataset.
3. Wikipedia: Artificial general intelligence.
4. ibm.com: What is artificial general intelligence (AGI)?
5. Wikipedia: Algorithmic trading.
6. Suleyman M. The coming wave: AI, power, and our future. – Random House, 2025.

Deep Learning

Explore More IT Terms