Deep learning is the branch of machine learning that uses neural networks with many layers to learn hierarchical representations of the data. Its distinctive trait compared with classic machine learning is that it learns the features by itself: where an engineer used to manually design what to measure from an image, a deep network discovers the edges on its own, then the shapes and finally the objects, layer by layer. This article explains how a neural network really works, walks through the three great architectures—CNN, RNN and transformers—and pins down which problems each one solves and where their practical limits lie.
How a neural network learns: the essential mechanics
A neural network is made up of neurons organised into layers. Each neuron receives inputs, multiplies them by some weights, adds a bias and applies a non-linear activation function (such as ReLU). That non-linearity is what allows the network to approximate complex relationships; without it, stacking layers would be equivalent to a simple linear regression.
Learning consists of adjusting those weights. In forward propagation the data passes through the network and produces a prediction; a loss function measures how far off it was. Then the backpropagation algorithm calculates, using the chain rule, how much each weight contributed to the error, and gradient descent corrects them in the direction that reduces the loss. Repeated millions of times over the training data, this process sculpts a network able to generalise. The term "deep" refers precisely to the number of layers: more layers allow more abstract representations, at the cost of greater difficulty in training.
Convolutional networks (CNN): the domain of vision
Convolutional neural networks revolutionised computer vision. Their central idea is the convolution: instead of connecting every pixel to every neuron—unfeasible in large images—a small filter is slid across the whole image, detecting a local pattern (an edge, a texture) wherever it appears. This translation invariance and the shared use of weights drastically reduce the parameters and capture the spatial structure.
The initial layers of a CNN detect simple features; the deep ones, increasingly complex combinations until they recognise whole objects. Their real applications span quality control through visual inspection in factories, medical image analysis, facial recognition and the perception of autonomous vehicles.
Recurrent networks (RNN/LSTM): data in sequence
When order matters—text, speech, time series—recurrent networks come in. Unlike a standard network, an RNN keeps an internal state that acts as a memory of what has been processed so far, which lets it take the previous context into account. Its best-known version, the LSTM (long short-term memory), introduces "gates" that decide what information to remember and what to forget, mitigating the vanishing-gradient problem that afflicted simple RNNs on long sequences. For years they were the standard in machine translation, speech recognition and time-series prediction.
Transformers: the architecture that changed the rules
In 2017, the paper "Attention Is All You Need" introduced the transformer, today the dominant architecture. Its innovation is the self-attention mechanism: instead of processing the sequence word by word like an RNN, the transformer weighs the relationship of each element with all the others all at once, capturing long-distance dependencies and enabling massively parallel training. That parallelisation is what has made it possible to train models of a size previously unthinkable.
Transformers are the basis of the large language models (LLMs) and of today's vision models. Their ability to scale with data and compute has defined the present generation of generative AI, from text understanding and generation to multimodal models that combine text and image.
| Architecture | Data type | Key mechanism | Typical application |
|---|---|---|---|
| CNN | Images, spatial data | Convolution + shared weights | Vision, quality inspection, medical imaging |
| RNN / LSTM | Sequences, time series | Internal state (memory) + gates | Speech, time prediction, classic translation |
| Transformer | Long sequences, multimodal | Self-attention + parallelisation | LLMs, NLP, generative AI, modern vision |
Practical requirements: data, compute and transfer learning
Deep learning has a high cost of entry. It needs large volumes of labelled data and intensive compute on GPUs or specialised accelerators, which entails a far from negligible energy and economic cost. The good news for most companies is transfer learning: starting from a model already pre-trained on a gigantic corpus and retraining only the last layers with a much smaller, in-house dataset. This technique democratises deep learning, because it avoids training from scratch and reduces the data and compute requirements by orders of magnitude.
To take a deep model to production without sending the cost soaring, the sector also turns to model optimisation techniques. Quantisation reduces the numerical precision of the weights (for example, from 32 to 8 bits) and pruning removes low-relevance connections, both with the aim of lightening the model so it runs inference faster and with less memory. Knowledge distillation, for its part, trains a small, cheap model (the "student") to imitate a large, expensive one (the "teacher"), preserving much of its capability. These techniques are what make it possible to deploy deep learning capabilities on modest devices or in services with large request volumes, where training is occasional but inference happens millions of times.
Limits and responsible considerations
A dose of realism is in order. For tabular business data, classic methods—gradient-boosted trees—often match or beat deep learning at a fraction of the cost; deep learning shines on unstructured data, not on everything. Moreover, deep networks are black boxes: explaining why they made a particular decision is difficult, which clashes with the transparency requirements of the EU AI Regulation (AI Act, Regulation 2024/1689) for high-risk systems. Added to this are the risk of inheriting and amplifying biases present in the training data and the phenomenon of "hallucinations" in generative models. A responsible deployment provides for human oversight, bias evaluation and, when the regulation requires it, explainable AI techniques.
Common mistakes when applying deep learning
- Using deep learning for everything: applying it to a simple tabular problem is using a sledgehammer to crack a nut; it adds cost and opacity without improving the result.
- Training from scratch when transfer learning is available: it wastes data, time and energy when a pre-trained model would solve the problem.
- Confusing fluency with truthfulness: a generative model produces convincing text even when it is wrong; without verification, the hallucinations reach the user.
- Ignoring data bias: if the training set is biased, the network learns it and perpetuates it at scale.
- Forgetting explainability in regulated systems: deploying a black box in a high-risk decision without the ability to justify it breaches the AI Act.
Frequently asked questions
What is the difference between deep learning and machine learning?
Deep learning is a subset of machine learning that uses deep neural networks and learns the features automatically. Classic machine learning usually requires an expert to design those features by hand, but it works very well with less data and less compute, especially on tabular data.
Why did transformers displace recurrent networks?
Because their self-attention mechanism processes the whole sequence in parallel—not element by element—and captures the dependencies between distant parts better. This makes it possible to train far larger and faster models, which proved decisive for the leap in quality of language models.
Do I need a GPU to do deep learning?
To train models from scratch, yes: compute on GPUs or accelerators is practically indispensable. However, with transfer learning and cloud services, many companies fine-tune pre-trained models with modest resources, and even run inference on conventional hardware.
What is transfer learning and why does it matter so much?
It is reusing a model trained on a large corpus and adapting it to your own task by retraining only a part. It matters because it drastically reduces the data, time and cost required, putting deep learning within reach of organisations that could not train from scratch.
Conclusion
Deep learning is not a universal magic wand but an extraordinarily powerful tool for a specific terrain: unstructured data such as images, speech and language. Understanding the genealogy of its architectures—from the convolution that sees, to the recurrence that remembers, to the attention that relates everything to everything—makes it possible to choose the right one rather than applying the one in fashion. And in a European context, its power comes with responsibility: the opacity of deep networks obliges incorporating human oversight, bias control and explainability when the AI Act requires it. Whoever combines the right architecture, transfer learning to make the path cheaper and responsible design from the start extracts the real value of deep learning. At Summum Artificial Intelligence we design deep learning solutions matched to the problem and to the regulation, without over-engineering the tool.