Skip to content
Getting Digital
Contact

Neural Networks

Also: artificial neural network, ANN, neural net

A neural network is a stack of layers of very simple arithmetic units whose connection strengths are adjusted by training until the whole arrangement turns the inputs you have into the outputs you want.

Assessment. The comparison with the brain has cost this subject more understanding than it ever bought, and teaching material should drop it. Call the thing what it is, function fitting at industrial scale, and its failures stop looking mysterious and start looking predictable.

A neural network is a large pile of trivial arithmetic arranged in layers and tuned by example. One unit takes the numbers arriving at it, multiplies each by a weight, adds them up and pushes the total through a small non-linear function. Nothing in that description is clever, and nothing in it is doing the interesting work. The interesting behaviour belongs to the scale of the pile and to the procedure that sets the weights. That procedure is comparison. The network produces an answer, the answer is scored against the one you wanted, and backpropagation apportions blame, calculating how much each weight contributed to the error. Gradient descent then shifts every weight a short distance in whichever direction would have made the error smaller. Run that loop over enough examples and the connection strengths settle into an arrangement that solves the task. Nobody picked those numbers and nobody can read them back afterwards, which is the bargain the entire field has made: you stop writing the rules down, and in exchange you give up the ability to inspect them.

What depth adds

Stacking is what moved this from curiosity to industry. A single layer of units can only cut fairly blunt boundaries through whatever it is shown. Put layers in sequence and each one operates on what the layer below it produced, so a description gets built in stages: edge-like fragments and textures near the input, parts and shapes further in, the decision at the end. Nobody specifies those stages. They appear because they are a convenient intermediate form for reducing the error, and that self-assembly is the whole argument for deep learning over the older practice of designing features by hand and passing them to a simpler classifier. The same mechanism explains the failure modes, which is where the biology metaphor earns its retirement. A network holds no model of the world to fall back on, so it will happily learn any regularity that lowers the error, including every one you did not intend. If each photograph of a given animal in your data happened to be taken on snow, snow becomes part of that animal. If past hiring decisions favoured one group, the pattern is sitting in the data and the network will find it, because finding patterns is the only thing it does. For anyone building with machine learning the consequence is unglamorous: the leverage lives in the dataset and the evaluation, not in the choice of architecture, and a team that swaps architectures before it fixes its labels is optimising the cheap end of its problem.

In practice

Take the oldest working application in the field, reading handwritten digits off a postal envelope. The input layer receives one number per pixel of a small greyscale image. The output layer holds ten units, one per digit, and the network's answer is whichever of them finishes with the highest value. Between the two sit the layers that do the work, and their weights begin as random numbers, so before training the network simply guesses. After being shown labelled envelopes and corrected each time, the identical architecture with different weights reads the post. Not one line of the code changed between those two states. Only the numbers did, which is why these systems are described as trained rather than written.

Often confused with

Deep Learning
Not a rival technique. Deep learning is what you call a neural network once it has enough layers to build its own intermediate representations.
Machine Learning
The wider category. Plenty of machine learning uses decision trees, regression or clustering and contains no network at all.
Transformers
One particular wiring diagram for a neural network, aimed at sequences; it is not a rival to the idea.

Key takeaways

  • →The units are trivial; the capability comes from their number and from the training loop that sets every weight at once.
  • →Depth lets a network invent its own intermediate features, and that is precisely why those features resist auditing afterwards.
  • →A network learns whatever regularity lowers the error, wanted or not, so data quality outranks architecture choice almost every time.

Related concepts

  • Part ofDeep Learning

    Neural networks are the building blocks deep learning is made of.

Courses that teach this

Udacitybeginnernanodegree

AI Programming with Python

Closes the gap most aspiring AI engineers hit first: they can write a little code, but not the programming-and-maths groundwork that machine-learning material takes for granted. It walks from language basics to a working model in PyTorch, with data libraries in between.

About 52 hours

Udacityintermediatenanodegree

Deep Learning

A guided tour of the field's main neural architectures in which you implement each one rather than just read about it: plain networks first, then convolutional models for vision, sequence models and Transformers for language, and finally generative models that synthesise new images.

About 50 hours

Where this concept sits in the field

Certifications that test this

Vendor exams whose syllabus covers this concept: facts, cost and a preparation path on each page.

More courses from these categories

Courses from the categories where this concept is taught. Details, price and the provider link are on each course page.

Analytics for Business Strategy: Core Executive Skills

Udemy

Project Management with ChatGPT (Traditional + Agile)

Udemy

AI for Business Operations and Management

Udemy

ChatGPT For Project Management: Practical AI Crash Course

Udemy

Artificial Intelligence with Crypto Stocks

Udemy

Build Your Leadership Brand with Generative AI

Udemy

FAQ

How is a neural network different from deep learning?
Depth is the only difference. Deep learning means neural networks with many layers, plus the training tricks and hardware that made many layers practical. Every deep model belongs to this family, and so do the shallow ones from earlier decades.
Do I need heavy mathematics to use one?
To build with an existing framework, a working feel for linear algebra and derivatives is plenty, and you can acquire it alongside the code. To design new architectures you need considerably more. Most practitioners never do the second thing, because the supply of proven designs is large and the supply of clean labelled data is not.
Can anyone explain why a trained network gave a given answer?
Only approximately. Tools exist that show which inputs mattered most to one decision, and they are useful, but they reconstruct a plausible account rather than read out a stored reason. If a decision must stand up before a regulator or an angry customer, treat explainability as a design constraint from the start rather than something to retrofit.

Sources

The primary text this definition rests on. Read it before relying on this one.

Last reviewed 26 September 2026 · Getting Digital