What Is Deep Learning?

Deep learning powers image recognition, speech assistants, and language models — but under the hood it's built from a shockingly simple unit: a weighted sum, a bias, and a squashing function. In this lesson you'll place deep learning inside the AI landscape, build a single neuron in NumPy, and train a real (tiny) neural network live in your browser to see why depth and nonlinearity matter.

AI → ML → Deep Learning

The three terms nest inside each other, and each level exists because the one above it hit a wall:

• Artificial intelligence — making machines behave intelligently. The classic approach was hand-written rules, but many problems (is this photo a cat?) are far too complex to program explicitly.
• Machine learning — instead of writing the rules, let the machine find patterns from examples. This works, but classic ML lives or dies on feature engineering: a human must decide which input combinations matter (ratios, interactions, edge detectors...), and the space of possible combinations is huge.
• Deep learning — let the machine learn the feature engineering too. A deep network's hidden layers automatically build useful intermediate features from raw inputs, stacking simple combinations into increasingly abstract ones.

That last point is the whole sales pitch: deep learning trades hand-crafted features for learned ones. You pay for it with more data and more compute.

A neuron: weighted sum + bias + activation

A single artificial neuron does three things: multiply each input by a weight, add a bias, then pass the result through an activation function:

a = f(w₁x₁ + w₂x₂ + ... + wₙxₙ + b)

Look familiar? With the identity function as f, this is exactly linear regression. With a sigmoid as f, it's logistic regression. A neural network is just many of these units wired together — you already know the atoms.

Let's build one in NumPy, right here:

Python — runs in your browser

Run

import numpy as np def sigmoid(z): return 1 / (1 + np.exp(-z)) def neuron(x, w, b, activation): z = np.dot(w, x) + b # weighted sum + bias return activation(z) # squash x = np.array([2.0, -1.0, 0.5]) # three input features w = np.array([0.4, 0.8, -1.2]) # one weight per input b = 0.1 print("pre-activation z :", np.dot(w, x) + b) print("identity (regression-style):", neuron(x, w, b, lambda z: z)) print("sigmoid (probability-style):", round(neuron(x, w, b, sigmoid), 4)) print("ReLU (hidden-layer favourite):", neuron(x, w, b, lambda z: max(0, z)))

The activation function is the neuron's personality. Explore the common ones and their derivatives — the derivative matters because training flows gradients backwards through these functions:

x1.00f(x)1.000f′(x)1.000

Function

sigmoidtanhReLULeaky ReLUGELU

Input x1.00

max(0, x): cheap and non-saturating for positive inputs, the default in modern nets. But for x < 0 the gradient is exactly 0 — units stuck there stop learning (“dead neurons”).

Notice that sigmoid and tanh flatten out at the extremes (their derivative goes to ~0 — a problem called vanishing gradients), while ReLU keeps a constant slope of 1 for positive inputs. That's a big part of why ReLU is the default hidden-layer activation in modern networks.

Why activations must be nonlinear

Here's the key theoretical fact of this lesson. Suppose you stack two linear layers with no activation in between:

h = W₁x + b₁, then y = W₂h + b₂ = W₂(W₁x + b₁) + b₂ = (W₂W₁)x + (W₂b₁ + b₂)

The result is... still a linear function of x. Stacking linear layers collapses into a single linear layer — a hundred of them have exactly the same expressive power as one. Depth buys you nothing without nonlinearity.

A nonlinear activation between layers breaks the collapse: each hidden unit can now carve out its own bent piece of the input space, and the next layer combines those pieces into shapes no straight line could make.

See it live: train a network in your browser

Time to test the theory. The playground below trains a small MLP on 2-D toy data in real time. Try this experiment on the circles dataset:

1. Set hidden units to 0 (no hidden layer — a linear model). Watch it fail: the best a line can do on a ring-inside-a-ring is roughly 50% territory.
2. Now give it a hidden layer with 4–6 units and ReLU. Each unit learns one "tilted line" feature; combined, they bend into a closed boundary around the inner cluster.

epoch0loss0.6930train acc50.0%

Dataset

blobsmoonscircles

Hidden units4

Activation

tanhReLU

▶ TrainReset weights

Try moons too — two or three hidden units are usually enough, because the boundary only needs one gentle bend. The harder the shape, the more units (width) or layers (depth) you need. This is the "power of combination" in action: straight lines → triangles → curves → arbitrary blobs.

Vocabulary you'll use constantly

• Input layer — one node per feature. Not counted as a "real" layer (it does no computation).
• Hidden layers — the layers between input and output. Their units are the learned features.
• Output layer — one node per regression target, or per class in classification.
• Width — how many units in a layer. Depth — how many layers. "Deep" learning literally means "more than one hidden layer".
• Weights and biases — the learnable parameters. A linear regression has a handful; modern networks have millions to billions, which is why they need so much more data.

When deep learning wins — and when it doesn't

Deep learning dominates on unstructured data: images, audio, video, and text. These have raw, high-dimensional inputs where hand-crafting features is hopeless, and learned feature hierarchies shine.

But on small tabular datasets — a few thousand rows of customer records — gradient-boosted trees and even logistic regression routinely beat neural networks, train in seconds, and are easier to interpret. More parameters means more data hunger: a network that can learn anything will happily memorize noise when examples are scarce.

Rule of thumb: reach for deep learning when the data is unstructured or truly huge; reach for classic ML first when it's a modest spreadsheet.

Check your understanding

5 questions · free

1. Q1.What is the main thing deep learning automates that classic machine learning leaves to humans?

   ACollecting the training dataBFeature engineering — building useful input combinationsCChoosing the loss functionDDeploying the model to production

2. Q2.A single neuron with a sigmoid activation is mathematically equivalent to which classic model?

   AK-nearest neighborsBA decision treeCLogistic regressionDK-Means clustering

3. Q3.You stack five Linear layers with no activation functions between them. What model do you effectively have?

   AA five-times-more-powerful nonlinear modelBA single linear modelCA random forestDAn untrainable model

4. Q4.In the playground, a network with 0 hidden units cannot separate the circles dataset. Why?

   AThe learning rate is too small for circlesBWithout a hidden layer the decision boundary is a straight line, and no line separates a ring from its centerCCircles data has too many pointsDIt needs a convolutional layer

5. Q5.For a 3,000-row table of loan applications, which is usually the most sensible first model?

   AA deep neural network with ten hidden layersBA gradient-boosted tree or logistic regressionCA convolutional neural networkDA generative adversarial network

Exercise: A two-layer network by hand

Extend the single-neuron code into a two-layer network in NumPy: 2 inputs → 3 hidden units with ReLU → 1 output with sigmoid. Use np.random.default_rng(42) to create the weight matrices (shapes 3×2 and 1×3) and biases. Then prove the linear-collapse claim: remove the ReLU and show that the stacked computation gives exactly the same output as a single pre-multiplied linear layer.

Next up: the tool we'll use to build real networks — PyTorch, its tensors, and the autograd engine that computes gradients for free.