Back to Archive

Translating Machine Learning Equations Into Code: What Linear Regression Taught Me About How Models Learn

How I stopped memorizing formulas and started understanding what machine learning algorithms are actually doing.

One of the biggest surprises in my machine learning journey was realizing that understanding an equation does not automatically mean understanding how to implement it.

I could look at a formula, recognize the symbols, and even explain some of the mathematics. But when Andrew Ng's Machine Learning Specialization asked me to implement the Cost Function and Gradient Function for Linear Regression from scratch, I often found myself staring at a blank notebook.

The problem was not mathematics.

The problem was translation.

I was trying to memorize equations instead of understanding them as a sequence of operations.

The breakthrough came when I developed a simple framework for reading machine learning formulas. Once I started using it consistently, concepts like cost functions, gradients, and gradient descent became much easier to understand.

In this article, I'll walk through that framework using two foundational machine learning exercises: implementing the Cost Function and the Gradient Function for Linear Regression.

Why Machine Learning Equations Feel Difficult

When I first saw the linear regression cost function, it looked intimidating:

J(w,b)=12m∑i=0m−1(fw,b(x(i))−y(i))2J(w,b)=\frac{1}{2m}\sum_{i=0}^{m-1}(f_{w,b}(x^{(i)})-y^{(i)})^2

There are superscripts. Subscripts. Summation symbols. Multiple variables. Nested expressions.

My instinct was to understand every symbol before doing anything else.

That approach slowed me down.

Eventually, I realized that machine learning equations are not fundamentally different from code. Both describe a process. The equation is simply a compact way of expressing instructions.

The challenge is learning how to unpack those instructions.

The Mental Model That Changed Everything

Whenever I encounter a new machine learning equation, I now ask four questions:

1. What are the inputs?

What data is available?

2. What happens to one example?

Ignore the dataset for a moment. What happens to a single row?

3. What gets repeated?

If there is a summation symbol, something usually needs to happen repeatedly.

4. What should be returned?

What is the final output?

This framework helped me stop seeing equations as abstract mathematics and start seeing them as algorithms.

Exercise 1: The Cost Function

The goal of the cost function is simple:

Measure how wrong the model is.

The exercise introduces three equations.

Prediction:

fw,b(x(i))=wx(i)+bf_{w,b}(x^{(i)})=wx^{(i)}+b

Cost for one example:

cost(i)=(fw,b(x(i))−y(i))2cost^{(i)}=(f_{w,b}(x^{(i)})-y^{(i)})^2

Total cost:

J(w,b)=12m∑i=0m−1cost(i)J(w,b)=\frac{1}{2m}\sum_{i=0}^{m-1}cost^{(i)}

Initially, this looked like three separate formulas.

In reality, they are three steps in the same process.

Step 1: What Are The Inputs?

The function receives:

x  # input feature
y  # actual value
w  # weight
b  # bias

That's all. Nothing mysterious is happening.

Step 2: What Happens To One Example?

Consider a tiny dataset:

PopulationProfit
12
24
36

Suppose:

w = 1
b = 0

For the first row:

prediction = w * x + b
prediction = 1 * 1 + 0
prediction = 1

But the actual value is y = 2.

The model predicted 1. Reality was 2.

The error is:

1 - 2 = -1

The cost function squares the error:

(-1) ** 2 = 1

This is the first major insight:

The cost function is simply measuring how wrong the prediction was.

Why Do We Square The Error?

This was something I didn't appreciate at first.

Suppose one prediction has an error of -3 and another has an error of 3.

If we simply added errors together:

-3 + 3 = 0

The model would appear perfect despite making mistakes.

Squaring prevents positive and negative errors from cancelling each other out. It also penalizes larger mistakes more heavily:

2 ** 2 = 4    # small mistake
10 ** 2 = 100  # large mistake

A mistake that is five times larger becomes twenty-five times more expensive.

This encourages the model to reduce large errors aggressively.

Step 3: What Gets Repeated?

The summation symbol:

∑\sum

is simply telling us:

Repeat this process for every training example.

In code:

for i in range(m):

Whenever I see a summation in machine learning, I immediately think:

Loop.

That single translation made equations much easier to read.

Step 4: What Should Be Returned?

The exercise asks for the total cost.

That means:

  1. Calculate the cost for each example
  2. Add them together
  3. Average them

The final implementation becomes:

def compute_cost(x, y, w, b):
    m = x.shape[0]
    cost_sum = 0
 
    for i in range(m):
        prediction = w * x[i] + b
        cost = (prediction - y[i]) ** 2
        cost_sum += cost
 
    total_cost = cost_sum / (2 * m)
 
    return total_cost

Exercise 2: The Gradient Function

Now that we can measure how wrong the model is, the next question is:

How should we change the parameters to make the model less wrong?

That is the job of the gradient function.

Step 1: What Are The Inputs?

The gradient function receives the same inputs as the cost function:

x  # input feature
y  # actual value
w  # weight
b  # bias

Step 2: What Happens To One Example?

For each training example, we need to:

  1. Calculate the prediction
  2. Calculate the prediction error
  3. Use this error to compute gradients for both weight and bias

Understanding The Gradient Formulas

The equations looked intimidating:

∂J∂b(i)=(fw,b(x(i))−y(i))\frac{\partial J}{\partial b}^{(i)} = (f_{w,b}(x^{(i)})-y^{(i)}) ∂J∂w(i)=(fw,b(x(i))−y(i))x(i)\frac{\partial J}{\partial w}^{(i)} = (f_{w,b}(x^{(i)})-y^{(i)})x^{(i)}

Let's simplify this. Remember that:

fw,b(x(i))−y(i)=prediction errorf_{w,b}(x^{(i)})-y^{(i)} = \text{prediction error}

So both gradients depend on the same prediction_error variable.

A Mental Model For Gradients

Imagine every training example gets a vote.

If:

prediction_error = -3

that example is saying:

The prediction is too low.

If:

prediction_error = 5

that example is saying:

The prediction is too high.

Every example contributes information about how the model should change. The gradients collect those signals.

For bias:

dj_db += prediction_error

For weight:

dj_dw += prediction_error * x[i]

After every example has voted, we average the result:

dj_db /= m
dj_dw /= m

The Final Implementation

def compute_gradient(x, y, w, b):
    m = x.shape[0]
    dj_dw = 0
    dj_db = 0
 
    for i in range(m):
        prediction = w * x[i] + b
        prediction_error = prediction - y[i]
 
        dj_db += prediction_error
        dj_dw += prediction_error * x[i]
 
    dj_db /= m
    dj_dw /= m
 
    return dj_dw, dj_db

What Are Gradients Actually Telling Us?

The best explanation I've found is this:

A gradient tells us:

How should the parameters change if we want the cost to decrease?

That is all. Nothing more complicated. Not magic. Not mysterious calculus.

Just information about which direction reduces error.

Connecting Everything To Gradient Descent

At this point, something clicked for me.

The cost function and gradient function are not separate concepts. They are parts of a learning loop:

Training Data
      │
      ▼
Make Predictions
      │
      ▼
Measure Error (Cost Function)
      │
      ▼
Compute Gradients (Gradient Function)
      │
      ▼
Update Parameters
      │
      ▼
Repeat

This loop is the foundation of modern machine learning.

Linear regression uses it. Logistic regression uses it. Neural networks use it. Deep learning uses it. Large language models use the same fundamental idea.

The prediction function becomes more sophisticated, but the learning loop remains remarkably similar.

Gradient Descent As A Landscape

One visualization helped me tremendously.

Imagine the cost function as a landscape. Every point on that landscape corresponds to a particular choice of w and b.

Higher points represent larger error. Lower points represent smaller error.

Gradient descent repeatedly asks:

Which direction goes downhill fastest?

The gradient provides the answer. The algorithm takes a small step downhill.

Repeating this process eventually leads to much lower error.

The Bigger Lesson

Looking back, the most valuable thing I learned from these exercises was not linear regression.

It was how to read machine learning equations.

Today, whenever I encounter a new formula, I ask:

  1. What are the inputs?
  2. What happens to one example?
  3. What gets repeated?
  4. What should be returned?

That framework helped me understand cost functions. It helped me understand gradients.

I expect it will continue helping me understand neural networks, backpropagation, transformers, and other machine learning systems.

The equations will become more sophisticated. The process of translating mathematics into code will remain the same.

Conclusion

My biggest mistake when learning machine learning was believing that understanding required memorization.

What actually helped was learning to translate formulas into operations.

Once I stopped seeing equations as symbols and started seeing them as instructions, machine learning became much more approachable.

Linear regression was my first example of this shift.

I suspect it won't be the last.