Skip to main content
Editor

Build Your First ML Model: Linear Regression with scikit-learn

Intermediate30 min6 exercises90 XP
0/6 exercises

Imagine you're a real estate agent. A client asks, "How much is my house worth?" You look at the square footage, the number of bedrooms, and the neighborhood - then you give an estimate. You just did regression in your head.

Linear regression is the same idea, but done mathematically. You feed a model examples of houses with known prices, and it learns a formula to predict the price of new houses. It is one of the most widely used techniques in machine learning, and scikit-learn makes it surprisingly easy.

In this tutorial, you'll build a complete regression pipeline: split data, train a model, make predictions, and evaluate how well it performs. By the end, you'll understand the core workflow that every ML project follows.

What Does the ML Workflow Look Like?

Every supervised machine learning project follows the same five steps:

  • Prepare your data - organize features (inputs) and target (output).
  • Split into train and test - hold some data back for honest evaluation.
  • Train the model - call .fit() to let it learn patterns.
  • Make predictions - call .predict() on new data.
  • Evaluate - measure how close the predictions are to reality.
  • Let's walk through each step with a concrete example.

    How Do You Prepare Data for scikit-learn?

    scikit-learn expects your data in a specific shape. Features (the inputs) go in a 2D array where each row is a sample and each column is a feature. The target (the value you want to predict) goes in a 1D array.

    Generating sample data
    import numpy as np
    from sklearn.datasets import make_regression
    # Generate a simple dataset: 100 samples, 1 feature
    X, y = make_regression(n_samples=100, n_features=1,
    noise=10, random_state=42)
    print('Features shape:', X.shape) # (100, 1)
    print('Target shape:', y.shape) # (100,)
    print('First 5 features:', X[:5].flatten().round(2))
    print('First 5 targets:', y[:5].round(2))
    Output
    Features shape: (100, 1)
    Target shape: (100,)
    First 5 features: [ 0.93  0.09 -1.06  0.31 -0.48]
    First 5 targets: [ 50.78 -10.07 -34.92  10.53 -17.74]

    Why Split Data into Train and Test?

    Imagine studying for a test by memorizing every answer on the practice exam. You'd score 100% on that practice exam - but bomb the real one. That's what happens when you evaluate a model on the same data it trained on.

    By splitting your data, you train on one portion and test on another. The test set acts as "unseen" data, giving you an honest measure of how well the model generalizes.

    Splitting data with train_test_split
    from sklearn.model_selection import train_test_split
    from sklearn.datasets import make_regression
    import numpy as np
    X, y = make_regression(n_samples=100, n_features=1,
    noise=10, random_state=42)
    # 80% train, 20% test
    X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
    )
    print('Training samples:', X_train.shape[0])
    print('Test samples:', X_test.shape[0])
    Output
    Training samples: 80
    Test samples: 20

    How Do You Train a Linear Regression Model?

    Training is the step where the model learns. Linear regression finds the best line (or hyperplane) through your data by minimizing the sum of squared errors between predictions and actual values.

    Fit, predict: the core workflow
    from sklearn.linear_model import LinearRegression
    from sklearn.model_selection import train_test_split
    from sklearn.datasets import make_regression
    X, y = make_regression(n_samples=100, n_features=1,
    noise=10, random_state=42)
    X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
    )
    # Create and train
    model = LinearRegression()
    model.fit(X_train, y_train)
    # Make predictions
    predictions = model.predict(X_test)
    print('Predictions (first 5):', predictions[:5].round(2))
    print('Actual (first 5): ', y_test[:5].round(2))
    Output
    Predictions (first 5): [-58.67  65.49  36.05 -17.25 -10.26]
    Actual (first 5):      [-48.95  89.03  44.41  -5.91  -7.62]

    What Do the Coefficients Tell You?

    After training, the model stores two things: coefficients (the slopes) and the intercept (the y-intercept). The coefficient tells you how much the target changes when a feature increases by one unit.

    Reading model coefficients
    from sklearn.linear_model import LinearRegression
    from sklearn.datasets import make_regression
    X, y = make_regression(n_samples=100, n_features=3,
    noise=5, random_state=42)
    model = LinearRegression()
    model.fit(X, y)
    print('Coefficients:', model.coef_.round(2))
    print('Intercept:', round(model.intercept_, 2))
    print()
    print('Interpretation:')
    for i, coef in enumerate(model.coef_):
    print(f' Feature {i}: +1 unit -> target changes by {coef:.2f}')
    Output
    Coefficients: [28.26 74.53 18.27]
    Intercept: 0.62
    
    Interpretation:
      Feature 0: +1 unit -> target changes by 28.26
      Feature 1: +1 unit -> target changes by 74.53
      Feature 2: +1 unit -> target changes by 18.27

    How Do You Measure Regression Performance?

    The R-squared score (R^2) tells you what fraction of the variation in the target your model explains. An R^2 of 1.0 means perfect predictions. An R^2 of 0.0 means the model is no better than guessing the average every time.

    Evaluating with R-squared and RMSE
    from sklearn.linear_model import LinearRegression
    from sklearn.model_selection import train_test_split
    from sklearn.datasets import make_regression
    from sklearn.metrics import r2_score, mean_squared_error
    import numpy as np
    X, y = make_regression(n_samples=200, n_features=1,
    noise=15, random_state=42)
    X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42)
    model = LinearRegression()
    model.fit(X_train, y_train)
    predictions = model.predict(X_test)
    r2 = r2_score(y_test, predictions)
    rmse = np.sqrt(mean_squared_error(y_test, predictions))
    print(f'R-squared: {r2:.4f}')
    print(f'RMSE: {rmse:.2f}')
    Output
    R-squared: 0.9681
    RMSE: 15.69

    When Should You Scale Your Features?

    If one feature is measured in thousands (like salary) and another in single digits (like years of experience), the model might give unfair weight to the larger-scale feature. StandardScaler transforms each feature to have a mean of 0 and a standard deviation of 1.

    Plain linear regression does not technically require scaling because it can adjust its coefficients. But scaling helps with regularized models (Ridge, Lasso) and makes coefficients directly comparable.

    StandardScaler in action
    from sklearn.preprocessing import StandardScaler
    import numpy as np
    # Imagine two features with very different scales
    X = np.array([[50000, 3], [60000, 5], [75000, 8],
    [80000, 10], [90000, 12]])
    scaler = StandardScaler()
    X_scaled = scaler.fit_transform(X)
    print('Before scaling:')
    print(X[:3])
    print('\nAfter scaling:')
    print(X_scaled[:3].round(2))
    print('\nMean of each feature:', X_scaled.mean(axis=0).round(2))
    print('Std of each feature:', X_scaled.std(axis=0).round(2))
    Output
    Before scaling:
    [[50000     3]
     [60000     5]
     [75000     8]]
    
    After scaling:
    [[-1.47 -1.41]
     [-0.77 -0.8 ]
     [ 0.28  0.12]]
    
    Mean of each feature: [0. 0.]
    Std of each feature: [1. 1.]

    What If the Relationship Isn't a Straight Line?

    Not every relationship is linear. If house prices grow faster as square footage increases (a curve, not a line), you need polynomial features. This adds squared and cubed terms so a linear model can capture curved patterns.

    Polynomial features for curved data
    from sklearn.preprocessing import PolynomialFeatures
    from sklearn.linear_model import LinearRegression
    from sklearn.metrics import r2_score
    import numpy as np
    # Curved data: y = x^2 + noise
    np.random.seed(42)
    X = np.random.uniform(-3, 3, 50).reshape(-1, 1)
    y = X.flatten()**2 + np.random.normal(0, 1, 50)
    # Plain linear regression
    linear = LinearRegression().fit(X, y)
    print('Linear R2:', round(r2_score(y, linear.predict(X)), 4))
    # Polynomial regression (degree 2)
    poly = PolynomialFeatures(degree=2)
    X_poly = poly.fit_transform(X)
    poly_model = LinearRegression().fit(X_poly, y)
    print('Polynomial R2:', round(r2_score(y, poly_model.predict(X_poly)), 4))
    Output
    Linear R2: 0.0126
    Polynomial R2: 0.8967

    Practice Exercises

    Train Your First Model
    Write Code

    Create a LinearRegression model, fit it on the given X_train and y_train, then print the R-squared score on X_test, y_test rounded to 4 decimal places. Print only the number.

    The data is already created for you.

    from sklearn.linear_model import LinearRegression
    from sklearn.datasets import make_regression
    from sklearn.model_selection import train_test_split
    X, y = make_regression(n_samples=100, n_features=1,
    noise=10, random_state=42)
    X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42)
    # Create a LinearRegression model, fit it, and print the R2 score
    Read the Coefficients
    Predict Output

    What will this code print? Think about what coef_ and intercept_ represent.

    from sklearn.linear_model import LinearRegression
    import numpy as np
    X = np.array([[1], [2], [3], [4], [5]])
    y = np.array([2, 4, 6, 8, 10]) # y = 2*x
    model = LinearRegression()
    model.fit(X, y)
    print(round(model.coef_[0], 4), round(model.intercept_, 4) + 0.0)
    Fix the Scaling Bug
    Fix the Bug

    This code has a common data leakage bug. The scaler is fit on the full dataset before splitting. Fix it so the scaler is fit only on training data, then used to transform both train and test. Print the R2 score rounded to 4 decimal places.

    from sklearn.linear_model import LinearRegression
    from sklearn.preprocessing import StandardScaler
    from sklearn.model_selection import train_test_split
    from sklearn.datasets import make_regression
    X, y = make_regression(n_samples=100, n_features=3,
    noise=10, random_state=42)
    # BUG: scaler is fit on ALL data (data leakage)
    scaler = StandardScaler()
    X_scaled = scaler.fit_transform(X)
    X_train, X_test, y_train, y_test = train_test_split(
    X_scaled, y, test_size=0.2, random_state=42)
    model = LinearRegression()
    model.fit(X_train, y_train)
    print(round(model.score(X_test, y_test), 4))
    Multi-Feature Regression
    Write Code

    Build a regression model with 5 features. Generate data with make_regression(n_samples=200, n_features=5, noise=15, random_state=42). Split 80/20, train a LinearRegression, and print all 5 coefficients rounded to 2 decimal places as a list. Use model.coef_.round(2).tolist().

    from sklearn.linear_model import LinearRegression
    from sklearn.model_selection import train_test_split
    from sklearn.datasets import make_regression
    # Generate data and build the model
    Polynomial vs Linear
    Write Code

    The data below is curved (quadratic). Fit a plain LinearRegression and a polynomial regression (degree 2). Print two R2 scores (on training data) each rounded to 4 decimal places, on separate lines. Label them "Linear R2: ..." and "Poly R2: ...".

    from sklearn.linear_model import LinearRegression
    from sklearn.preprocessing import PolynomialFeatures
    from sklearn.metrics import r2_score
    import numpy as np
    np.random.seed(42)
    X = np.random.uniform(-3, 3, 80).reshape(-1, 1)
    y = 2 * X.flatten()**2 + np.random.normal(0, 2, 80)
    # Fit linear model and print R2
    # Fit polynomial model (degree 2) and print R2
    Complete Regression Pipeline
    Write Code

    Build a complete pipeline: generate data with make_regression(n_samples=150, n_features=4, noise=20, random_state=0), split 75/25 with random_state=0, scale with StandardScaler (fit on train only), train LinearRegression, and print the RMSE on the test set rounded to 2 decimal places. Use np.sqrt(mean_squared_error(...)).

    from sklearn.linear_model import LinearRegression
    from sklearn.preprocessing import StandardScaler
    from sklearn.model_selection import train_test_split
    from sklearn.datasets import make_regression
    from sklearn.metrics import mean_squared_error
    import numpy as np
    # Build the full pipeline

    Harder problems

    More problems on this topic: Working with data

    Related Tutorials

    Was this tutorial helpful?