Build a house price predictor
Predict a flat's price from its size, age and location using linear regression, then watch a random forest fail in an instructive way.
- 20 min read
- 3 reading levels
- Updated
Read these first
On this page 9
One lesson, three depths. Pick the one that fits you today — you can switch any time.
Beginner — No maths. Plain English.
The short answer
A house price predictor takes a few facts about a flat and returns a fair price estimate.
The analogy you have already lived
You have bought tomatoes. After fifty visits to the same market, somebody quotes you a price and you know instantly whether it is fair. Nobody gave you a price list. Your sense of "about right" was built from many past purchases.
Now think about a friend describing a flat: two bedrooms, nine hundred square feet, ten years old, near the station. You already have a rough feeling for what that should cost.
This project builds that feeling into a program.
Why this project exists
Property is the largest purchase most families ever make, and the price is famously hard to pin down. Two flats in the same building can differ by twenty percent for reasons nobody writes down.
Before software, this was a job for a valuer who had walked the area for twenty years. That works, but it is slow, expensive, and different valuers disagree.
The idea here is different. Collect flats that already sold, along with their facts and their final prices. Let a program find the pattern connecting facts to price.
This is called regression — predicting a number, rather than sorting things into groups. Your spam detector picked between two labels. This one picks a rupee amount from an unlimited range.
What you are actually going to build
You will start with forty flats. For each one you know four facts: floor area, number of bedrooms, age in years, and distance from the city centre. You also know what it sold for.
The program finds how much each fact is worth. It might learn that every extra square foot adds a certain amount, and every kilometre away from the centre takes some away.
Then you hand it a flat it has never seen, and it adds up the parts.
How it works
area: 1200 sq ft -> worth some amount per square foot
bedrooms: 2 -> worth something extra each
age: 5 years -> takes a little off each year
distance: 8 km -> takes a little off each kilometre
|
v
[ add all the parts together ]
|
v
about 75 lakhThe honest beauty of this method is that you can read the answer back. The program will tell you, in plain numbers, what it thinks a bedroom is worth. That is rare. Most AI cannot explain itself at all.
Where you have already seen this
- Housing sites showing an estimated price next to a listing.
- Ola and Uber fare estimates before you book — same idea, different facts.
- Insurance quotes that change when you enter your car's age.
- Resale value estimates on used-phone sites.
The honest part
A price predictor is confidently wrong outside the range it has seen.
Our forty flats top out around two and a half thousand square feet. Ask the model about a six thousand square foot penthouse and it will answer with total confidence, having never seen anything remotely similar. You will run exactly that experiment later and watch two different models give two wildly different wrong answers.
There is a second, heavier problem. A model trained on past sale prices learns the past, including its unfairness. If flats in one neighbourhood historically sold cheap because of who lived there, the model learns to price that neighbourhood cheap and calls it maths.
This is not hypothetical. The most famous housing dataset in machine learning was removed from scikit-learn in 2023 for containing a feature built on exactly that assumption. Prediction repeats history unless somebody actively stops it.
Remember this
- Regression predicts a number, not a category.
- The model works out what each fact is worth, then adds the parts up.
- It is trustworthy only inside the range of flats it was trained on — and it repeats whatever unfairness is in the past data.
What to learn next
- Linear regression — the algorithm underneath this project, on its own.
- Random forest — why many trees beat one, and when they still lose.
- Feature engineering — the work that improves a model more than swapping algorithms.
Developer — Code and libraries.
The problem, stated precisely
Given four numeric features of a flat, predict its sale price in lakh rupees. This is supervised regression: a continuous target, and a labelled training set.
We will fit two models on identical data. Linear regression, which assumes the price is a weighted sum of the features. And a random forest, which assumes nothing of the sort. Comparing them teaches more than either one alone.
Setup
pip install scikit-learn pandasNo downloads, no GPU. The whole script finishes in about two seconds.
The dataset
Forty flats, written directly into the file as a CSV string. Prices are in lakh rupees — one lakh is 100,000.
This data is synthetic, and this page says so plainly rather than pretending otherwise. It was generated from a known formula plus random noise, which gives us something valuable: at the end we can check whether the model recovered the recipe that produced it. You cannot do that with real data, because nobody knows the true recipe.
For real data afterwards, the Ames Housing dataset (2,930 real sales, 80 features, about 1 MB) is the standard modern replacement for the retired Boston dataset. Link at the end of this block.
The full build
import io
import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_absolute_error, r2_score
# 40 flats. Price is in lakh rupees (1 lakh = 100,000).
CSV = """area_sqft,bedrooms,age_years,km_to_centre,price_lakh
2200,4,9,4.8,139.5
2410,4,14,16.6,131.7
700,1,6,9.9,40.0
1120,2,4,10.0,61.9
2560,4,14,14.3,142.8
1960,4,9,12.0,118.5
1480,3,23,9.7,77.3
1370,3,3,1.0,96.1
2300,4,3,2.9,151.0
1870,4,8,15.9,103.0
680,1,19,12.0,23.8
1170,2,16,15.6,58.3
1340,2,1,21.1,74.4
1550,3,0,15.3,95.0
870,1,16,2.1,54.2
1810,4,22,7.5,104.0
2120,4,12,13.4,119.0
1130,2,7,5.9,72.3
2210,4,23,21.3,100.3
1720,3,10,20.8,84.8
1800,4,9,18.8,105.6
2170,4,3,10.9,136.9
450,1,0,18.7,18.7
1200,2,7,3.8,77.5
1000,2,23,7.5,45.3
510,1,23,10.7,15.4
640,1,0,16.6,35.0
2330,4,21,11.2,119.8
890,1,4,3.9,63.3
2360,4,3,8.2,145.3
1140,2,1,7.8,81.6
1010,2,3,7.3,68.3
1970,4,1,4.5,140.0
2280,4,19,9.7,124.9
2260,4,4,10.4,137.0
1570,3,19,17.3,74.7
2340,4,1,17.7,139.2
2370,4,25,12.0,125.3
790,1,5,10.7,40.5
1010,2,9,17.3,49.0
"""
df = pd.read_csv(io.StringIO(CSV))
print(df.head(3))
print("\nrows, columns:", df.shape)
print("\nhow each column moves with price:")
print(df.corr(numeric_only=True)["price_lakh"].round(3))
X = df[["area_sqft", "bedrooms", "age_years", "km_to_centre"]]
y = df["price_lakh"]
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, random_state=0
)
lin = LinearRegression()
lin.fit(X_train, y_train)
pred = lin.predict(X_test)
print("\n--- linear regression ---")
print("mean absolute error: %.2f lakh" % mean_absolute_error(y_test, pred))
print("R2 on the held-out flats: %.3f" % r2_score(y_test, pred))
print("\nwhat one unit of each feature is worth, in lakh:")
for name, coef in zip(X.columns, lin.coef_):
print(f" {name:14} {coef:+.4f}")
print(f" {'(intercept)':14} {lin.intercept_:+.2f}")area_sqft bedrooms age_years km_to_centre price_lakh 0 2200 4 9 4.8 139.5 1 2410 4 14 16.6 131.7 2 700 1 6 9.9 40.0 rows, columns: (40, 5) how each column moves with price: area_sqft 0.958 bedrooms 0.928 age_years -0.075 km_to_centre -0.096 price_lakh 1.000 Name: price_lakh, dtype: float64 --- linear regression --- mean absolute error: 3.58 lakh R2 on the held-out flats: 0.992 what one unit of each feature is worth, in lakh: area_sqft +0.0567 bedrooms +3.2568 age_years -1.0268 km_to_centre -1.2816 (intercept) +16.32
Read the coefficients — the model recovered the recipe
This data was generated with a formula I have not shown you yet. Here it is: area was worth 0.055 per square foot, a bedroom 3.5, a year of age minus 0.9, a kilometre minus 1.4, on a base of 18. Then random noise was added to every price.
Compare that with what the model learned from thirty training rows: 0.0567, 3.2568, -1.0268, -1.2816, base 16.32.
It found the recipe without being told it existed. That is what fitting a model means.
Watching it succeed on data whose truth you already know is the fastest way to trust it on data whose truth you do not.
The correlation column is a trap, and it is important
Look again at the correlation output. age_years scores -0.075 and km_to_centre scores -0.096. Both are close to zero. A reasonable person would drop both features as useless.
That reasonable person would be wrong. The fitted model gives age a coefficient of -1.03 and distance -1.28. Both matter a great deal.
Why the disagreement? Correlation looks at one column against price, alone. Area varies so enormously across our flats that it drowns everything else out. A brand-new flat far from the centre still costs more than an old flat nearby, if it is twice the size.
Regression looks at all four columns together. It asks what age is worth among flats of the same size. That question has a clear answer, and the single-column correlation cannot ask it.
This is confusing the first time. Read it twice — that is normal. The short version: a feature with near-zero correlation can still be strongly predictive once other features are held steady.
Now add a second model
forest = RandomForestRegressor(n_estimators=200, random_state=0)
forest.fit(X_train, y_train)
fpred = forest.predict(X_test)
print("\n--- random forest ---")
print("mean absolute error: %.2f lakh" % mean_absolute_error(y_test, fpred))
print("R2 on the held-out flats: %.3f" % r2_score(y_test, fpred))
print("\n5-fold R2, linear:", cross_val_score(lin, X, y, cv=5).round(3))
print("5-fold R2, forest:", cross_val_score(forest, X, y, cv=5).round(3))
new_flats = pd.DataFrame([
{"area_sqft": 1200, "bedrooms": 2, "age_years": 5, "km_to_centre": 8.0},
{"area_sqft": 1200, "bedrooms": 2, "age_years": 5, "km_to_centre": 18.0},
{"area_sqft": 6000, "bedrooms": 5, "age_years": 1, "km_to_centre": 3.0},
])
print("\nprice estimates (lakh):")
print(" linear:", lin.predict(new_flats).round(1))
print(" forest:", forest.predict(new_flats).round(1))--- random forest --- mean absolute error: 15.35 lakh R2 on the held-out flats: 0.864 5-fold R2, linear: [0.986 0.986 0.989 0.99 0.993] 5-fold R2, forest: [0.863 0.896 0.868 0.925 0.932] price estimates (lakh): linear: [ 75.4 62.6 367.7] forest: [ 77.5 76.4 146.3]
The forest lost, and the reason is the point
A random forest is the more powerful model. It handles curves, interactions and thresholds that a straight line cannot. Here it was beaten by a wide margin: 15.35 lakh of error against 3.58.
Two reasons, and both generalise far beyond this example.
The truth really is a weighted sum. We built the data that way. Linear regression assumes exactly that shape, so the assumption is free accuracy. A forest has to discover the shape from data, and discovery costs examples.
Thirty training rows is nothing for a tree. Trees split data into boxes and average within each box. With thirty rows the boxes hold two or three flats each, so every prediction is a coarse average.
The lesson is not "linear regression is better". It is that model choice is a claim about the shape of reality, and a correct claim beats a flexible model on small data. Reach for a forest or XGBoost when you have thousands of rows and suspect the relationship bends.
The three test flats reveal the real difference
Flats one and two differ only in distance: 8 km against 18 km. Linear drops the price by 12.8 lakh, matching its learned rate. The forest drops it by 1.1 lakh — almost nothing. With so little data the forest never split meaningfully on distance, so it is nearly blind to that column.
Flat three is a 6,000 square foot penthouse. The largest flat in the training rows is 2,410 square feet. Both models answer with complete confidence and neither has any right to.
Linear says 367.7 lakh, extending its straight line into territory it has never observed. Forest says 146.3 lakh, which sits a little below the 151.0 lakh top price in its training rows.
That gap is not noise. A tree predicts by averaging training values, so a forest can never output a number above the highest price in its training set. Ever. Linear regression extrapolates without limit; trees refuse to extrapolate at all. Neither behaviour is correct — they are two different ways of being wrong outside the data.
Guard against it in code:
lo, hi = X_train["area_sqft"].min(), X_train["area_sqft"].max()
for area in new_flats["area_sqft"]:
if not (lo <= area <= hi):
print(f"warning: {area} sqft is outside the trained range {lo}-{hi}")warning: 6000 sqft is outside the trained range 640-2410
Only the penthouse trips the guard. The two 1,200 square foot flats sit comfortably inside the range the model actually learned from.
Common mistakes
Reading R-squared as a percentage grade. R-squared of 0.99 means the model explains 99% of the variance in this sample. On ten test rows that number is extremely unstable. Report mean absolute error too, because it is in rupees and a human can judge it.
Leaving an ID column in the features. A row number or listing ID correlates with nothing real but can still fit noise. Select feature columns explicitly, as the code above does, rather than dropping the target and keeping the rest.
Feeding text columns straight in. A city column of strings raises ValueError: could not convert string to float. Wrap it with OneHotEncoder inside a ColumnTransformer.
Forgetting that money is skewed. Real prices have a long right tail, so a few mansions dominate the squared error. Fitting np.log(price) and exponentiating the prediction back is the standard fix, and it usually improves everything at once.
Splitting sales randomly when they span years. Prices drift over time. A random split lets the model see 2024 sales while being tested on 2023, which flatters it. Split by date for anything time-dependent.
How to make this genuinely good
- Move to real data. The Ames Housing dataset loads with
fetch_openml(name="house_prices", as_frame=True)— 2,930 real sales, about 1 MB. - Add location properly. Postcode as a one-hot feature, or latitude and longitude, usually beats every other improvement on the list.
- Predict
log(price)and report errors in the original units after exponentiating. - Use
HistGradientBoostingRegressoronce you have thousands of rows. It handles missing values natively and is fast on CPU. - Give a range, not a point.
GradientBoostingRegressor(loss="quantile", alpha=0.1)andalpha=0.9bracket a plausible interval. "Between 70 and 82 lakh" is honest; "75.4 lakh" is not. - Check fairness before shipping. Group your errors by neighbourhood and look for a group the model reliably under-prices.
Try it yourself
Delete area_sqft from the feature list and re-run. Watch R-squared collapse, then watch bedrooms absorb the missing signal by growing a much larger coefficient.
That is the model compensating with a correlated feature. It explains why coefficients shift whenever you add or remove a column.
It also explains why "the coefficient of bedrooms is the value of a bedroom" is a claim you have to earn, not assume.
What to learn next
- Linear regression — the algorithm underneath this project, on its own.
- Random forest — why many trees beat one, and when they still lose.
- Feature engineering — the work that improves a model more than swapping algorithms.
Researcher — Mathematics and papers.
Formulas here are written in plain text, since the site renders no maths typesetting library.
Ordinary least squares
Given a design matrix X of shape (n, p) with a column of ones prepended for the intercept, and a target vector y of length n, OLS minimises the residual sum of squares:
beta_hat = argmin || y - X beta ||^2
betawith the closed-form normal-equation solution:
beta_hat = ( X^T X )^-1 X^T ybeta_hatis the estimated coefficient vector, lengthp + 1.X^Tis the transpose of the design matrix.( X^T X )^-1requiresXto have full column rank; exact collinearity makes it singular.
Nobody inverts that matrix in practice. sklearn.linear_model.LinearRegression calls scipy.linalg.lstsq, which uses SVD-based least squares. This costs more than a Cholesky solve but stays stable when X^T X is ill-conditioned, and it returns the minimum-norm solution when the system is rank-deficient rather than failing outright.
Complexity. Forming X^T X is O(n p^2) and factorising it is O(p^3); SVD of X is O(n p^2) for n > p. Prediction is O(p). For p in the thousands and n in the millions, switch to SGDRegressor and accept an iterative solution.
Gauss-Markov, and which assumptions actually bite
Under the assumptions of linearity in parameters, exogeneity (E[eps | X] = 0), homoskedasticity (Var(eps | X) = sigma^2 I) and no perfect collinearity, OLS is the best linear unbiased estimator. Normality of errors is not required for that result; it is required only for exact finite-sample t and F inference.
In housing data three of these fail routinely:
Heteroskedasticity. Error variance grows with price. Point predictions stay unbiased but standard errors are wrong, so use heteroskedasticity-consistent (HC3) errors, or model log(price) where the variance is closer to constant.
Spatial autocorrelation. Nearby properties have correlated residuals, violating independence. This is the single largest source of over-optimistic validation scores in property modelling, because a random k-fold split places neighbours on both sides of the split. Use spatially blocked cross-validation.
Omitted variable bias. School catchment, floor level, view, and legal title status are rarely in the data and are correlated with the features that are. Every coefficient absorbs part of what is missing, which is why coefficients should be read as associations rather than causal effects. Rosen (1974) sets out the hedonic-pricing framework under which such coefficients can be interpreted as implicit marginal prices, and the conditions are strong.
Regularisation
When p grows or features are collinear, constrain the coefficients.
Ridge: argmin || y - X b ||^2 + lambda * || b ||_2^2
Lasso: argmin || y - X b ||^2 + lambda * || b ||_1Ridge (Hoerl and Kennard, 1970) has closed form ( X^T X + lambda I )^-1 X^T y, always invertible for lambda > 0. It shrinks coefficients toward zero without reaching it, and handles collinear groups by splitting weight among them.
Lasso (Tibshirani, 1996) has no closed form and is solved by coordinate descent. The L1 geometry has corners on the axes, so solutions are sparse — it performs selection. With correlated features it picks one arbitrarily, which elastic net (Zou and Hastie, 2005) fixes by combining both penalties.
Standardise features before penalising. Otherwise the penalty is applied in whatever units each column happens to use, and area_sqft measured in square metres is regularised differently from the same feature in square feet.
Why the forest lost, formally
A random forest (Breiman, 2001) predicts by averaging over B trees, each grown on a bootstrap sample with a random feature subset at each split. Its estimator is piecewise constant on an axis-aligned partition of feature space.
Two consequences are visible in the output above.
No extrapolation. Every leaf value is a mean of training targets. The predictor's range is therefore bounded by [min(y_train), max(y_train)]. Linear models are unbounded. Gradient-boosted trees inherit the same bound on the base level plus the boosting offsets, which loosens but does not remove it.
Approximation error on smooth functions. Fitting a linear surface with axis-aligned constants incurs an approximation error that decreases only as the partition refines, and refinement requires samples. With n = 30 and p = 4, the effective leaves are tiny and the bias-variance trade-off is unfavourable. The asymptotic consistency results for forests (Scornet, Biau and Vert, 2015) say nothing useful at this sample size.
The general statement: correct inductive bias substitutes for data. When the hypothesis class contains the true function, restricting the class costs nothing and buys sample efficiency.
Prediction intervals
A point estimate is the wrong deliverable for a valuation. Three options, in increasing cost:
- Quantile regression (Koenker and Bassett, 1978): minimise the pinball loss at
tau = 0.1and0.9. Available asGradientBoostingRegressor(loss="quantile", alpha=tau). - Conformal prediction (Vovk et al.; Lei et al., 2018 for split conformal): wrap any model, calibrate residual quantiles on a held-out set, and obtain finite-sample marginal coverage guarantees under exchangeability alone. This is the strongest guarantee available with the least modelling commitment.
- Bayesian regression with a full posterior predictive, at the cost of specifying priors and running inference.
Note that exchangeability fails under temporal drift, so conformal coverage degrades in a moving market. Adaptive conformal methods (Gibbs and Candès, 2021) address exactly this.
The dataset that was withdrawn
sklearn.datasets.load_boston, drawn from Harrison and Rubinfeld (1978), was deprecated in scikit-learn 1.0 and removed in 1.2. The dataset contains a feature B, defined as a transformation of the proportion of Black residents by town, constructed on the explicit assumption that racial composition affects house prices. Carlisle (2019) documented that the transformation is non-invertible and its stated justification unverifiable.
This matters beyond one dataset. Hedonic models trained on historical transactions encode historical discrimination, and a model that reproduces redlining-era price gradients will be evaluated as accurate, because it is accurate about the past. Fairness auditing for property models means examining residuals grouped by protected attributes and asking whether systematic under-valuation persists after controlling for physical characteristics. Removing the offending column is not enough on its own, since location proxies it.
Papers
- Rosen, Hedonic Prices and Implicit Markets, Journal of Political Economy, 1974.
- Harrison and Rubinfeld, Hedonic Housing Prices and the Demand for Clean Air, JEEM, 1978.
- Koenker and Bassett, Regression Quantiles, Econometrica, 1978.
- Hoerl and Kennard, Ridge Regression, Technometrics, 1970.
- Tibshirani, Regression Shrinkage and Selection via the Lasso, JRSS-B, 1996.
- Zou and Hastie, Regularization and Variable Selection via the Elastic Net, JRSS-B, 2005.
- Breiman, Random Forests, Machine Learning 45, 2001.
- De Cock, Ames, Iowa: Alternative to the Boston Housing Data, Journal of Statistics Education, 2011.
- Lei, G'Sell, Rinaldo, Tibshirani and Wasserman, Distribution-Free Predictive Inference for Regression, JASA, 2018.
What to learn next
- Linear regression — the algorithm underneath this project, on its own.
- Random forest — why many trees beat one, and when they still lose.
- Feature engineering — the work that improves a model more than swapping algorithms.