Least Square Method

Least square method refers to a statistical technique used to understand relationships between two variables, make predictions, and summarize data. The technique is implemented by finding the best-fitting line through a set of data points. Here, the best fit line is the line drawn across the scatter plot to show the relationship between the variables. 

Example:
Imagine a teacher wants to predict a student's quiz score based on how many hours they studied. She collects data from 4 students.
If you plot these points on a graph, they form a rough upward trend, but they don't create a perfect straight line.

• The Goal: Find the equation of the line (y = mx + c) that passes through the middle of these points most accurately.

Calculating the "Least Squares":
The method seeks a line that minimizes the sum of the squared vertical distances (errors) between the points and the line.
Let's say a line. For every data point, the method calculates the Residual (Error):
\(\text{Error} = \text{Actual Score} - \text{Predicted Score on Line} \)

Then, it squares that error (to get rid of negative numbers and punish big errors more):
\(\text{Squared Error} = (\text{Error})^2 \)

The best-fit line is the one that minimizes the Total Sum of Squared Errors.
The Result (The Best Fit)
Using the Least Squares formula on our data:

• Slope (m): 1.4
• Intercept (c): 0.5

The Equation:
\(y = 1.4x + 0.5\)
Using this Model,
Now the teacher can make a prediction.

• Question: If a student studies for 5 hours, what score will they get?
• Calculation: y = \(1.4(5) + 0.5\)
• Prediction: They will score \(7.5.\)

The least square method formula to find the slope and the intercept is given below: 

Slope (m) = \(n\sum xy \space - \space (\sum x)(\sum y) \over n\sum x^2 \space - \space (\sum x)^2\)

Intercept (c) = ȳ - mx̄ where \(x̄ = {\sum x \over n}\) and \(ȳ = {\sum y \over n}\)

\(c = {\sum y \space- \space m (\sum x) \over n}\)

Here,

• n is the total number of data points
   
• x is the independent variable and y is the dependent variable
   
• Σ is the sum of the values
   
• m is the slope 
   
• c is the y-intercept 
  
   

We follow a certain method to calculate the least squares. Here, we shall analyze the method step-by-step:

Step 1: Identify \( x_i \) and  \(y_i\)
We list our data points clearly.

• \(x = [1, 2, 3, 4]\)
• \(y = [2, 3, 5, 6]\)
• Number of data points \((n) = 4\)

Step 2: Find the Average Values \((\bar{x})\) and \((\bar{y})\)
Calculate the mean for both variables.
\(\bar{x} = \frac{1 + 2 + 3 + 4}{4} = \frac{10}{4} = 2.5 \)
\(\bar{y} = \frac{2 + 3 + 5 + 6}{4} = \frac{16}{4} = 4\)

Step 3: Define the Equation
We are looking for the line: \( y = mx + c\)

Step 4: Calculate the Slope (m)
The formula is:
\(m = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}\)
Let's create a table to calculate the numerator and denominator easily:
Now, divide the Sum of Numerator by the Sum of Denominator:
\(m = \frac{7.0}{5.0} = 1.4\)

Step 5: Calculate the Intercept (c)
Now substitute the values of \(\bar{y}, m, \bar{x}\) into the formula:
\(c = \bar{y} - m\bar{x}\)
\(c = 4.0 - (1.4 \times 2.5)\)
\(c = 4.0 - 3.5\)
\(c = \mathbf{0.5}\)

c = 4.0 - 3.5
\(c = \mathbf{0.5}\)
 

Final Result
The Best Fit Line equation is:
\(\mathbf{y = 1.4x + 0.5}\)

The least square method works by minimizing the differences between the actual data and the predicted value on the line.  Now let’s see how the least square method graph looks like: 
  

The data points are marked in red points. The x-axis represents the independent variable, and the y-axis represents the dependent variable. The line of best fit should minimize the vertical distances (residuals) from all data points to the line. This shows the method can be used to obtain the equation of the best fit line.

The least square method is considered as the best way to find the line of best fit, but also it has some disadvantages. Here are some of the pros and cons of the least square method. 
 

For calculating the least square  method, the students need to find out the variables, use the proper slope formula and also sort out data. It is one of the most accurate method to collect data with minimal errors. Few tips and tricks to master least squares methods are discussed below -

1. Start with the “Ruler Guess”: Before looking at any formulas, print out the scatter plot and ask the student to place a clear ruler where they think the line should go. This builds their intuition for “balancing” the points equally on both sides before they get lost in the math.

2. Explain Squaring as a “Fairness Rule”: Students often ask why we square the numbers. Explain it simply: “Squaring makes big mistakes cost way more than small mistakes.” It forces the line to pay attention to the lonely points (outliers) far away, so no data point is ignored.

3. Master the Table Method: The formula looks scary, but it's just a list of small tasks, distinct insist on using a 6-column table (Columns for: x, y, deviations, products, squares). If they organize the data into a table first, the actual calculation becomes almost impossible to mess up.

4. Personalize the Data: Textbook examples can be dry. Create a custom problem using things they actually love:

• Video Games: Hours Played vs. Level Reached
• Sports: Height vs. Jump Height
• Screen Time: Battery % vs. Time of Day

5. The “Slope Check” (Sanity Check): Teach them to look at the graph before calculating.

• If the dots go UP (like climbing a hill), the Slope ($m$) must be positive.
• If the dots go DOWN (like sliding down), the Slope must be negative.

This simple check catches 50% of calculation errors immediately.

6. Focus on the “Why” (Prediction): Don't let them forget the goal. Remind them that we do this to predict the future. “We have this line, we can guess exactly how much you'll level up in the game if you play for two more hours!”

The least square method is used in various fields. It is mostly used to predict stock prices and analyze scientific data. Here, we'll be looking at some real-life applications of the least square method:

• To predict the future trends in stock market, investors use the least square method by analyzing the historical stock price.

• Least square method is used for weather forecasting by analyzing the past data.

• In medical research, the least square method is used to determine life expectancy based on the lifestyle. For example, the effects of alcoholism on life expectancy or predicting a patient's blood pressure based on age.