多元线性回归算法python实现_机器学习算法python实现-线性回归
线性回归-Linear Regreesion
给定数据集
线性回归试图学得
，使
与
之间的差别尽可能小。
如何确定
和
，关键在于如何衡量
与
之间的差别,可以通过均方误差最小化
基于均方误差最小化来进行模型求解的方法称为“最小二乘法”
这里
是关于w和b的凸函数，当它关于w和b的导数均为零时，得到w和b的最优解。
求w和b的偏导：
令偏导为0：
推广到矩阵形式w的梯度为：
线性回归、批梯度下降、随机梯度下降优化具体算法可参考：kamidox.comblog.kamidox.com
1.一元线性回归
import numpy as np
import matplotlib.pyplot as plt
x=np.array([1,2,3,4,5],dtype=np.float)
y=np.array([1,3.0,2,3,5],dtype=np.float)
plt.scatter(x,y)
x_mean=np.mean(x)
y_mean=np.mean(y)
num=0.0
d=0.0
for x_i,y_i in zip(x,y):
num+=(x_i-x_mean)*(y_i-y_mean)
d+=(x_i-x_mean)**2
a=num/d
b=y_mean-a*x_mean
y_hat=a*x+b
plt.figure(2)
plt.scatter(x,y) #描绘散点
plt.plot(x,y_hat,c='r') #描绘直线
2.多元线性回归批梯度下降
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
diabetes = load_diabetes()
data = diabetes.data
target = diabetes.target.reshape((-1,1))
X_train, X_test, y_train, y_test = train_test_split( data, target, test_size=0.3, random_state=2020)
print('X_train=', X_train.shape)
print('X_test=', X_test.shape)
print('y_train=', y_train.shape)
print('y_test=', y_test.shape)
out:
X_train= (309, 10)
X_test= (133, 10)
y_train= (309, 1)
y_test= (133, 1)
模型主体：
def linar_train(X, y, learning_rate=0.01, epochs=30000):
w, b = initialize(X.shape[1]) #初始化参数
for i in range(1, epochs):
y_hat, loss, dw, db = linar_loss(X, y, w, b)
w += -learning_rate * dw
b += -learning_rate * db
if i % 10000 == 0:
print('epoch %d loss %f' % (i, loss))
params = {
'w': w,
'b': b
}
grads = {
'dw': dw,
'db': db
}
return loss, params, grads
def initialize(dims):
"""将线性回归中w全部初始化为0w为dims行 1列的0矩阵"""
w = np.zeros((dims, 1))
b = 0
return w, b
def linar_loss(X, y, w, b):
"""计算损失函数num_train为数据集的数目num_feature为数据特征数y_hat为预测的y值loss 计算均方误差"""
num_train = X.shape[0]
num_feature = X.shape[1]
y_hat = np.dot(X, w) + b
loss = np.sum((y_hat-y)**2)/num_train #计算均方误差
dw = np.dot(X.T, (y_hat-y)) /num_train #X.T矩阵转置
db = np.sum((y_hat-y)) /num_train
return y_hat, loss, dw, db
训练数据：
loss, params, grads = linar_train(X_train, y_train, 0.001, 500000)
查看参数：
params
{'w': array([[ 18.12432661],
[-107.67245554],
[ 404.37142851],
[ 226.28953116],
[ -10.89733854],
[ -76.80225482],
[-174.88980099],
[ 126.33236945],
[ 353.19542078],
[ 170.24624517]]),
'b': 151.95335186539964}
可视化：
import matplotlib.pyplot as plt
f = X_test.dot(params['w']) + params['b']
plt.scatter(range(X_test.shape[0]), y_test)
plt.plot(f, color = 'darkorange')
plt.xlabel('X')
plt.ylabel('y')
plt.show()
参考：