paddlepaddle 复现论文LeNet-5网络结构（仅仅关注模型）
paddlepaddle 复现论文LeNet-5网络结构（仅仅关注模型）
选用LeNet-5网络结构，论文地址：Gradient-based learning applied to document recognition | IEEE Journals & Magazine | IEEE Xplore
每个阶段用到的Layer
原论文模型复现(方法一)
import paddle.nn as nn
network = nn.Sequential(
nn.Conv2D(in_channels=1, out_channels=6, kernel_size=5, stride=1, padding=0), # C1 卷积层
nn.Tanh(),
nn.AvgPool2D(kernel_size=2, stride=2), # S2 平局池化层
nn.Sigmoid(), # Sigmoid激活函数
nn.Conv2D(in_channels=6, out_channels=16, kernel_size=5, stride=1, padding=0), # C3 卷积层
nn.Tanh(),
nn.AvgPool2D(kernel_size=2, stride=2), # S4 平均池化层
nn.Sigmoid(), # Sigmoid激活函数
nn.Conv2D(in_channels=16, out_channels=120, kernel_size=5, stride=1, padding=0), # C5 卷积层
nn.Tanh(),
nn.Flatten(),
nn.Linear(in_features=120, out_features=84), # F6 全连接层
nn.Tanh(),
nn.Linear(in_features=84, out_features=10) # OUTPUT 全连接层
)
模型可视化
paddle.summary(network, (1, 1, 32, 32))输出模型结果
--------------------------------------------------------------------------
Layer (type) Input Shape Output Shape Param #
===========================================================================
Conv2D-1 [[1, 1, 32, 32]] [1, 6, 28, 28] 156
Tanh-1 [[1, 6, 28, 28]] [1, 6, 28, 28] 0
AvgPool2D-1 [[1, 6, 28, 28]] [1, 6, 14, 14] 0
Sigmoid-1 [[1, 6, 14, 14]] [1, 6, 14, 14] 0
Conv2D-2 [[1, 6, 14, 14]] [1, 16, 10, 10] 2,416
Tanh-2 [[1, 16, 10, 10]] [1, 16, 10, 10] 0
AvgPool2D-2 [[1, 16, 10, 10]] [1, 16, 5, 5] 0
Sigmoid-2 [[1, 16, 5, 5]] [1, 16, 5, 5] 0
Conv2D-3 [[1, 16, 5, 5]] [1, 120, 1, 1] 48,120
Tanh-3 [[1, 120, 1, 1]] [1, 120, 1, 1] 0
Flatten-1 [[1, 120, 1, 1]] [1, 120] 0
Linear-1 [[1, 120]] [1, 84] 10,164
Tanh-4 [[1, 84]] [1, 84] 0
Linear-2 [[1, 84]] [1, 10] 850
===========================================================================
Total params: 61,706
Trainable params: 61,706
Non-trainable params: 0
---------------------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.12
Params size (MB): 0.24
Estimated Total Size (MB): 0.36
---------------------------------------------------------------------------对模型进行简单修改（方法二）
改进模型
import paddle.nn as nn
network_2 = nn.Sequential(
nn.Conv2D(in_channels=1, out_channels=6, kernel_size=3, stride=1, padding=1),
nn.ReLU(),
nn.MaxPool2D(kernel_size=2, stride=2),
nn.Conv2D(in_channels=6, out_channels=16, kernel_size=5, stride=1, padding=0),
nn.ReLU(),
nn.MaxPool2D(kernel_size=2, stride=2),
nn.Flatten(),
nn.Linear(in_features=400, out_features=120), # 400 = 5x5x16，输入形状为32x32， 输入形状为28x28时调整为256
nn.Linear(in_features=120, out_features=84),
nn.Linear(in_features=84, out_features=10)
)
模型可视化
paddle.summary(network_2, (1, 1, 28, 28))输出结果
---------------------------------------------------------------------------
Layer (type) Input Shape Output Shape Param #
===========================================================================
Conv2D-4 [[1, 1, 28, 28]] [1, 6, 28, 28] 60
ReLU-1 [[1, 6, 28, 28]] [1, 6, 28, 28] 0
MaxPool2D-1 [[1, 6, 28, 28]] [1, 6, 14, 14] 0
Conv2D-5 [[1, 6, 14, 14]] [1, 16, 10, 10] 2,416
ReLU-2 [[1, 16, 10, 10]] [1, 16, 10, 10] 0
MaxPool2D-2 [[1, 16, 10, 10]] [1, 16, 5, 5] 0
Flatten-2 [[1, 16, 5, 5]] [1, 400] 0
Linear-3 [[1, 400]] [1, 120] 48,120
Linear-4 [[1, 120]] [1, 84] 10,164
Linear-5 [[1, 84]] [1, 10] 850
===========================================================================
Total params: 61,610
Trainable params: 61,610
Non-trainable params: 0
---------------------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.11
Params size (MB): 0.24
Estimated Total Size (MB): 0.35
---------------------------------------------------------------------------方法三
Sub Class写法
class LeNet(nn.Layer):
"""
继承paddle.nn.Layer定义网络结构
"""
def __init__(self, num_classes=10):
"""
初始化函数
"""
super(LeNet, self).__init__()
self.features = nn.Sequential(
nn.Conv2D(in_channels=1, out_channels=6, kernel_size=3, stride=1, padding=1), # 第一层卷积
nn.ReLU(), # 激活函数
nn.MaxPool2D(kernel_size=2, stride=2), # 最大池化，下采样
nn.Conv2D(in_channels=6, out_channels=16, kernel_size=5, stride=1, padding=0), # 第二层卷积
nn.ReLU(), # 激活函数
nn.MaxPool2D(kernel_size=2, stride=2) # 最大池化，下采样
)
self.fc = nn.Sequential(
nn.Linear(400, 120), # 全连接
nn.Linear(120, 84), # 全连接
nn.Linear(84, num_classes) # 输出层
)
def forward(self, inputs):
"""
前向计算
"""
y = self.features(inputs)
y = paddle.flatten(y, 1)
out = self.fc(y)
return out
network_3 = LeNet()
模型可视化
paddle.summary(network_3, (1, 1, 28, 28))---------------------------------------------------------------------------
Layer (type) Input Shape Output Shape Param #
===========================================================================
Conv2D-6 [[1, 1, 28, 28]] [1, 6, 28, 28] 60
ReLU-3 [[1, 6, 28, 28]] [1, 6, 28, 28] 0
MaxPool2D-3 [[1, 6, 28, 28]] [1, 6, 14, 14] 0
Conv2D-7 [[1, 6, 14, 14]] [1, 16, 10, 10] 2,416
ReLU-4 [[1, 16, 10, 10]] [1, 16, 10, 10] 0
MaxPool2D-4 [[1, 16, 10, 10]] [1, 16, 5, 5] 0
Linear-6 [[1, 400]] [1, 120] 48,120
Linear-7 [[1, 120]] [1, 84] 10,164
Linear-8 [[1, 84]] [1, 10] 850
===========================================================================
Total params: 61,610
Trainable params: 61,610
Non-trainable params: 0
---------------------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.11
Params size (MB): 0.24
Estimated Total Size (MB): 0.35
---------------------------------------------------------------------------PaddlePaddle高层API写法（方法四）
高层API ，最简单，只需一行就可以。
network_4 = paddle.vision.models.LeNet(num_classes=10)
模型可视化
paddle.summary(network_4, (1, 1, 28, 28))输出结果
---------------------------------------------------------------------------
Layer (type) Input Shape Output Shape Param #
===========================================================================
Conv2D-8 [[1, 1, 28, 28]] [1, 6, 28, 28] 60
ReLU-5 [[1, 6, 28, 28]] [1, 6, 28, 28] 0
MaxPool2D-5 [[1, 6, 28, 28]] [1, 6, 14, 14] 0
Conv2D-9 [[1, 6, 14, 14]] [1, 16, 10, 10] 2,416
ReLU-6 [[1, 16, 10, 10]] [1, 16, 10, 10] 0
MaxPool2D-6 [[1, 16, 10, 10]] [1, 16, 5, 5] 0
Linear-9 [[1, 400]] [1, 120] 48,120
Linear-10 [[1, 120]] [1, 84] 10,164
Linear-11 [[1, 84]] [1, 10] 850
===========================================================================
Total params: 61,610
Trainable params: 61,610
Non-trainable params: 0
---------------------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.11
Params size (MB): 0.24
Estimated Total Size (MB): 0.35
---------------------------------------------------------------------------总结一下 方法二、方法三、方法四。 他们是同一种网络结构，不同的表现形式，网络的模型结构是一样的。
最后是重要的 模型训练与调优
选方法二的网络模型结构 进行训练与调优
# 模型封装
model = paddle.Model(network_2)
# 模型配置
model.prepare(paddle.optimizer.Adam(learning_rate=0.001, parameters=model.parameters()), # 优化器
paddle.nn.CrossEntropyLoss(), # 损失函数
paddle.metric.Accuracy()) # 评估指标
# 启动全流程训练
model.fit(train_dataset, # 训练数据集
eval_dataset, # 评估数据集
epochs=5, # 训练轮次
batch_size=64, # 单次计算数据样本量
verbose=1) # 日志展示形式
模型评估
result = model.evaluate(eval_dataset, verbose=1)
print(result)
批量预测
# 进行预测操作
result = model.predict(eval_dataset)
# 定义画图方法
def show_img(img, predict):
plt.figure()
plt.title('predict: {}'.format(predict))
plt.imshow(img.reshape([28, 28]), cmap=plt.cm.binary)
plt.show()
# 抽样展示
indexs = [2, 15, 38, 211]
for idx in indexs:
show_img(eval_dataset[idx][0], np.argmax(result[0][idx]))
参考链接：
(79条消息) PaddlePaddle入门实践——初识卷积神经网络_面包猎人的博客-CSDN博客_paddle 卷积
(117条消息) 深度学习之paddle之卷积神经网络定义，自定义数据集_小码农--向日葵的博客-CSDN博客_paddle 卷积
(119条消息) 模型量化（2）：Paddle 模型的静态量化和动态量化_AI Studio的博客-CSDN博客_paddle模型量化
#Json 数据集加载和训练，适用于缺陷检测
(130条消息) 飞桨深度学习零基础入门（二）——使用飞浆（Paddle）实现卷积神经网络的手写数字（mnist）识别_AiXing-w的博客-CSDN博客_mnist 飞桨