cs231n神经网络作业笔记

Q1: Fully-connected Neural Network (25 points)

参考资料:

affine/linear(仿射/线性)变换函数详解及全连接层反向传播的梯度求导

矩阵梯度计算:

1
2
3
4
5
x_ = x.reshape(x.shape[0], -1)
dx = dout @ w.T
dx = dx.reshape(x.shape)
dw = x_.T @ dout
db = np.sum(dout, axis=0)

ReLU的正反向传播:

1
2
3
4
# forward
out = np.maximum(0, x)
# backward
dx = (x > 0) * dout

推荐一本好书《深度学习入门:基于Python的理论与实现》!