SVM梯度计算

想法

夯实基础吧…

笔记

多分类器SVM梯度计算

SVM损失函数公式:http://cs231n.github.io/optimization-1/
参考文章:https://blog.csdn.net/yc461515457/article/details/51921607

SVM的梯度计算其实很简单,损失函数Liw求导就可以发现,每一个大于零(其他类的score和正确类的score差值大于delta)的项,对dW的贡献体现在两项上:对于第j列贡献了xi的值,对于第y列贡献了-xi的值:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# 对每个输入(1*3073)和W(3073*10)相乘,得到结果 scores(1*10)
for i in xrange(num_train):
scores = X[i].dot(W)
correct_class_score = scores[y[i]]
# 计算Li,对每个非本类分类结果计算损失并累加
for j in xrange(num_classes):
if j == y[i]:
continue
margin = scores[j] - correct_class_score + 1 # note delta = 1
if margin > 0:
loss += margin
# Li对w求导后在两项上有贡献,第j项和第yi项
# dW[:, j] (3073*1),而X[i]是(1*3073)的所以要转置
dW[:, j] += X[i].T
dW[:, y[i]] += -X[i].T
# Right now the loss is a sum over all training examples, but we want it
# to be an average instead so we divide by num_train.
loss /= num_train
dW /= num_train
# Add regularization to the loss.
loss += 0.5 * reg * np.sum(W * W)
dW += reg * W

其他