GCN（二）GCN模型介绍

红太狼 2022-09-09 13:54 293阅读 0赞

上一节介绍了处理`cora`数据集，以及返回的结果：

*  features：论文的属性特征，维度 2708 × 1433 2708 \\times 1433 2708×1433，并且做了归一化，即每一篇论文属性值的和为1.
 *  labels：每一篇论文对应的分类编号：0-6
 *  adj：邻接矩阵，维度 2708 × 2708 2708 \\times 2708 2708×2708
 *  idx\_train：0-139
 *  idx\_val：200-499
 *  idx\_test：500-1499

这一节介绍GCN的模型。

# GCN 模型 #

model:

import torch.nn as nn
    import torch.nn.functional as F
    from pygcn.layers import GraphConvolution
    
    
    class GCN(nn.Module):
        def __init__(self, nfeat, nhid, nclass, dropout):
            super(GCN, self).__init__()
    
            self.gc1 = GraphConvolution(nfeat, nhid)  # 构建第一层 GCN
            self.gc2 = GraphConvolution(nhid, nclass)  # 构建第二层 GCN
            self.dropout = dropout
    
        def forward(self, x, adj):
            x = F.relu(self.gc1(x, adj))
            x = F.dropout(x, self.dropout, training=self.training)
            x = self.gc2(x, adj)
            return F.log_softmax(x, dim=1)

layers:

import math
    
    import torch
    
    from torch.nn.parameter import Parameter
    from torch.nn.modules.module import Module
    
    
    class GraphConvolution(Module):
        """ Simple GCN layer, similar to https://arxiv.org/abs/1609.02907 """
    
        def __init__(self, in_features, out_features, bias=True):
            super(GraphConvolution, self).__init__()
            self.in_features = in_features
            self.out_features = out_features
            self.weight = Parameter(torch.FloatTensor(in_features, out_features))  # input_features, out_features
            if bias:
                self.bias = Parameter(torch.FloatTensor(out_features))
            else:
                self.register_parameter('bias', None)
            self.reset_parameters()
    
        def reset_parameters(self):
            stdv = 1. / math.sqrt(self.weight.size(1))
            self.weight.data.uniform_(-stdv, stdv)  # 随机化参数
            if self.bias is not None:
                self.bias.data.uniform_(-stdv, stdv)
    
        def forward(self, input, adj):
            support = torch.mm(input, self.weight)  # GraphConvolution forward。input*weight
            output = torch.spmm(adj, support)  # 稀疏矩阵的相乘，和mm一样的效果
            if self.bias is not None:
                return output + self.bias
            else:
                return output
    
        def __repr__(self):
            return self.__class__.__name__ + ' (' \
                   + str(self.in_features) + ' -> ' \
                   + str(self.out_features) + ')'

# 初始化模型 #

调用模型：

model = GCN(nfeat=features.shape[1],
                nhid=args.hidden,
                nclass=labels.max().item() + 1,
                dropout=args.dropout)

具体参数：

model = GCN(nfeat=1433,
                nhid=16,
                nclass=7,
                dropout=0.5)

初始化模型两层`GCN`:

self.gc1 = GraphConvolution(nfeat=1433, nhid=16)  # 构建第一层 GCN
    self.gc2 = GraphConvolution(nhid=16, nclass=7)  # 构建第二层 GCN
    self.dropout = 0.5

初始化具体layer:  
**第一层：gc1**

def __init__(self, in_features, out_features, bias=True):
        super(GraphConvolution, self).__init__()
        self.in_features = 1433
        self.out_features = 16
        self.weight = Parameter(torch.FloatTensor(1433, 16))  # input_features, out_features
    	self.bias = Parameter(torch.FloatTensor(16))
        self.reset_parameters() # 初始化w和b

参数 w w w的维度： W 1433 × 16 W\_\{1433 \\times 16\} W1433×16  
参数 b b b的维度： b 1 × 16 b\_\{1 \\times 16\} b1×16  
**第二层：gc2**

def __init__(self, in_features, out_features, bias=True):
        super(GraphConvolution, self).__init__()
        self.in_features = 16
        self.out_features = 7
        self.weight = Parameter(torch.FloatTensor(16, 7))  # input_features, out_features
    	self.bias = Parameter(torch.FloatTensor(7))
        self.reset_parameters() # 初始化w和b

参数 w w w的维度： W 1433 × 16 W\_\{1433 \\times 16\} W1433×16  
参数 b b b的维度：  b 1 × 7 b\_\{1 \\times 7\} b1×7

# forward执行模型 #

1.  首先执行model:

def forward(self, x, adj):
        x = F.relu(self.gc1(x, adj))
        x = F.dropout(x, self.dropout, training=self.training)
        x = self.gc2(x, adj)
        return F.log_softmax(x, dim=1)

1.  执行`self.gc1(x, adj)`，`x`表示输入特征，维度 2708 × 1433 2708 \\times 1433 2708×1433，`adj`表示邻接矩阵，维度 2708 × 2708 2708 \\times 2708 2708×2708
2.  执行GCN layer gc1层，

support = torch.mm(input, self.weight)  # GraphConvolution forward。input*weight
            output = torch.spmm(adj, support)

计算output， o u t p u t 2708 × 16 = a d j 2708 × 2708 × i n p u t 2708 × 1433 × W 1433 × 16 output\_\{2708 \\times 16\} = adj\_\{2708 \\times 2708\} \\times input\_\{2708 \\times 1433\} \\times W\_\{1433 \\times 16\} output2708×16=adj2708×2708×input2708×1433×W1433×16，然后返回 o u t p u t = o u t p u t 2708 × 16 + b i a s 1 × 16 output = output\_\{2708 \\times 16\} + bias\_\{1 \\times 16\} output=output2708×16\+bias1×16

output[0]=
    tensor([ 0.0201, -0.0242,  0.0608,  0.0272,  0.0133,  0.0085,  0.0084, -0.0265,
             0.0149, -0.0100,  0.0077,  0.0029,  0.0145, -0.0181, -0.0021, -0.0183],
           grad_fn=<SelectBackward>)
    self.bias=
    Parameter containing:
    tensor([-0.2232, -0.0295, -0.1387,  0.2170, -0.1749, -0.1551,  0.1056, -0.1860,
            -0.0666, -0.1327,  0.0212,  0.1587,  0.2496, -0.0154, -0.1683,  0.0151],
           requires_grad=True)
    (output + self.bias)[0]=
    tensor([-0.2030, -0.0537, -0.0779,  0.2442, -0.1616, -0.1466,  0.1140, -0.2125,
            -0.0516, -0.1427,  0.0289,  0.1615,  0.2641, -0.0336, -0.1704, -0.0032],
           grad_fn=<SelectBackward>)

1.  使用 R e l u Relu Relu激活函数，

x = F.relu(self.gc1(x, adj))
    x[0]=
    tensor([0.0000, 0.0000, 0.0000, 0.2442, 0.0000, 0.0000, 0.1140, 0.0000, 0.0000,
           0.0000, 0.0289, 0.1615, 0.2641, 0.0000, 0.0000, 0.0000],
          grad_fn=<SelectBackward>)

1.  在training阶段，使用 d r o p o u t dropout dropout， 执行 x = x 1 − 0.5 x=\\frac\{x\}\{1-0.5\} x=1−0.5x，并以0.5的概率去除:

x = F.dropout(x, self.dropout, training=self.training)
    x[0]=
    tensor([0.0000, 0.0000, 0.0000, 0.4884, 0.0000, 0.0000, 0.2280, 0.0000, 0.0000,
            0.0000, 0.0000, 0.3230, 0.5282, 0.0000, 0.0000, 0.0000],
           grad_fn=<SelectBackward>)

1.  执行第二层 gc2

support = torch.mm(input, self.weight)  # GraphConvolution forward。input*weight
            output = torch.spmm(adj, support)

计算output， o u t p u t 2708 × 7 = a d j 2708 × 2708 × i n p u t 2708 × 16 × W 16 × 7 output\_\{2708 \\times 7\} = adj\_\{2708 \\times 2708\} \\times input\_\{2708 \\times 16\} \\times W\_\{16 \\times 7\} output2708×7=adj2708×2708×input2708×16×W16×7，然后返回 o u t p u t = o u t p u t 2708 × 7 + b i a s 1 × 7 output = output\_\{2708 \\times 7\} + bias\_\{1 \\times 7\} output=output2708×7\+bias1×7

output[0]=
    tensor([-0.1928,  0.1723,  0.1689, -0.0516,  0.0387, -0.0276, -0.1027],
           grad_fn=<SelectBackward>)

1.  将返回结果x，直接吐给 F . l o g \_ s o f t m a x ( x , d i m = 1 ) F.log\\\_softmax(x, dim=1) F.log\_softmax(x,dim=1)， d i m = 1 dim=1 dim=1表示对1433维度进行log\_softmax

x[0]=
    tensor([-2.1474, -1.7823, -1.7856, -2.0062, -1.9158, -1.9822, -2.0573],
           grad_fn=<SelectBackward>)

1.  将output与label进行计算loss 与 acc\_train  
    `loss=tensor(1.9186, grad_fn=<NllLossBackward>)` `acc_train=tensor(0.1357, dtype=torch.float64)`
2.  最后进行反向传播，更新梯度W和b
3.  完成一次train的过程