【Pytorch学习笔记】2.Pytoch基础

Dear 丶 2022-09-04 05:54 399阅读 0赞

文章目录

  • 4.基本数据类型
    • 4.1 All is about Tensor
    • 4.2 How to denote string
    • 4.3 Data type
    • 4.4 Type check
    • 4.5 Dimension / rank
    • 4.6 Mixed
  • 5.创建Tensor
    • 5.1 Import from numpy
    • 5.2 Import from List
    • 5.3 uninitialized
    • 5.4 set default type
    • 5.5 rand / rand_like, randint, randn
    • 5.6 full
    • 5.7 arrange / range
    • 5.8 linspace / logspace
    • 5.9 Ones / zeros / eye
    • 5.10 randperm
  • 6.索引与切片
    • 6.1 …
    • 6.2 select by mask
    • 6.3 select by flatten index
  • 7.维度变换
    • 7.1 View / reshape
    • 7.2 Sequeeze / unsqueeze
    • 7.3 Expand / repeat
    • 7.4 Transpose / t / permute
    1. Broadcasting
  • 9.拼接与拆分
    • 9.1 Cat
    • 9.2 Stack
    • 9.3 split: by len
    • 9.4 Chunk: by num
  • 10.基本运算
  • 11.数据统计
  • 12.高级OP
    • 12.1 Where
    • 12.2 Gather

根据龙良曲Pytorch学习视频整理，视频链接：
【计算机-AI】PyTorch学这个就够了！
(好课推荐)深度学习与PyTorch入门实战——主讲人龙良曲

4.基本数据类型

4.1 All is about Tensor






























python PyTorch
Int IntTensor of size()
float FloatTensor of size()
Int array IntTensor of size[d1,d2,…]
Float array FloatTensor of size[d1,d2…]
string –

4.2 How to denote string

  • One-hot：[0, 1, 0, 0,…]
  • Embedding：Word2vec，glove

4.3 Data type

在这里插入图片描述

4.4 Type check

  1. a = torch.randn(2, 3)
  2. print(a)    # tensor([[ 0.3765, 0.6532, -0.8246],
  3.             # [ 1.3567, -0.3963, 0.1719]])
  4. print(a.type())     # torch.FloatTensor
  5. print(type(a))      # <class 'torch.Tensor'>
  6. print(isinstance(a, torch.FloatTensor)) # True
  8. # 数据位置对数据类型的影响
  9. print(isinstance(a, torch.cuda.FloatTensor))    # False
  10. a = a.cuda()
  11. print(isinstance(a, torch.cuda.FloatTensor))    # True

4.5 Dimension / rank

  • Dimension 0 ：Loss
  • Dimension 1 ：Bias和Linear Input
  • Dimension 2 ：Linear Input batch
  • Dimension 3 ：RNN Input Batch
  • Dimension 4 ：CNN[b, c, h, w]

4.6 Mixed

  1. a = torch.rand(2, 3, 4, 5)
  2. print(a.size())     # torch.Size([2, 3, 4, 5])
  3. print(a.shape)      # torch.Size([2, 3, 4, 5])
  4. print(len(a.shape)) # 4
  5. print(a.dim())      # 维度：4
  6. print(a.numel())    # 具体的数据大小：120

5.创建Tensor

5.1 Import from numpy

  1. a = np.ones(2)
  2. print(a)            # [1. 1.]
  3. print(torch.from_numpy(a))  # tensor([1., 1.], dtype=torch.float64)

5.2 Import from List

  1. a = torch.tensor([2, 3.2])
  2. print(a)    # tensor([2.0000, 3.2000])

torch.Tensor()接受数据的类型
torch.tensor()接收现成的数据

5.3 uninitialized

初始化的数据可能会非常大或者非常小，不建议使用

  • torch.empty()
  • torch.FloatTensor(d1, d2, d3)
  • torch.IntTensor(d1, d2, d3)

5.4 set default type

  1. print(torch.tensor([1.2, 3]).type())    # torch.FloatTensor
  2. torch.set_default_tensor_type(torch.DoubleTensor)
  3. print(torch.tensor([1.2, 3]).type())    # torch.DoubleTensor

5.5 rand / rand_like, randint, randn

推荐使用这个随机初始化

  1. a = torch.rand(2, 1)
  2. print(a)    # tensor([[0.1746],
  3.             # [0.6831]])
  4. print(torch.rand_like(a))   # tensor([[0.5452],
  5.                             # [0.4782]])
  6. print(torch.randint(1, 5, [2, 1]))  # tensor([[3],
  7. # 限定了最小值和最大值，左闭右开 # [4]])

torch.randn() 输出随机数服从正态分布(0, 1)
torch.normal(mean=torch.full([10], 0), std=torch.arrange(1, 0, -0.1)) 自定义正态分布均值和标准差

5.6 full

  1. print(torch.full([], 7))    # 标量：tensor(7)
  2. print(torch.full([1], 7))   # 张量：tensor([7])

5.7 arrange / range

  1. print(torch.arange(0, 10, 2))   # tensor([0, 2, 4, 6, 8])
  2. print(torch.range(0, 10, 2))    # 不建议使用，tensor([ 0., 2., 4., 6., 8., 10.])

5.8 linspace / logspace

  1. print(torch.linspace(0, 10, steps=3))    # 等差均分为steps份：tensor([ 0., 5., 10.])
  2. print(torch.logspace(0, -1, steps=5))    # 对数均分为steps份：tensor([1.0000, 0.5623, 0.3162, 0.1778, 0.1000])

5.9 Ones / zeros / eye

  1. print(torch.ones(1, 2))     # 全1矩阵：tensor([[1., 1.]])
  2. print(torch.zeros(1, 2))    # 全0矩阵：tensor([[0., 0.]])
  3. print(torch.eye(2))         # 单位矩阵：tensor([[1., 0.],
  4.                             # [0., 1.]])

5.10 randperm

  1. a = torch.rand(4, 3)
  2. print(a)
  3. idx = torch.randperm(3)     # 随机生成索引
  4. print(idx)
  5. print(a[[1, 0, 2]])
  6. """ tensor([[0.1708, 0.2821, 0.8163], [0.8898, 0.6628, 0.2350], [0.3743, 0.4281, 0.5309], [0.4996, 0.7259, 0.5485]]) tensor([1, 0, 2]) tensor([[0.8898, 0.6628, 0.2350], [0.1708, 0.2821, 0.8163], [0.3743, 0.4281, 0.5309]]) """

6.索引与切片

索引和切片的方式同python list[start:end:steps]

  • Indexing
  • select first / last N
  • select by steps
  • select by specific index

6.1 …

连续全部取值

  1. a = torch.rand(4, 3, 28, 28)
  2. print(a[0, 1].shape)    # torch.Size([28, 28])
  3. print(a[0, 0, 2, 4])    # tensor(0.9485)
  4. print(a.index_select(2, torch.arange(8)).shape)         # torch.Size([4, 3, 8, 28])
  5. print(a[:,:1,...].shape)       #torch.Size([4, 1, 28, 28])

6.2 select by mask

  1. x = torch.randn(3, 4)
  2. print(x)
  3. mask = x.ge(0.5)    # 元素大于0.5的
  4. print(mask)
  5. print(torch.masked_select(x, mask))
  6. """ tensor([[-1.8692, 0.9687, -0.4978, 0.7836], [-2.5662, 0.0487, 0.3978, -0.3676], [-1.5896, -0.1129, -1.9687, 0.5585]]) tensor([[False, True, False, True], [False, False, False, False], [False, False, False, True]]) tensor([0.9687, 0.7836, 0.5585]) """

6.3 select by flatten index

reshape为一维后取索引

  1. src = torch.tensor([[4, 3, 5],
  2.                     [6, 7, 8]])
  3. print(torch.take(src, torch.tensor([0, 2, 5])))       # tensor([4, 5, 8])

7.维度变换

7.1 View / reshape

view的size前后必须一致，view的size要有实际意义，避免产生数据污染

  1. a = torch.rand(4, 1, 28, 28)
  2. print(a.view(4, 28*28).shape)   # torch.Size([4, 784])
  3. print(a.unsqueeze(4).shape)     # torch.Size([4, 1, 28, 28, 1])

7.2 Sequeeze / unsqueeze

unsqueeze插入一个维度，不会改变数据本身
squeeze若给定的挤压维度为1则被挤压掉，若不为1则保持原来数据维度不变

  1. b = torch.tensor([1, 2])
  2. print(b.unsqueeze(-1))  # tensor([[1],
  3.                         # [2]])
  4. print(b.unsqueeze(0))   # tensor([[1, 2]])
  6. c = torch.rand(1, 31, 1, 1)
  7. print(c.shape)                  # torch.Size([1, 31, 1, 1])
  8. print(c.squeeze().shape)        # torch.Size([31])
  9. print(c.squeeze(0).shape)       # torch.Size([31, 1, 1])
  10. print(c.squeeze(1).shape)       # torch.Size([1, 31, 1, 1])

7.3 Expand / repeat

  • Expand(broadcasting)

  • repeat(memory copied)

    a = torch.rand(1,32,1,1)
    print(a.shape) # torch.Size([1, 32, 1, 1])
    print(a.expand(4,32,14,14).shape) # torch.Size([4, 32, 14, 14])
    print(a.repeat(4,32,1,1).shape) # torch.Size([4, 1024, 1, 1])

7.4 Transpose / t / permute

t()只能转置二维矩阵，否则报错

  1. a = torch.rand(2,3)
  2. print(a.t().shape)      # torch.Size([3, 2])
  4. b = torch.rand(4,3,28,28)
  5. print(b.transpose(1,3).transpose(1,2).shape)    # torch.Size([4, 28, 28, 3])
  6. print(b.permute(0,2,3,1).shape)                 # torch.Size([4, 28, 28, 3])

在这里插入图片描述

8. Broadcasting

  • Expand
  • without copying data

key idea

  • insert 1 dim ahead
  • Expand dims with size 1 to same size
    在这里插入图片描述
    Why broadcasting
  • for actual demanding
  • memory consumption

Is it broadcating-able?

  • Match from Last dim
    if current dim=1, expand to same
    if either has no dim, insert one dim and expand to same
    otherwise, NOT broadcasting-able

9.拼接与拆分

9.1 Cat

只能在维度不同的dim上进行cat

  1. a = torch.rand(4, 32, 8)
  2. b = torch.rand(5, 32, 8)
  3. print(torch.cat([a, b], dim=0).shape)    # torch.Size([9, 32, 8])

9.2 Stack

会创建新的维度，两个维度必须一致

  1. a = torch.rand(32, 8)
  2. b = torch.rand(32, 8)
  3. print(torch.stack([a, b], dim=0).shape)    # torch.Size([2, 32, 8])

9.3 split: by len

  1. a = torch.rand(4, 32, 8)
  2. aa, bb = a.split(2, dim=0)
  3. print(aa.shape, bb.shape)    # torch.Size([2, 32, 8]) torch.Size([2, 32, 8])
  4. cc, dd = a.split([3, 1], dim=0)
  5. print(cc.shape, dd.shape)    # torch.Size([3, 32, 8]) torch.Size([1, 32, 8])

9.4 Chunk: by num

  1. a = torch.rand(4, 32, 8)
  2. aa, bb = a.chunk(2, dim=0)
  3. print(aa.shape, bb.shape)    #torch.Size([2, 32, 8]) torch.Size([2, 32, 8])

10.基本运算

  • Add / minus / multiply / divide
    torch.add() torch.sub() torch.mul() torch.div()
  • Matmul矩阵乘法
    torh.mm() only for 2d
    torch.matmul()
    @
  • Pow
    tensor.pow(n) = tensor**(n)
  • Sqrt / rsqrt
    sqrt()开平方
    rsqrt()开平方的倒数
  • Exp log
    torch.exp()
    torch.log()
  • Approximation
    tensor.floor() tensor.ceil() tensor.round()
    tensor.trunc()整数部分 tensor.frac()小数部分

  • clamp
    gradient clipping

    import torch
    import numpy as np

    grad = torch.rand(2, 3) * 15
    print(grad)
    print(grad.min())
    print(grad.median())
    print(grad.max())
    print(grad.clamp(10))
    print(grad.clamp(0, 10))
    “”” tensor([[11.9217, 3.4733, 1.3133], [ 7.1854, 13.8410, 13.8098]]) tensor(1.3133) tensor(7.1854) tensor(13.8410) tensor([[11.9217, 10.0000, 10.0000], [10.0000, 13.8410, 13.8098]]) tensor([[10.0000, 3.4733, 1.3133], [ 7.1854, 10.0000, 10.0000]]) “””

11.数据统计

  • norm-p 范数（简单的理解）、范数的用途、什么是范数1

    import torch
    import numpy as np

    a = torch.full([8], 1.0)
    print(a)
    b = a.view(2, 4)
    print(b)
    c = a.view(2, 2, 2)
    print(c)

    print(a.norm(1), b.norm(1), c.norm(1))
    print(a.norm(2), b.norm(2), c.norm(2))

    print(b.norm(1, dim=1))
    print(b.norm(2, dim=1))

    print(c.norm(1, dim=0))
    print(c.norm(2, dim=0))
    “”” tensor([1., 1., 1., 1., 1., 1., 1., 1.]) tensor([[1., 1., 1., 1.], [1., 1., 1., 1.]]) tensor([[[1., 1.], [1., 1.]], [[1., 1.], [1., 1.]]]) tensor(8.) tensor(8.) tensor(8.) tensor(2.8284) tensor(2.8284) tensor(2.8284) tensor([4., 4.]) tensor([2., 2.]) tensor([[2., 2.], [2., 2.]]) tensor([[1.4142, 1.4142], [1.4142, 1.4142]]) “””

  • tensor.mean()均值

  • tensor.prod()累乘
  • tensor.sum() 累加
  • tensor.argmin(), tensor.argmax() 最大、小值索引
  • dim=n, keepdim=True 设置维度，保持维度不变
  • kthvalue, topk
    tensor.topk(n, dim=, largest=False) 某一维度上最小的前n个
    tensor.kthvalue(n, dim=) 某一维度上最小的第n个

  • compare

    a = torch.rand(2, 3)
    print(a)
    print(torch.gt(a, 0.5)) # 大于0.5
    print(torch.eq(a, a))
    print(torch.equal(a, a))
    “”” tensor([[0.5229, 0.7868, 0.0872], [0.3480, 0.4691, 0.2402]]) tensor([[ True, True, False], [False, False, False]]) tensor([[True, True, True], [True, True, True]]) True “””

12.高级OP

12.1 Where

torch.where(condition, x, y)

  • Return a tensor of elements selected from either x or y, depending on condition.
  • where相较于python的if else优点是可以部署在GPU上，实现高度并行

12.2 Gather

torch.gather(input, dim, index, out=None)

  • Gathers values along an axis specified by dim.

  • retrieve global label
    argmax(pred) to get relative labeling
    On some condtion, our label is dinstinct from relative labeling

    import torch
    import numpy as np

    prob = torch.randn(4, 10)
    idx = prob.topk(dim=1, k=3)
    print(idx)
    “”” torch.return_types.topk( values=tensor([[1.3319, 0.8338, 0.4407], [2.0854, 0.9303, 0.2608], [1.7225, 1.0136, 0.5523], [1.3715, 1.3115, 1.0049]]), indices=tensor([[2, 9, 3], [1, 3, 8], [3, 5, 0], [3, 7, 1]])) “””

    label = torch.arange(10) + 100
    print(label)
    “”” tensor([100, 101, 102, 103, 104, 105, 106, 107, 108, 109]) “””

    print(torch.gather(label.expand(4, 10), dim=1, index=idx1.long()))
    “”” tensor([[102, 109, 103], [101, 103, 108], [103, 105, 100], [103, 107, 101]]) “””

发表评论

表情：
评论列表 （有 0 条评论，399人围观）

还没有评论，来说两句吧...

相关阅读