pandas的基本使用

旧城等待， 2024-04-17 15:15 81阅读 0赞

### 下载 ###

下载pandas在命令行中输入`conda install pandas`，如果想要查看自己下载的版本，在jupyter notebook中输入`!conda list pandas`，在命令行中输入`conda list pandas`，就可以返回版本号。

### pandas series ###

Pandas series是一个带标签的一维数组，或者说像是一个字典。与NumPy不同，pandas可以有不同类型的数据，而且还可以为每一个元素定义一个索引标签。  
  
首先，我们需要信用pandas库。

import pandas as pd

### pandas series ###

我们可以使用pd.Series(date,index)来创建，其中，index代表一系列索引标签。

import pandas as pd
    
    groceries = pd.Series(data = [30,6,'Yes','No'],index = ['eggs','apples','milk','bread'])
    
    groceries

eggs       30
    apples      6
    milk      Yes
    bread      No
    dtype: object

可以看出，第一列输出为索引，并且索引不是0到3，而是我们设置的标签。第二列是数据，且有各种数据形式。  
  
Pandas和NumPy类似，有很多可以返回信息的函数。比如说`.shape`,`.ndim`,`.size`。

print('Groceries has shape:', groceries.shape)
    print('Groceries has dimension:', groceries.ndim)
    print('Groceries has a total of', groceries.size, 'elements')

Groceries has shape: (4,)
    Groceries has dimension: 1
    Groceries has a total of 4 elements

pandas series还支持单独输出索引和数据。

print(groceries.values)
    print(groceries.index)

[30 6 'Yes' 'No']
    Index(['eggs', 'apples', 'milk', 'bread'], dtype='object')

Pandas Series还支持使用in查找是否含有这个标签。

x = 'bananas' in groceries
    y = 'bread' in groceries
    
    print(x)
    print(y)

False
    True

### 访问，修改，删除 ###

首先，我们可以通过标签索引访问数据，也可以使用数字索引（支持正索引和负索引）。为了消除歧义，pandas有两个属性.loc和.iloc。.loc特别代表标签索引，.iloc特别代表数字索引。

print(groceries['eggs'])
    print(groceries[['eggs','bread']])
    print(groceries.loc[['eggs','apples']])
    print()
    
    print(groceries[[0,1]])
    print(groceries[[-1]])
    print(groceries.iloc[[0,1]])

30
    eggs     30
    bread    No
    dtype: object
    eggs      30
    apples     6
    dtype: object
    
    eggs      30
    apples     6
    dtype: object
    bread    No
    dtype: object
    eggs      30
    apples     6
    dtype: object

如果想要修改某个数据的值，可以直接访问然后等号右边修改。

groceries['eggs'] = 2
    print(groceries)

eggs        2
    apples      6
    milk      Yes
    bread      No
    dtype: object

我们可以删除pandas series中的数据使用.drop()方法，但是这样修改的话只是返回了删除某个索引的值的数据，在原数据中并没有改变。如果想要改变原数组，需要添加关键字inplace = True

groceries = pd.Series(data = [30,6,'Yes','No'],index = ['eggs','apples','milk','bread'])
    print(groceries.drop('apples'))
    print(groceries)

eggs      30
    milk     Yes
    bread     No
    dtype: object
    eggs       30
    apples      6
    milk      Yes
    bread      No
    dtype: object

groceries.drop('apples',inplace = True)
    print(groceries)

eggs      30
    milk     Yes
    bread     No
    dtype: object

### 算术运算 ###

和numpy一样，Pandas Series支持元素层面上的运算。

fruits = pd.Series(data=[10,6,3], index = ['apples','oranges','bananas'])
    print(fruits)
    print()
    
    print(fruits + 2)
    print(fruits - 2)
    print(fruits * 2)
    print(fruits / 2)

apples     10
    oranges     6
    bananas     3
    dtype: int64
    
    apples     12
    oranges     8
    bananas     5
    dtype: int64
    apples     8
    oranges    4
    bananas    1
    dtype: int64
    apples     20
    oranges    12
    bananas     6
    dtype: int64
    apples     5.0
    oranges    3.0
    bananas    1.5
    dtype: float64

pandas还支持一些NumPy的数学函数。比如sqrt()。

import numpy as np
    
    print(fruits)
    
    print(np.exp(fruits))
    print(np.sqrt(fruits))
    print(np.power(fruits,2))

apples     10
    oranges     6
    bananas     3
    dtype: int64
    apples     22026.465795
    oranges      403.428793
    bananas       20.085537
    dtype: float64
    apples     3.162278
    oranges    2.449490
    bananas    1.732051
    dtype: float64
    apples     100
    oranges     36
    bananas      9
    dtype: int64

pandas还支持选择特定的数据进行操作。

print(fruits['bananas'] + 2)
    print(fruits.iloc[0]-2)
    print(fruits[['apples','oranges']]*2)
    print(fruits[['apples','oranges']] / 2)

5
    8
    apples     20
    oranges    12
    dtype: int64
    apples     5.0
    oranges    3.0
    dtype: float64

对于pandas series混合了几种数据类型，有时也可以进行算术操作，比如说。

groceries*2

eggs         60
    milk     YesYes
    bread      NoNo
    dtype: object

groceries / 2

---------------------------------------------------------------------------
    
    TypeError                                 Traceback (most recent call last)
    
    ~\Anaconda3\envs\yang\lib\site-packages\pandas\core\ops.py in na_op(x, y)
       1504         try:
    -> 1505             result = expressions.evaluate(op, str_rep, x, y, **eval_kwargs)
       1506         except TypeError:
    
    
    ~\Anaconda3\envs\yang\lib\site-packages\pandas\core\computation\expressions.py in evaluate(op, op_str, a, b, use_numexpr, **eval_kwargs)
        207     if use_numexpr:
    --> 208         return _evaluate(op, op_str, a, b, **eval_kwargs)
        209     return _evaluate_standard(op, op_str, a, b)
    
    
    ~\Anaconda3\envs\yang\lib\site-packages\pandas\core\computation\expressions.py in _evaluate_standard(op, op_str, a, b, **eval_kwargs)
         67     with np.errstate(all='ignore'):
    ---> 68         return op(a, b)
         69 
    
    
    TypeError: unsupported operand type(s) for /: 'str' and 'int'
    
    
    During handling of the above exception, another exception occurred:
    
    
    TypeError                                 Traceback (most recent call last)
    
    ~\Anaconda3\envs\yang\lib\site-packages\pandas\core\ops.py in safe_na_op(lvalues, rvalues)
       1528             with np.errstate(all='ignore'):
    -> 1529                 return na_op(lvalues, rvalues)
       1530         except Exception:
    
    
    ~\Anaconda3\envs\yang\lib\site-packages\pandas\core\ops.py in na_op(x, y)
       1506         except TypeError:
    -> 1507             result = masked_arith_op(x, y, op)
       1508 
    
    
    ~\Anaconda3\envs\yang\lib\site-packages\pandas\core\ops.py in masked_arith_op(x, y, op)
       1025             with np.errstate(all='ignore'):
    -> 1026                 result[mask] = op(xrav[mask], y)
       1027 
    
    
    TypeError: unsupported operand type(s) for /: 'str' and 'int'
    
    
    During handling of the above exception, another exception occurred:
    
    
    TypeError                                 Traceback (most recent call last)
    
    <ipython-input-23-3d64b17810f9> in <module>
    ----> 1 groceries / 2
    
    
    ~\Anaconda3\envs\yang\lib\site-packages\pandas\core\ops.py in wrapper(left, right)
       1581             rvalues = rvalues.values
       1582 
    -> 1583         result = safe_na_op(lvalues, rvalues)
       1584         return construct_result(left, result,
       1585                                 index=left.index, name=res_name, dtype=None)
    
    
    ~\Anaconda3\envs\yang\lib\site-packages\pandas\core\ops.py in safe_na_op(lvalues, rvalues)
       1531             if is_object_dtype(lvalues):
       1532                 return libalgos.arrmap_object(lvalues,
    -> 1533                                               lambda x: op(x, rvalues))
       1534             raise
       1535 
    
    
    pandas/_libs/algos.pyx in pandas._libs.algos.arrmap()
    
    
    ~\Anaconda3\envs\yang\lib\site-packages\pandas\core\ops.py in <lambda>(x)
       1531             if is_object_dtype(lvalues):
       1532                 return libalgos.arrmap_object(lvalues,
    -> 1533                                               lambda x: op(x, rvalues))
       1534             raise
       1535 
    
    
    TypeError: unsupported operand type(s) for /: 'str' and 'int'

### Pandas DataFrames ###

Pandas DataFrames是一个有行标签和列标签的两维数据结构，有点像Excel表格的结构。  
  
建立一个DataFrame，需要先建立一个Pandas Series的字典，然后将这个字典放入pd.DataFrame()函数。

items = { 'Bob': pd.Series(data =[245,25,55], index = ['bike','pants','watch']),
            'Alice' : pd.Series(data = [40,110,500,45], index = ['book','glasses','bike','pants'])}
    
    print(type(items))

shopping_carts = pd.DataFrame(items)
    shopping_carts

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>Bob</th> 
   <th>Alice</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>bike</th> 
   <td>245.0</td> 
   <td>500.0</td> 
  </tr> 
  <tr> 
   <th>book</th> 
   <td>NaN</td> 
   <td>40.0</td> 
  </tr> 
  <tr> 
   <th>glasses</th> 
   <td>NaN</td> 
   <td>110.0</td> 
  </tr> 
  <tr> 
   <th>pants</th> 
   <td>25.0</td> 
   <td>45.0</td> 
  </tr> 
  <tr> 
   <th>watch</th> 
   <td>55.0</td> 
   <td>NaN</td> 
  </tr> 
 </tbody> 
</table>

我们可以注意到第一列的标签按照字母表的顺序排列了下来。其中，还有很多位置显示的是NaN。这是pandas表示那个特定行和列索引的位置没有数据的一种方式。  
  
如果我们创建Pandas DataFrame的时候Pandas Series没有定义清楚的标签，那么pandas会生成数字索引的行标签。

data = { 'Bob': pd.Series([245,25,55]),'Alice' : pd.Series([40,110,500,45])}
    
    df = pd.DataFrame(data)
    df

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>Bob</th> 
   <th>Alice</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>0</th> 
   <td>245.0</td> 
   <td>40</td> 
  </tr> 
  <tr> 
   <th>1</th> 
   <td>25.0</td> 
   <td>110</td> 
  </tr> 
  <tr> 
   <th>2</th> 
   <td>55.0</td> 
   <td>500</td> 
  </tr> 
  <tr> 
   <th>3</th> 
   <td>NaN</td> 
   <td>45</td> 
  </tr> 
 </tbody> 
</table>

我们可以利用DataFrame的属性提取一些信息。

print(shopping_carts.shape)
    print(shopping_carts.ndim)
    print(shopping_carts.size)
    print(shopping_carts.values)
    print()
    print(shopping_carts.index)
    print(shopping_carts.columns)

(5, 2)
    2
    10
    [[245. 500.]
     [ nan  40.]
     [ nan 110.]
     [ 25.  45.]
     [ 55.  nan]]
    
    Index(['bike', 'book', 'glasses', 'pants', 'watch'], dtype='object')
    Index(['Bob', 'Alice'], dtype='object')

有些时候我们不需要将整个字典都放到DataFrame中，我们可以使用关键字columns和index来限定。

bob_shopping_cart = pd.DataFrame(items,columns=['Bob'])
    
    bob_shopping_cart

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>Bob</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>bike</th> 
   <td>245</td> 
  </tr> 
  <tr> 
   <th>pants</th> 
   <td>25</td> 
  </tr> 
  <tr> 
   <th>watch</th> 
   <td>55</td> 
  </tr> 
 </tbody> 
</table>

alice_sel_shopping_cart = pd.DataFrame(items,index = ['glasses','bike'],columns = ['Alice'])
    
    alice_sel_shopping_cart

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>Alice</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>glasses</th> 
   <td>110</td> 
  </tr> 
  <tr> 
   <th>bike</th> 
   <td>500</td> 
  </tr> 
 </tbody> 
</table>

我们也可以创建DataFrame使用列表字典，但是字典中的所有的列表必须有相同的长度。

data = { 'Integers' : [1,2,3],
            'Floats' : [4.5, 8.2, 9.6]}
    
    df = pd.DataFrame(data)
    
    df

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>Integers</th> 
   <th>Floats</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>0</th> 
   <td>1</td> 
   <td>4.5</td> 
  </tr> 
  <tr> 
   <th>1</th> 
   <td>2</td> 
   <td>8.2</td> 
  </tr> 
  <tr> 
   <th>2</th> 
   <td>3</td> 
   <td>9.6</td> 
  </tr> 
 </tbody> 
</table>

我们可以注意到上面的data字典没有索引标签，我们可以在pd.DataFrame中设置index关键字设置标签。

data = { 'Integers' : [1,2,3],
            'Floats' : [4.5, 8.2, 9.6]}
    
    df = pd.DataFrame(data, index = ['label 1', 'label 2', 'label 3'])
    
    df

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>Integers</th> 
   <th>Floats</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>label 1</th> 
   <td>1</td> 
   <td>4.5</td> 
  </tr> 
  <tr> 
   <th>label 2</th> 
   <td>2</td> 
   <td>8.2</td> 
  </tr> 
  <tr> 
   <th>label 3</th> 
   <td>3</td> 
   <td>9.6</td> 
  </tr> 
 </tbody> 
</table>

我们也可以使用一系列的python字典来创建dataframe。用法和上面类似，也可以使用index关键字。

items2 = [{ 'bikes': 20, 'pants': 30, 'watches': 35}, 
              { 'watches': 10, 'glasses': 50, 'bikes': 15, 'pants':5}]
    
    store_items = pd.DataFrame(items2, index = ['store 1', 'store 2'])
    
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>35</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>10</td> 
  </tr> 
 </tbody> 
</table>

## 访问数据 ##

DataFrame支持通过使用行标签，列标签来访问DataFrame中的行元素，列元素或者单个元素。

import pandas as pd
    items2 = [{ 'bikes': 20, 'pants': 30, 'watches': 35}, 
              { 'watches': 10, 'glasses': 50, 'bikes': 15, 'pants':5}]
    
    store_items = pd.DataFrame(items2, index = ['store 1', 'store 2'])
    
    
    print(store_items)
    print()
    print(store_items[['bikes']])
    print()
    print(store_items[['bikes','pants']])
    print()
    print(store_items.loc[['store 1']])
    print()
    print(store_items['bikes'],['store 2'])

bikes  glasses  pants  watches
    store 1     20      NaN     30       35
    store 2     15     50.0      5       10
    
             bikes
    store 1     20
    store 2     15
    
             bikes  pants
    store 1     20     30
    store 2     15      5
    
             bikes  glasses  pants  watches
    store 1     20      NaN     30       35
    
    store 1    20
    store 2    15
    Name: bikes, dtype: int64 ['store 2']

如果我们要访问单个元素，我们需要先列标签，再行标签，否则程序会报错。  
  
如果我们想要修改DataFrame，比如说加一列。如果我们想要给store\_items设置一个衬衫列，可以这样

store_items['shirts'] = [15,2]
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>watches</th> 
   <th>shirts</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>35</td> 
   <td>15</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>10</td> 
   <td>2</td> 
  </tr> 
 </tbody> 
</table>

我们也可以使用列间算术操作增加新的列。

store_items['suits'] = store_items['pants'] + store_items['shirts']
    
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>watches</th> 
   <th>shirts</th> 
   <th>suits</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>35</td> 
   <td>15</td> 
   <td>45</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>10</td> 
   <td>2</td> 
   <td>7</td> 
  </tr> 
 </tbody> 
</table>

如果想要添加新的行，我们需要先创建一个新的DataFrame，然后将它加入到原来的DataFrame中。比如说你新开了第三家商店，想要加入到其中。

new_items = [{ 'bikes':20,'pants':30, 'watches': 35,'glass': 4}]
    
    new_store = pd.DataFrame(new_items,index = ['store 3'])
    new_store

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glass</th> 
   <th>pants</th> 
   <th>watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>4</td> 
   <td>30</td> 
   <td>35</td> 
  </tr> 
 </tbody> 
</table>

然后我们可以使用append函数将这个新的商店信息加入到总的商店信息中

store_items = store_items.append(new_store)
    
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glass</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shirts</th> 
   <th>suits</th> 
   <th>watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>NaN</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>15.0</td> 
   <td>45.0</td> 
   <td>35</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>NaN</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>2.0</td> 
   <td>7.0</td> 
   <td>10</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>NaN</td> 
   <td>NaN</td> 
   <td>35</td> 
  </tr> 
 </tbody> 
</table>

我们会发现列元素会按照字母表顺序排列。  
  
你还可以通过控制特定列的特定行来创建一个新的列。比如说新建一个watches列通过下面的方式。

store_items['new watches'] = store_items['watches'][1:]
    
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glass</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shirts</th> 
   <th>suits</th> 
   <th>watches</th> 
   <th>new watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>NaN</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>15.0</td> 
   <td>45.0</td> 
   <td>35</td> 
   <td>NaN</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>NaN</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>2.0</td> 
   <td>7.0</td> 
   <td>10</td> 
   <td>10.0</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>NaN</td> 
   <td>NaN</td> 
   <td>35</td> 
   <td>35.0</td> 
  </tr> 
 </tbody> 
</table>

DataFrame支持再任意地方插入一个新行，使用dataframe.insert(loc,label,data)其中loc表示位置，label表示新的列的标签，data表示数据。例子：

store_items.insert(4,'shoes',[8,5,0])
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glass</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shoes</th> 
   <th>shirts</th> 
   <th>suits</th> 
   <th>watches</th> 
   <th>new watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>NaN</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>8</td> 
   <td>15.0</td> 
   <td>45.0</td> 
   <td>35</td> 
   <td>NaN</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>NaN</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>5</td> 
   <td>2.0</td> 
   <td>7.0</td> 
   <td>10</td> 
   <td>10.0</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>0</td> 
   <td>NaN</td> 
   <td>NaN</td> 
   <td>35</td> 
   <td>35.0</td> 
  </tr> 
 </tbody> 
</table>

我们可以添加行和列元素，也可以删除行和列元素。我们可以使用.pop()方法来删除列元素，可以使用.drop()方法，通过使用axis关键字来删除行或者列

store_items.pop('new watches')
    
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glass</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shoes</th> 
   <th>shirts</th> 
   <th>suits</th> 
   <th>watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>NaN</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>8</td> 
   <td>15.0</td> 
   <td>45.0</td> 
   <td>35</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>NaN</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>5</td> 
   <td>2.0</td> 
   <td>7.0</td> 
   <td>10</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>0</td> 
   <td>NaN</td> 
   <td>NaN</td> 
   <td>35</td> 
  </tr> 
 </tbody> 
</table>

store_items = store_items.drop(['watches','shoes'],axis = 1)
    
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glass</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shirts</th> 
   <th>suits</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>NaN</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>15.0</td> 
   <td>45.0</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>NaN</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>2.0</td> 
   <td>7.0</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>NaN</td> 
   <td>NaN</td> 
  </tr> 
 </tbody> 
</table>

store_items = store_items.drop(['store 2','store 1'],axis = 0)
    
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glass</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shirts</th> 
   <th>suits</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>NaN</td> 
   <td>NaN</td> 
  </tr> 
 </tbody> 
</table>

我们有时需要改变行标签或者列标签，我们可以使用.rename()方法。

store_items = store_items.rename(index = { 'store 3': 'last store'})
    
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glass</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shirts</th> 
   <th>suits</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>last store</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>NaN</td> 
   <td>NaN</td> 
  </tr> 
 </tbody> 
</table>

你也可以将索引改为列标签之一。

store_items = store_items.set_index('pants')
    
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glass</th> 
   <th>glasses</th> 
   <th>shirts</th> 
   <th>suits</th> 
  </tr> 
  <tr> 
   <th>pants</th> 
   <th></th> 
   <th></th> 
   <th></th> 
   <th></th> 
   <th></th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>30</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>NaN</td> 
   <td>NaN</td> 
   <td>NaN</td> 
  </tr> 
 </tbody> 
</table>

## 处理NaN ##

在我们处理大量的数据的时候，我们必须先处理一下其中的异常数据。其中大部分异常数据是应为数据缺失造成的。在pandas中，用NaN来代表缺失数据，下面我们将学习如何处理NaN的值。  
  
首先，我们先创建一个含有NaN的DataFrame。

items2 = [{ 'bikes': 20, 'pants': 30, 'watches': 35, 'shirts': 15, 'shoes':8, 'suits':45},
    { 'watches': 10, 'glasses': 50, 'bikes': 15, 'pants':5, 'shirts': 2, 'shoes':5, 'suits':7},
    { 'bikes': 20, 'pants': 30, 'watches': 35, 'glasses': 4, 'shoes':10}]
    
    store_items = pd.DataFrame(items2, index = ['store 1', 'store 2', 'store 3'])
    
    store_items

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shirts</th> 
   <th>shoes</th> 
   <th>suits</th> 
   <th>watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>15.0</td> 
   <td>8</td> 
   <td>45.0</td> 
   <td>35</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>2.0</td> 
   <td>5</td> 
   <td>7.0</td> 
   <td>10</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>30</td> 
   <td>NaN</td> 
   <td>10</td> 
   <td>NaN</td> 
   <td>35</td> 
  </tr> 
 </tbody> 
</table>

在这个例子中，我们可以看出来有三个NaN，但是如果数据量很大的话，我们就没办法直接看出来有多少个。我们可以使用一个组合方法来查看有多少个NaN。

x = store_items.isnull()
    
    print('Number of NaN values in our DataFrame:\n',x)

Number of NaN values in our DataFrame:
              bikes  glasses  pants  shirts  shoes  suits  watches
    store 1  False     True  False   False  False  False    False
    store 2  False    False  False   False  False  False    False
    store 3  False    False  False    True  False   True    False

x = store_items.isnull().sum()
    
    print('Number of NaN values in our DataFrame:\n',x)

Number of NaN values in our DataFrame:
     bikes      0
    glasses    1
    pants      0
    shirts     1
    shoes      0
    suits      1
    watches    0
    dtype: int64

x =  store_items.isnull().sum().sum()
    
    print('Number of NaN values in our DataFrame:', x)

Number of NaN values in our DataFrame: 3

我们可以看出.isnull()返回一个布尔型的DataFrame，True代表这个地方是NaN。在pandas中，我们将1代表true，0代表False。所以我们可以使用.sum()方法计算。  
  
当然，我们可以使用.count()来查询有多少个不是NaN的数。

print('Number of non-NaN values in the columns of our DataFrame:\n', store_items.count())

Number of non-NaN values in the columns of our DataFrame:
     bikes      3
    glasses    2
    pants      3
    shirts     2
    shoes      3
    suits      2
    watches    3
    dtype: int64

对于是NaN的地方，我们有两个选择，一是删除它，二是替代它。  
  
如果我们想要删除它，我们可以使用.dropna(axis)，其中当axis=0时删除行，当axis=1时删除列。

store_items.dropna(axis = 1)

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>pants</th> 
   <th>shoes</th> 
   <th>watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>30</td> 
   <td>8</td> 
   <td>35</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>5</td> 
   <td>5</td> 
   <td>10</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>30</td> 
   <td>10</td> 
   <td>35</td> 
  </tr> 
 </tbody> 
</table>

我们需要注意，我们这样子操作，store\_items并没有什么变化。如果想要将原来的DataFrame改变，需要使用关键字inplace=True。  
  
我们也可以选择使用合适的值来取代NaN。我们可以使用.fillna（）。比如说用0取代所有NaN。

store_items.fillna(0)

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shirts</th> 
   <th>shoes</th> 
   <th>suits</th> 
   <th>watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>0.0</td> 
   <td>30</td> 
   <td>15.0</td> 
   <td>8</td> 
   <td>45.0</td> 
   <td>35</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>2.0</td> 
   <td>5</td> 
   <td>7.0</td> 
   <td>10</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>30</td> 
   <td>0.0</td> 
   <td>10</td> 
   <td>0.0</td> 
   <td>35</td> 
  </tr> 
 </tbody> 
</table>

我么们也可以使用.fillna()方法来取代NaN使用先前值。使用.fillna(method = ‘ffill’,axis)。当使用的方法是ffill时，使用的时前面的值填充，当使用的方法是backfill时，使用的是后填充。axis控制的是行或者列。方法与之前一样。

store_items.fillna(method = 'ffill',axis = 0)

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shirts</th> 
   <th>shoes</th> 
   <th>suits</th> 
   <th>watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>NaN</td> 
   <td>30</td> 
   <td>15.0</td> 
   <td>8</td> 
   <td>45.0</td> 
   <td>35</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>2.0</td> 
   <td>5</td> 
   <td>7.0</td> 
   <td>10</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>30</td> 
   <td>2.0</td> 
   <td>10</td> 
   <td>7.0</td> 
   <td>35</td> 
  </tr> 
 </tbody> 
</table>

store_items.fillna(method = 'ffill',axis=1)

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shirts</th> 
   <th>shoes</th> 
   <th>suits</th> 
   <th>watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20.0</td> 
   <td>20.0</td> 
   <td>30.0</td> 
   <td>15.0</td> 
   <td>8.0</td> 
   <td>45.0</td> 
   <td>35.0</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15.0</td> 
   <td>50.0</td> 
   <td>5.0</td> 
   <td>2.0</td> 
   <td>5.0</td> 
   <td>7.0</td> 
   <td>10.0</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20.0</td> 
   <td>4.0</td> 
   <td>30.0</td> 
   <td>30.0</td> 
   <td>10.0</td> 
   <td>10.0</td> 
   <td>35.0</td> 
  </tr> 
 </tbody> 
</table>

store_items.fillna(method = 'backfill', axis = 0)

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shirts</th> 
   <th>shoes</th> 
   <th>suits</th> 
   <th>watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20</td> 
   <td>50.0</td> 
   <td>30</td> 
   <td>15.0</td> 
   <td>8</td> 
   <td>45.0</td> 
   <td>35</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15</td> 
   <td>50.0</td> 
   <td>5</td> 
   <td>2.0</td> 
   <td>5</td> 
   <td>7.0</td> 
   <td>10</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20</td> 
   <td>4.0</td> 
   <td>30</td> 
   <td>NaN</td> 
   <td>10</td> 
   <td>NaN</td> 
   <td>35</td> 
  </tr> 
 </tbody> 
</table>

store_items.fillna(method = 'backfill', axis = 1)

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>bikes</th> 
   <th>glasses</th> 
   <th>pants</th> 
   <th>shirts</th> 
   <th>shoes</th> 
   <th>suits</th> 
   <th>watches</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>store 1</th> 
   <td>20.0</td> 
   <td>30.0</td> 
   <td>30.0</td> 
   <td>15.0</td> 
   <td>8.0</td> 
   <td>45.0</td> 
   <td>35.0</td> 
  </tr> 
  <tr> 
   <th>store 2</th> 
   <td>15.0</td> 
   <td>50.0</td> 
   <td>5.0</td> 
   <td>2.0</td> 
   <td>5.0</td> 
   <td>7.0</td> 
   <td>10.0</td> 
  </tr> 
  <tr> 
   <th>store 3</th> 
   <td>20.0</td> 
   <td>4.0</td> 
   <td>30.0</td> 
   <td>10.0</td> 
   <td>10.0</td> 
   <td>35.0</td> 
   <td>35.0</td> 
  </tr> 
 </tbody> 
</table>

我们会发现，如果我们没有使用合适的方法，NaN前面或者后面并没有值，那么会保持是NaN。而且，.fillna方法并没有修改原本的DataFrame，如果想要在原值修改，那么使用关键字inplace = True。  
  
也可以使用.interpolate(method = ‘linear’,axis)方法线性插入合适的值。但是如果前面没有值的话，还保持是NaN。

## 加载数据 ##

我们可能需要加载数据从各种来源的数据库文件，DataFrame支持加载各种格式的数据库文件，其中最流行的是CSV格式。我们可以使用pd.read\_csv()函数加载CSV文件。

Google_stock = pd.read_csv('./GooG.csv')
    
    print('Google_stock is of type:',type(Google_stock))
    print('Google_stock has shape:',Google_stock.shape)

Google_stock is of type: <class 'pandas.core.frame.DataFrame'>
    Google_stock has shape: (3313, 7)

Google_stock

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>Date</th> 
   <th>Open</th> 
   <th>High</th> 
   <th>Low</th> 
   <th>Close</th> 
   <th>Adj Close</th> 
   <th>Volume</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>0</th> 
   <td>2004-08-19</td> 
   <td>49.676899</td> 
   <td>51.693783</td> 
   <td>47.669952</td> 
   <td>49.845802</td> 
   <td>49.845802</td> 
   <td>44994500</td> 
  </tr> 
  <tr> 
   <th>1</th> 
   <td>2004-08-20</td> 
   <td>50.178635</td> 
   <td>54.187561</td> 
   <td>49.925285</td> 
   <td>53.805050</td> 
   <td>53.805050</td> 
   <td>23005800</td> 
  </tr> 
  <tr> 
   <th>2</th> 
   <td>2004-08-23</td> 
   <td>55.017166</td> 
   <td>56.373344</td> 
   <td>54.172661</td> 
   <td>54.346527</td> 
   <td>54.346527</td> 
   <td>18393200</td> 
  </tr> 
  <tr> 
   <th>3</th> 
   <td>2004-08-24</td> 
   <td>55.260582</td> 
   <td>55.439419</td> 
   <td>51.450363</td> 
   <td>52.096165</td> 
   <td>52.096165</td> 
   <td>15361800</td> 
  </tr> 
  <tr> 
   <th>4</th> 
   <td>2004-08-25</td> 
   <td>52.140873</td> 
   <td>53.651051</td> 
   <td>51.604362</td> 
   <td>52.657513</td> 
   <td>52.657513</td> 
   <td>9257400</td> 
  </tr> 
  <tr> 
   <th>5</th> 
   <td>2004-08-26</td> 
   <td>52.135906</td> 
   <td>53.626213</td> 
   <td>51.991844</td> 
   <td>53.606342</td> 
   <td>53.606342</td> 
   <td>7148200</td> 
  </tr> 
  <tr> 
   <th>6</th> 
   <td>2004-08-27</td> 
   <td>53.700729</td> 
   <td>53.959049</td> 
   <td>52.503513</td> 
   <td>52.732029</td> 
   <td>52.732029</td> 
   <td>6258300</td> 
  </tr> 
  <tr> 
   <th>7</th> 
   <td>2004-08-30</td> 
   <td>52.299839</td> 
   <td>52.404160</td> 
   <td>50.675404</td> 
   <td>50.675404</td> 
   <td>50.675404</td> 
   <td>5235700</td> 
  </tr> 
  <tr> 
   <th>8</th> 
   <td>2004-08-31</td> 
   <td>50.819469</td> 
   <td>51.519913</td> 
   <td>50.749920</td> 
   <td>50.854240</td> 
   <td>50.854240</td> 
   <td>4954800</td> 
  </tr> 
  <tr> 
   <th>9</th> 
   <td>2004-09-01</td> 
   <td>51.018177</td> 
   <td>51.152302</td> 
   <td>49.512966</td> 
   <td>49.801090</td> 
   <td>49.801090</td> 
   <td>9206800</td> 
  </tr> 
  <tr> 
   <th>10</th> 
   <td>2004-09-02</td> 
   <td>49.274517</td> 
   <td>50.854240</td> 
   <td>49.150326</td> 
   <td>50.427021</td> 
   <td>50.427021</td> 
   <td>15232100</td> 
  </tr> 
  <tr> 
   <th>11</th> 
   <td>2004-09-03</td> 
   <td>50.148830</td> 
   <td>50.541279</td> 
   <td>49.339096</td> 
   <td>49.681866</td> 
   <td>49.681866</td> 
   <td>5191000</td> 
  </tr> 
  <tr> 
   <th>12</th> 
   <td>2004-09-07</td> 
   <td>50.178635</td> 
   <td>50.670437</td> 
   <td>49.483158</td> 
   <td>50.461796</td> 
   <td>50.461796</td> 
   <td>5891300</td> 
  </tr> 
  <tr> 
   <th>13</th> 
   <td>2004-09-08</td> 
   <td>50.044510</td> 
   <td>51.182110</td> 
   <td>49.925285</td> 
   <td>50.819469</td> 
   <td>50.819469</td> 
   <td>5023000</td> 
  </tr> 
  <tr> 
   <th>14</th> 
   <td>2004-09-09</td> 
   <td>50.933723</td> 
   <td>51.023144</td> 
   <td>50.173668</td> 
   <td>50.824436</td> 
   <td>50.824436</td> 
   <td>4092100</td> 
  </tr> 
  <tr> 
   <th>15</th> 
   <td>2004-09-10</td> 
   <td>50.471729</td> 
   <td>52.935703</td> 
   <td>50.322701</td> 
   <td>52.324677</td> 
   <td>52.324677</td> 
   <td>8764200</td> 
  </tr> 
  <tr> 
   <th>16</th> 
   <td>2004-09-13</td> 
   <td>52.970478</td> 
   <td>53.854729</td> 
   <td>52.886028</td> 
   <td>53.402668</td> 
   <td>53.402668</td> 
   <td>7902900</td> 
  </tr> 
  <tr> 
   <th>17</th> 
   <td>2004-09-14</td> 
   <td>53.377831</td> 
   <td>55.638126</td> 
   <td>53.049961</td> 
   <td>55.384777</td> 
   <td>55.384777</td> 
   <td>10910200</td> 
  </tr> 
  <tr> 
   <th>18</th> 
   <td>2004-09-15</td> 
   <td>54.922779</td> 
   <td>56.745922</td> 
   <td>54.743942</td> 
   <td>55.638126</td> 
   <td>55.638126</td> 
   <td>10793500</td> 
  </tr> 
  <tr> 
   <th>19</th> 
   <td>2004-09-16</td> 
   <td>55.807030</td> 
   <td>57.525848</td> 
   <td>55.464260</td> 
   <td>56.616764</td> 
   <td>56.616764</td> 
   <td>9335800</td> 
  </tr> 
  <tr> 
   <th>20</th> 
   <td>2004-09-17</td> 
   <td>56.840309</td> 
   <td>58.365391</td> 
   <td>56.408119</td> 
   <td>58.365391</td> 
   <td>58.365391</td> 
   <td>9543500</td> 
  </tr> 
  <tr> 
   <th>21</th> 
   <td>2004-09-20</td> 
   <td>58.097134</td> 
   <td>60.407108</td> 
   <td>58.007717</td> 
   <td>59.294346</td> 
   <td>59.294346</td> 
   <td>10708500</td> 
  </tr> 
  <tr> 
   <th>22</th> 
   <td>2004-09-21</td> 
   <td>59.517895</td> 
   <td>59.820923</td> 
   <td>58.375324</td> 
   <td>58.539257</td> 
   <td>58.539257</td> 
   <td>7282900</td> 
  </tr> 
  <tr> 
   <th>23</th> 
   <td>2004-09-22</td> 
   <td>58.320679</td> 
   <td>59.448345</td> 
   <td>58.027588</td> 
   <td>58.807514</td> 
   <td>58.807514</td> 
   <td>7638000</td> 
  </tr> 
  <tr> 
   <th>24</th> 
   <td>2004-09-23</td> 
   <td>59.036026</td> 
   <td>60.918781</td> 
   <td>58.131908</td> 
   <td>60.019630</td> 
   <td>60.019630</td> 
   <td>8599600</td> 
  </tr> 
  <tr> 
   <th>25</th> 
   <td>2004-09-24</td> 
   <td>60.079243</td> 
   <td>61.649033</td> 
   <td>59.493053</td> 
   <td>59.527828</td> 
   <td>59.527828</td> 
   <td>9191900</td> 
  </tr> 
  <tr> 
   <th>26</th> 
   <td>2004-09-27</td> 
   <td>59.393700</td> 
   <td>60.049435</td> 
   <td>58.519386</td> 
   <td>58.747902</td> 
   <td>58.747902</td> 
   <td>7119100</td> 
  </tr> 
  <tr> 
   <th>27</th> 
   <td>2004-09-28</td> 
   <td>60.258080</td> 
   <td>63.288372</td> 
   <td>59.716602</td> 
   <td>63.020115</td> 
   <td>63.020115</td> 
   <td>17056100</td> 
  </tr> 
  <tr> 
   <th>28</th> 
   <td>2004-09-29</td> 
   <td>62.940632</td> 
   <td>67.073753</td> 
   <td>62.707150</td> 
   <td>65.116478</td> 
   <td>65.116478</td> 
   <td>30745600</td> 
  </tr> 
  <tr> 
   <th>29</th> 
   <td>2004-09-30</td> 
   <td>64.530296</td> 
   <td>65.722542</td> 
   <td>64.083199</td> 
   <td>64.381264</td> 
   <td>64.381264</td> 
   <td>13861300</td> 
  </tr> 
  <tr> 
   <th>...</th> 
   <td>...</td> 
   <td>...</td> 
   <td>...</td> 
   <td>...</td> 
   <td>...</td> 
   <td>...</td> 
   <td>...</td> 
  </tr> 
  <tr> 
   <th>3283</th> 
   <td>2017-09-01</td> 
   <td>941.130005</td> 
   <td>942.479980</td> 
   <td>935.150024</td> 
   <td>937.340027</td> 
   <td>937.340027</td> 
   <td>947400</td> 
  </tr> 
  <tr> 
   <th>3284</th> 
   <td>2017-09-05</td> 
   <td>933.080017</td> 
   <td>937.000000</td> 
   <td>921.960022</td> 
   <td>928.450012</td> 
   <td>928.450012</td> 
   <td>1326400</td> 
  </tr> 
  <tr> 
   <th>3285</th> 
   <td>2017-09-06</td> 
   <td>930.150024</td> 
   <td>930.914978</td> 
   <td>919.270020</td> 
   <td>927.809998</td> 
   <td>927.809998</td> 
   <td>1527700</td> 
  </tr> 
  <tr> 
   <th>3286</th> 
   <td>2017-09-07</td> 
   <td>931.729980</td> 
   <td>936.409973</td> 
   <td>923.619995</td> 
   <td>935.950012</td> 
   <td>935.950012</td> 
   <td>1212700</td> 
  </tr> 
  <tr> 
   <th>3287</th> 
   <td>2017-09-08</td> 
   <td>936.489990</td> 
   <td>936.989990</td> 
   <td>924.880005</td> 
   <td>926.500000</td> 
   <td>926.500000</td> 
   <td>1011500</td> 
  </tr> 
  <tr> 
   <th>3288</th> 
   <td>2017-09-11</td> 
   <td>934.250000</td> 
   <td>938.380005</td> 
   <td>926.919983</td> 
   <td>929.080017</td> 
   <td>929.080017</td> 
   <td>1267000</td> 
  </tr> 
  <tr> 
   <th>3289</th> 
   <td>2017-09-12</td> 
   <td>932.590027</td> 
   <td>933.479980</td> 
   <td>923.861023</td> 
   <td>932.070007</td> 
   <td>932.070007</td> 
   <td>1134400</td> 
  </tr> 
  <tr> 
   <th>3290</th> 
   <td>2017-09-13</td> 
   <td>930.659973</td> 
   <td>937.250000</td> 
   <td>929.859985</td> 
   <td>935.090027</td> 
   <td>935.090027</td> 
   <td>1102600</td> 
  </tr> 
  <tr> 
   <th>3291</th> 
   <td>2017-09-14</td> 
   <td>931.250000</td> 
   <td>932.770020</td> 
   <td>924.000000</td> 
   <td>925.109985</td> 
   <td>925.109985</td> 
   <td>1397600</td> 
  </tr> 
  <tr> 
   <th>3292</th> 
   <td>2017-09-15</td> 
   <td>924.659973</td> 
   <td>926.489990</td> 
   <td>916.359985</td> 
   <td>920.289978</td> 
   <td>920.289978</td> 
   <td>2505400</td> 
  </tr> 
  <tr> 
   <th>3293</th> 
   <td>2017-09-18</td> 
   <td>920.010010</td> 
   <td>922.080017</td> 
   <td>910.599976</td> 
   <td>915.000000</td> 
   <td>915.000000</td> 
   <td>1306900</td> 
  </tr> 
  <tr> 
   <th>3294</th> 
   <td>2017-09-19</td> 
   <td>917.419983</td> 
   <td>922.419983</td> 
   <td>912.549988</td> 
   <td>921.809998</td> 
   <td>921.809998</td> 
   <td>936700</td> 
  </tr> 
  <tr> 
   <th>3295</th> 
   <td>2017-09-20</td> 
   <td>922.979980</td> 
   <td>933.880005</td> 
   <td>922.000000</td> 
   <td>931.580017</td> 
   <td>931.580017</td> 
   <td>1669800</td> 
  </tr> 
  <tr> 
   <th>3296</th> 
   <td>2017-09-21</td> 
   <td>933.000000</td> 
   <td>936.530029</td> 
   <td>923.830017</td> 
   <td>932.450012</td> 
   <td>932.450012</td> 
   <td>1290600</td> 
  </tr> 
  <tr> 
   <th>3297</th> 
   <td>2017-09-22</td> 
   <td>927.750000</td> 
   <td>934.729980</td> 
   <td>926.479980</td> 
   <td>928.530029</td> 
   <td>928.530029</td> 
   <td>1052700</td> 
  </tr> 
  <tr> 
   <th>3298</th> 
   <td>2017-09-25</td> 
   <td>925.450012</td> 
   <td>926.400024</td> 
   <td>909.700012</td> 
   <td>920.969971</td> 
   <td>920.969971</td> 
   <td>1856800</td> 
  </tr> 
  <tr> 
   <th>3299</th> 
   <td>2017-09-26</td> 
   <td>923.719971</td> 
   <td>930.820007</td> 
   <td>921.140015</td> 
   <td>924.859985</td> 
   <td>924.859985</td> 
   <td>1666900</td> 
  </tr> 
  <tr> 
   <th>3300</th> 
   <td>2017-09-27</td> 
   <td>927.739990</td> 
   <td>949.900024</td> 
   <td>927.739990</td> 
   <td>944.489990</td> 
   <td>944.489990</td> 
   <td>2239400</td> 
  </tr> 
  <tr> 
   <th>3301</th> 
   <td>2017-09-28</td> 
   <td>941.359985</td> 
   <td>950.690002</td> 
   <td>940.549988</td> 
   <td>949.500000</td> 
   <td>949.500000</td> 
   <td>1020300</td> 
  </tr> 
  <tr> 
   <th>3302</th> 
   <td>2017-09-29</td> 
   <td>952.000000</td> 
   <td>959.786011</td> 
   <td>951.510010</td> 
   <td>959.109985</td> 
   <td>959.109985</td> 
   <td>1581000</td> 
  </tr> 
  <tr> 
   <th>3303</th> 
   <td>2017-10-02</td> 
   <td>959.979980</td> 
   <td>962.539978</td> 
   <td>947.840027</td> 
   <td>953.270020</td> 
   <td>953.270020</td> 
   <td>1283400</td> 
  </tr> 
  <tr> 
   <th>3304</th> 
   <td>2017-10-03</td> 
   <td>954.000000</td> 
   <td>958.000000</td> 
   <td>949.140015</td> 
   <td>957.789978</td> 
   <td>957.789978</td> 
   <td>888300</td> 
  </tr> 
  <tr> 
   <th>3305</th> 
   <td>2017-10-04</td> 
   <td>957.000000</td> 
   <td>960.390015</td> 
   <td>950.690002</td> 
   <td>951.679993</td> 
   <td>951.679993</td> 
   <td>952400</td> 
  </tr> 
  <tr> 
   <th>3306</th> 
   <td>2017-10-05</td> 
   <td>955.489990</td> 
   <td>970.909973</td> 
   <td>955.179993</td> 
   <td>969.960022</td> 
   <td>969.960022</td> 
   <td>1213800</td> 
  </tr> 
  <tr> 
   <th>3307</th> 
   <td>2017-10-06</td> 
   <td>966.700012</td> 
   <td>979.460022</td> 
   <td>963.359985</td> 
   <td>978.890015</td> 
   <td>978.890015</td> 
   <td>1173900</td> 
  </tr> 
  <tr> 
   <th>3308</th> 
   <td>2017-10-09</td> 
   <td>980.000000</td> 
   <td>985.424988</td> 
   <td>976.109985</td> 
   <td>977.000000</td> 
   <td>977.000000</td> 
   <td>891400</td> 
  </tr> 
  <tr> 
   <th>3309</th> 
   <td>2017-10-10</td> 
   <td>980.000000</td> 
   <td>981.570007</td> 
   <td>966.080017</td> 
   <td>972.599976</td> 
   <td>972.599976</td> 
   <td>968400</td> 
  </tr> 
  <tr> 
   <th>3310</th> 
   <td>2017-10-11</td> 
   <td>973.719971</td> 
   <td>990.710022</td> 
   <td>972.250000</td> 
   <td>989.250000</td> 
   <td>989.250000</td> 
   <td>1693300</td> 
  </tr> 
  <tr> 
   <th>3311</th> 
   <td>2017-10-12</td> 
   <td>987.450012</td> 
   <td>994.119995</td> 
   <td>985.000000</td> 
   <td>987.830017</td> 
   <td>987.830017</td> 
   <td>1262400</td> 
  </tr> 
  <tr> 
   <th>3312</th> 
   <td>2017-10-13</td> 
   <td>992.000000</td> 
   <td>997.210022</td> 
   <td>989.000000</td> 
   <td>989.679993</td> 
   <td>989.679993</td> 
   <td>1157700</td> 
  </tr> 
 </tbody> 
</table>

3313 rows × 7 columns

我们可以发现pandas自动给DataFrame添加了行标签，还会自动使用CSV中的列标签当作列标签。  
  
我们也可以使用一些有用的函数来查看太大的DataFrame，比如说.head()和.tail()。

Google_stock.head(5)

Google_stock.tail(5)

我们可以使用.isnull和.any()组合方法来查看是否某列有NaN。

Google_stock.isnull().any()

Date         False
    Open         False
    High         False
    Low          False
    Close        False
    Adj Close    False
    Volume       False
    dtype: bool

我们可以使用.describe()来获得DataFrame每一列的描述性信息。

Google_stock.describe()

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>Open</th> 
   <th>High</th> 
   <th>Low</th> 
   <th>Close</th> 
   <th>Adj Close</th> 
   <th>Volume</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>count</th> 
   <td>3313.000000</td> 
   <td>3313.000000</td> 
   <td>3313.000000</td> 
   <td>3313.000000</td> 
   <td>3313.000000</td> 
   <td>3.313000e+03</td> 
  </tr> 
  <tr> 
   <th>mean</th> 
   <td>380.186092</td> 
   <td>383.493740</td> 
   <td>376.519309</td> 
   <td>380.072458</td> 
   <td>380.072458</td> 
   <td>8.038476e+06</td> 
  </tr> 
  <tr> 
   <th>std</th> 
   <td>223.818650</td> 
   <td>224.974534</td> 
   <td>222.473232</td> 
   <td>223.853780</td> 
   <td>223.853780</td> 
   <td>8.399521e+06</td> 
  </tr> 
  <tr> 
   <th>min</th> 
   <td>49.274517</td> 
   <td>50.541279</td> 
   <td>47.669952</td> 
   <td>49.681866</td> 
   <td>49.681866</td> 
   <td>7.900000e+03</td> 
  </tr> 
  <tr> 
   <th>25%</th> 
   <td>226.556473</td> 
   <td>228.394516</td> 
   <td>224.003082</td> 
   <td>226.407440</td> 
   <td>226.407440</td> 
   <td>2.584900e+06</td> 
  </tr> 
  <tr> 
   <th>50%</th> 
   <td>293.312286</td> 
   <td>295.433502</td> 
   <td>289.929291</td> 
   <td>293.029114</td> 
   <td>293.029114</td> 
   <td>5.281300e+06</td> 
  </tr> 
  <tr> 
   <th>75%</th> 
   <td>536.650024</td> 
   <td>540.000000</td> 
   <td>532.409973</td> 
   <td>536.690002</td> 
   <td>536.690002</td> 
   <td>1.065370e+07</td> 
  </tr> 
  <tr> 
   <th>max</th> 
   <td>992.000000</td> 
   <td>997.210022</td> 
   <td>989.000000</td> 
   <td>989.679993</td> 
   <td>989.679993</td> 
   <td>8.276810e+07</td> 
  </tr> 
 </tbody> 
</table>

我们也可以指定某一列。

Google_stock['Adj Close'].describe()

count    3313.000000
    mean      380.072458
    std       223.853780
    min        49.681866
    25%       226.407440
    50%       293.029114
    75%       536.690002
    max       989.679993
    Name: Adj Close, dtype: float64

当然，我们也可以使用pandas提供的众多统计函数。

print()
    print('Maximum values of each column:\n', Google_stock.max())
    print()
    print('Minimum Close value:', Google_stock['Close'].min())
    print()
    print('Average value of each column:\n', Google_stock.mean())

Maximum values of each column:
     Date         2017-10-13
    Open                992
    High             997.21
    Low                 989
    Close            989.68
    Adj Close        989.68
    Volume         82768100
    dtype: object
    
    Minimum Close value: 49.681866
    
    Average value of each column:
     Open         3.801861e+02
    High         3.834937e+02
    Low          3.765193e+02
    Close        3.800725e+02
    Adj Close    3.800725e+02
    Volume       8.038476e+06
    dtype: float64

另一个重要的指标是统计相关，我们可以使用.corr()函数查看不同列之间的相关性。

Google_stock.corr()

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>Open</th> 
   <th>High</th> 
   <th>Low</th> 
   <th>Close</th> 
   <th>Adj Close</th> 
   <th>Volume</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>Open</th> 
   <td>1.000000</td> 
   <td>0.999904</td> 
   <td>0.999845</td> 
   <td>0.999745</td> 
   <td>0.999745</td> 
   <td>-0.564258</td> 
  </tr> 
  <tr> 
   <th>High</th> 
   <td>0.999904</td> 
   <td>1.000000</td> 
   <td>0.999834</td> 
   <td>0.999868</td> 
   <td>0.999868</td> 
   <td>-0.562749</td> 
  </tr> 
  <tr> 
   <th>Low</th> 
   <td>0.999845</td> 
   <td>0.999834</td> 
   <td>1.000000</td> 
   <td>0.999899</td> 
   <td>0.999899</td> 
   <td>-0.567007</td> 
  </tr> 
  <tr> 
   <th>Close</th> 
   <td>0.999745</td> 
   <td>0.999868</td> 
   <td>0.999899</td> 
   <td>1.000000</td> 
   <td>1.000000</td> 
   <td>-0.564967</td> 
  </tr> 
  <tr> 
   <th>Adj Close</th> 
   <td>0.999745</td> 
   <td>0.999868</td> 
   <td>0.999899</td> 
   <td>1.000000</td> 
   <td>1.000000</td> 
   <td>-0.564967</td> 
  </tr> 
  <tr> 
   <th>Volume</th> 
   <td>-0.564258</td> 
   <td>-0.562749</td> 
   <td>-0.567007</td> 
   <td>-0.564967</td> 
   <td>-0.564967</td> 
   <td>1.000000</td> 
  </tr> 
 </tbody> 
</table>

1代表有很大的相关性，0代表数据根本不相关。

我们可以使用.groupby()函数来使用不同的分组方式分组数据。

data = pd.read_csv('./fake_company.csv')
    
    data

<table> 
 <thead> 
  <tr> 
   <th></th> 
   <th>Year</th> 
   <th>Name</th> 
   <th>Department</th> 
   <th>Age</th> 
   <th>Salary</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <th>0</th> 
   <td>1990</td> 
   <td>Alice</td> 
   <td>HR</td> 
   <td>25</td> 
   <td>50000</td> 
  </tr> 
  <tr> 
   <th>1</th> 
   <td>1990</td> 
   <td>Bob</td> 
   <td>RD</td> 
   <td>30</td> 
   <td>48000</td> 
  </tr> 
  <tr> 
   <th>2</th> 
   <td>1990</td> 
   <td>Charlie</td> 
   <td>Admin</td> 
   <td>45</td> 
   <td>55000</td> 
  </tr> 
  <tr> 
   <th>3</th> 
   <td>1991</td> 
   <td>Alice</td> 
   <td>HR</td> 
   <td>26</td> 
   <td>52000</td> 
  </tr> 
  <tr> 
   <th>4</th> 
   <td>1991</td> 
   <td>Bob</td> 
   <td>RD</td> 
   <td>31</td> 
   <td>50000</td> 
  </tr> 
  <tr> 
   <th>5</th> 
   <td>1991</td> 
   <td>Charlie</td> 
   <td>Admin</td> 
   <td>46</td> 
   <td>60000</td> 
  </tr> 
  <tr> 
   <th>6</th> 
   <td>1992</td> 
   <td>Alice</td> 
   <td>Admin</td> 
   <td>27</td> 
   <td>60000</td> 
  </tr> 
  <tr> 
   <th>7</th> 
   <td>1992</td> 
   <td>Bob</td> 
   <td>RD</td> 
   <td>32</td> 
   <td>52000</td> 
  </tr> 
  <tr> 
   <th>8</th> 
   <td>1992</td> 
   <td>Charlie</td> 
   <td>Admin</td> 
   <td>28</td> 
   <td>62000</td> 
  </tr> 
 </tbody> 
</table>

我们可以使用groupby()方法计算每年公司花多少钱在薪水上。

data.groupby(['Year'])['Salary'].sum()

Year
    1990    153000
    1991    162000
    1992    174000
    Name: Salary, dtype: int64

如果我们想知道每一年的平均薪水。

data.groupby(['Year'])['Salary'].mean()

Year
    1990    51000
    1991    54000
    1992    58000
    Name: Salary, dtype: int64

如果我们想知道，每个部门每年分配多少薪水。

data.groupby(['Year','Department'])['Salary'].sum()

大概就这么多。