RuntimeWarning: divide by zero encountered in log错误解决

╰+哭是因爲堅強的太久メ 2023-09-28 13:50 11阅读 0赞

### 问题描述 ###

最近在学习《机器学习实战》这本书时，朴素贝叶斯那里遇到了这样的问题。

def train_native_bayes(train_matrix,train_category):
        num_train_docs=len(train_matrix)
        num_words=len(train_matrix[0])
        p=sum(train_category)/float(num_train_docs)
        p_0_num=zeros(num_words)
        p_1_num=zeros(num_words)
        p_0_denom=0.0
        p_1_denom=0.0
        for i in range(num_train_docs):
            if train_category[i]==1:
                p_1_num+=train_matrix[i]
                p_1_denom+=sum(train_matrix[i])
            else:
                p_0_num+=train_matrix[i]
                p_0_denom+=sum(train_matrix[i])
        p_1_vector=log(p_1_num/p_1_denom)
        p_0_vector=log(p_0_num/p_0_denom)
        return p_0_vector,p_1_vector,p

然后运行时出现了下面的问题：

F:\PycharmProject\bayes_practice_1.py:74: RuntimeWarning: divide by zero encountered in log
      p_1_vector=log(p_1_num/p_1_denom)
    F:\PycharmProject\bayes_practice_1.py:75: RuntimeWarning: divide by zero encountered in log
      p_0_vector=log(p_0_num/p_0_denom)
    F:\PycharmProject\bayes_practice_1.py:84: RuntimeWarning: invalid value encountered in multiply
      p_1 = sum(need_to_classify_vector * p_1_vector) + log(p_class)    #element-wise mult
    F:\PycharmProject\bayes_practice_1.py:85: RuntimeWarning: invalid value encountered in multiply
      p_0 = sum(need_to_classify_vector * p_0_vector) + log(1.0 - p_class)

虽然不影响最终的结果，但是警告看起来让人不舒服。  
我们排查原因，是存在数字太小的原因，溢出，计算过程中出现-inf，再做其他运算，结果还是-inf。  
比如我们展示一下结果：

train_mat=[]
    for i in dataset:
        train_mat.append(set_of_words_vector(my_vacab_set,i))
    p_0_vector,p_1_vector,p=train_native_bayes(train_mat,class_vector)
    print(p_0_vector)

结果如下：

[-3.17805383 -3.17805383 -3.17805383        -inf -3.17805383 -2.48490665
     -3.17805383 -3.17805383        -inf -3.17805383 -3.17805383 -3.17805383
            -inf        -inf        -inf        -inf -3.17805383        -inf
     -3.17805383 -3.17805383        -inf        -inf -3.17805383 -2.07944154
     -3.17805383 -3.17805383        -inf -3.17805383 -3.17805383        -inf
     -3.17805383 -3.17805383]

### 探索原因 ###

当概率很小时，取对数后结果趋于负无穷大。

### 解决方法 ###

我们改变浮点数的精度为1e-5

p_1_vector=log(p_1_num/p_1_denom+1e-5)
    p_0_vector=log(p_0_num/p_0_denom+1e-5)

这样就不会再报错，结果也没有-inf了。

[ -3.17781386  -3.17781386  -3.17781386  -3.17781386  -3.17781386
      -3.17781386  -3.17781386  -3.17781386  -2.07936154  -3.17781386
     -11.51292546  -3.17781386 -11.51292546 -11.51292546  -3.17781386
      -3.17781386 -11.51292546  -3.17781386  -3.17781386 -11.51292546
     -11.51292546 -11.51292546 -11.51292546 -11.51292546  -3.17781386
      -3.17781386 -11.51292546  -2.48478666  -3.17781386  -3.17781386
     -11.51292546  -3.17781386]