Notes—Random Forest-feature importance随机森林对特征排序

迈不过友情╰ 2022-06-13 07:46 162阅读 0赞

……未完待补充……  
ref:[http://blog.datadive.net/selecting-good-features-part-iii-random-forests/][http_blog.datadive.net_selecting-good-features-part-iii-random-forests]

**two methods:**  
**1.Mean decrease impurity**  
大概是对于每颗树，按照impurity（gini /entropy /information gain）给特征排序，然后整个森林取平均

**2.Mean decrease accuracy**  
大概就是measure一下对每个特征加躁，看对结果的准确率的影响。影响小说明这个特征不重要，反之重要  
具体步骤如下：  
在随机森林中某个特征X的重要性的计算方法如下：  
1：对于随机森林中的每一颗决策树,使用相应的OOB(袋外数据)数据来计算它的袋外数据误差,记为errOOB1.  
2: 随机地对袋外数据OOB所有样本的特征X加入噪声干扰(就可以随机的改变样本在特征X处的值),再次计算它的袋外数据误差,记为errOOB2.  
3：假设随机森林中有Ntree棵树,那么对于特征X的重要性=∑(errOOB2-errOOB1)/Ntree,之所以可以用这个表达式来作为相应特征的重要性的度量值是因为：若给某个特征随机加入噪声之后,袋外的准确率大幅度降低,则说明这个特征对于样本的分类结果影响很大,也就是说它的重要程度比较高。  
ref：[https://www.cnblogs.com/justcxtoworld/p/3447231.html][https_www.cnblogs.com_justcxtoworld_p_3447231.html]  
[http://www.cnblogs.com/justcxtoworld/p/3434266.html][http_www.cnblogs.com_justcxtoworld_p_3434266.html]

**1.Mean decrease impurity**  
sklearn中实现如下：

from sklearn.datasets import load_boston
    from sklearn.ensemble import RandomForestRegressor
    import numpy as np
    #Load boston housing dataset as an example
    boston = load_boston()
    X = boston["data"]
    print type(X),X.shape
    Y = boston["target"]
    names = boston["feature_names"]
    print names
    rf = RandomForestRegressor()
    rf.fit(X, Y)
    print "Features sorted by their score:"
    print sorted(zip(map(lambda x: round(x, 4), rf.feature_importances_), names), reverse=True)

结果如下：

Features sorted by their score:
    [(0.5104, 'RM'), (0.2837, 'LSTAT'), (0.0812, 'DIS'), (0.0303, 'CRIM'), (0.0294, 'NOX'), (0.0176, 'PTRATIO'), (0.0134, 'AGE'), (0.0115, 'B'), (0.0089, 'TAX'), (0.0077, 'INDUS'), (0.0051, 'RAD'), (0.0006, 'ZN'), (0.0004, 'CHAS')]

**2.Mean decrease accuracy**  
sklearn中并没有……具体实现见链接

**3.Spark中的feature importance使用**  
ml.classification.RandomForestClassificationModel中有featureImportance可以直接调用  
定义如下：  
lazy val featureImportances:Vector  
调用：  
Model.featureImportances.toArray.mkString(“,”)

[http_blog.datadive.net_selecting-good-features-part-iii-random-forests]: http://blog.datadive.net/selecting-good-features-part-iii-random-forests/
[https_www.cnblogs.com_justcxtoworld_p_3447231.html]: https://www.cnblogs.com/justcxtoworld/p/3447231.html
[http_www.cnblogs.com_justcxtoworld_p_3434266.html]: http://www.cnblogs.com/justcxtoworld/p/3434266.html