数据清洗练习问题汇总

ゞ浴缸里的玫瑰 2023-07-22 14:28 2阅读 0赞

## 1介绍 ##

源数据集包括四个文件：  
![在这里插入图片描述][20200406113028751.png]  
其中第一个压缩文件解压后是tsv格式文件  
labeledTrainData - The labeled training set. The file is tab-delimited and has a header row followed by 25,000 rows containing an id, sentiment, and text for each review.  
testData - The test set. The tab-delimited file has a header row followed by 25,000 rows containing an id and text for each review. Your task is to predict the sentiment for each one.  
unlabeledTrainData - An extra training set with no labels. The tab-delimited file has a header row followed by 50,000 rows containing an id and text for each review.  
sampleSubmission - A comma-delimited sample submission file in the correct format.

## 数据导入：OSError: Initializing from file failed ##

问题原因：复制文件路径的时候（win10）从地址栏直接复制过去的，只包含文件夹名不包含文件名

## Error tokenizing data. C error: Expected 11 fields in line 4, saw 23 ##

问题原因：pd.read\_csv读入，如果只有

train=pd.read_csv('D:\Kaggle/word2vec-nlp-tutorial\labeledTrainData\labeledTrainData.tsv',)

就是错的，但如果加上:

train=pd.read_csv('D:\Kaggle/word2vec-nlp-tutorial\labeledTrainData\labeledTrainData.tsv',header=0, \
                        delimiter="\t", quoting=3)

就是对的  
\#Here, “header=0” indicates that the first line of the file contains column names,  
“delimiter=\\t” indicates that the fields are separated by tabs,  
and quoting=3 tells Python to ignore doubled quotes, otherwise you may encounter errors trying to read the file.

## 正则表达式 ##

## re.sub（‘a’,‘b’,context） ##

把context中为a的内容替换为b  
r’a’表示rawstring原生字符串，不会转义  
\[\]表示group membership 一组字符  
^表示not

## nltk库及其导入 ##

首先我之前电脑上下载过nltk包  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2t1bkFVR1VTVA_size_16_color_FFFFFF_t_70]  
但是我尝试

from nltk.corpus import stopwords # Import the stop word list
    print(stopwords.words("english"))

却报错：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2t1bkFVR1VTVA_size_16_color_FFFFFF_t_70 1]  
一开始我以为是路径问题，加了之后无效（可能是错了），然后又下载了stopwords库，最后成功了。错误步骤不写了先试试加路径，不行的再下载。

import nltk
    nltk.data.path.append(r"D:\NLP\nlp_data")

import nltk
    nltk.download("stopwords")
    
    from nltk.corpus import stopwords # Import the stop word list
    print(stopwords.words("english"))

## 一个很神奇的路径问题 ##

train=pd.read_csv('D:\K\word2vec-nlp-tutorial\labeledTrainData\labeledTrainData.tsv',header=0, \
                        delimiter="\t", quoting=3)
    test = pd.read_csv("D:/K/word2vec-nlp-tutorial/testData/testData.tsv", header=0, delimiter="\t", \
                       quoting=3 )

第一次导入的时候用的是win10默认的路径符号  
OK  
第二次再用默认的不行了。。。改成/就成了。。。迷幻

[20200406113028751.png]: /images/20230528/aeb8c8926f1a47a2a831389091e4f799.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2t1bkFVR1VTVA_size_16_color_FFFFFF_t_70]: /images/20230528/373d80858b1041c3b420b0fa20f39d8c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2t1bkFVR1VTVA_size_16_color_FFFFFF_t_70 1]: /images/20230528/936f2fa46d9e4006be8aa5697600fabf.png