我的最終目標(biāo)是使用 NER 模型來識別自定義實(shí)體。在此之前，我將文本數(shù)據(jù)標(biāo)記為單詞和句子。我有一個(gè)文本文件（.txt）文件夾，我使用操作系統(tǒng)庫打開并讀入 Jupyter。讀取文本文件后，每當(dāng)我嘗試標(biāo)記文本文件時(shí)，都會收到類型錯(cuò)誤。請告訴我我做錯(cuò)了什么？我的代碼如下，謝謝。import osoutfile = open('result.txt', 'w')path = "C:/Users/okeke/Documents/Work flow/IT Text analytics Project/Extract/Dubuque_text-nlp"files = os.listdir(path)for file in files: outfile.write(str(os.stat(path + "/" + file).st_size) + '\n')outfile.close()這段代碼運(yùn)行良好，每當(dāng)我運(yùn)行輸出文件時(shí)，我都會在下面得到這個(gè)outfile<_io.TextIOWrapper name='result.txt' mode='w' encoding='cp1252'>接下來，標(biāo)記化。from nltk.tokenize import sent_tokenize, word_tokenize sent_tokens = sent_tokenize(outfile)print(outfile)word_tokens = word_tokenize(outfile)print(outfile但運(yùn)行上面的代碼后出現(xiàn)錯(cuò)誤。檢查下面是否有錯(cuò)誤---------------------------------------------------------------------------TypeError Traceback (most recent call last)<ipython-input-22-62f66183895a> in <module> 1 from nltk.tokenize import sent_tokenize, word_tokenize----> 2 sent_tokens = sent_tokenize(outfile) 3 print(outfile) 4 5 #word_tokens = word_tokenize(text)~\AppData\Local\Continuum\anaconda3\envs\nlp_course\lib\site-packages\nltk\tokenize\__init__.py in sent_tokenize(text, language) 93 """ 94 tokenizer = load('tokenizers/punkt/{0}.pickle'.format(language))---> 95 return tokenizer.tokenize(text) 96 97 # Standard word tokenizer.TypeError: expected string or bytes-like object
1 回答
不負(fù)相思意
TA貢獻(xiàn)1777條經(jīng)驗(yàn) 獲得超10個(gè)贊
（移動評論來回答）
您正在嘗試處理文件對象而不是文件中的文本。創(chuàng)建文本文件后，重新打開它并在標(biāo)記化之前讀取整個(gè)文件。
試試這個(gè)代碼：
import os
outfile = open('result.txt', 'w')
path = "C:/Users/okeke/Documents/Work flow/IT Text analytics Project/Extract/Dubuque_text-nlp"
files = os.listdir(path)
for file in files:
with open(path + "/" + file) as f:
outfile.write(f.read() + '\n')
#outfile.write(str(os.stat(path + "/" + file).st_size) + '\n')
outfile.close() # done writing
from nltk.tokenize import sent_tokenize, word_tokenize
with open('result.txt') as outfile: # open for read
alltext = outfile.read() # read entire file
print(alltext)
sent_tokens = sent_tokenize(alltext) # process file text. tokenize sentences
word_tokens = word_tokenize(alltext) # process file text. tokenize words
添加回答
舉報(bào)
0/150
提交
取消