假設(shè)我有一個句子列表(在一個大型語料庫中)作為標(biāo)記詞的集合。樣本格式如下:tokenized_raw_data 的格式如下:[['arxiv', ':', 'astro-ph/9505066', '.'], ['seds', 'page', 'on', '``', 'globular', 'star', 'clusters', "''", 'douglas', 'scott', '``', 'independent', 'age', 'estimates', "''", 'krysstal', '``', 'the', 'scale', 'of', 'the', 'universe', "''", 'space', 'and', 'time', 'scaled', 'for', 'the', 'beginner', '.'], ['icosmos', ':', 'cosmology', 'calculator', '(', 'with', 'graph', 'generation', ')', 'the', 'expanding', 'universe', '(', 'american', 'institute', 'of', 'physics', ')']]我想申請pos_tag.到目前為止,我嘗試過的內(nèi)容如下。import os, nltk, refrom nltk.corpus import stopwordsfrom unidecode import unidecodefrom nltk.tokenize import word_tokenize, sent_tokenizefrom nltk.tag import pos_tagdef read_data(): global tokenized_raw_data with open("path//merge_text_results_pu.txt", 'r', encoding='utf-8', errors = 'replace') as f: raw_data = f.read() tokenized_raw_data = '\n'.join(nltk.line_tokenize(raw_data))read_data()def function1(): tokens_sentences = sent_tokenize(tokenized_raw_data.lower()) unfiltered_tokens = [[word for word in word_tokenize(word)] for word in tokens_sentences] tagged_tokens = nltk.pos_tag(unfiltered_tokens) nouns = [word.encode('utf-8') for word,pos in tagged_tokens if (pos == 'NN' or pos == 'NNP' or pos == 'NNS' or pos == 'NNPS')] joined_nouns_text = (' '.join(map(bytes.decode, nouns))).strip() noun_tokens = [t for t in wordpunct_tokenize(joined_nouns_text)] stop_words = set(stopwords.words("english"))function1()我收到以下錯誤。> AttributeError: 'list' object has no attribute 'isdigit'請幫助如何以高效的方式克服這個錯誤?我哪里出錯了?注意:我在 Windows 10 上使用 Python 3.7。
1 回答

慕田峪7331174
TA貢獻(xiàn)1828條經(jīng)驗 獲得超13個贊
嘗試這個-
word_list=[]
for i in range(len(unfiltered_tokens)):
word_list.append([])
for i in range(len(unfiltered_tokens)):
for word in unfiltered_tokens[i]:
if word[1:].isalpha():
word_list[i].append(word[1:])
然后在做之后
tagged_tokens=[]
for token in word_list:
tagged_tokens.append(nltk.pos_tag(token))
你會得到你想要的結(jié)果!希望這有幫助。
添加回答
舉報
0/150
提交
取消