我正在研究向量空間模型，數(shù)據(jù)集由 50 個文本文件組成。遍歷它們分解成單詞并將它們保存在字典中?，F(xiàn)在我想使用嵌套字典，如：dictionary = { {someword: {Doc1:23},{Doc21:2},{Doc34:3}},{someword: {Doc1:23},{Doc21:2},{Doc34:3}},{someword: {Doc1:23},{Doc21:2},{Doc34:3}} }但是當(dāng)我運行我的程序時，它不僅會替換文檔，而且不會通過添加“某個詞”在特定文檔中出現(xiàn)的次數(shù)來計算頻率。for iterator in range(1, 51): f = open(directory + str(iterator) + ext, "r") for line in f.read().lower().split(): line = getwords(line) for word in line: if check(word, stopwords) == 0: if existence(word, terms, iterator) != 1: terms[word] = {} terms[word]["Doc"+str(iterator)] = 1 else: terms[word]["Doc"+str(iterator)] = int(terms[word]["Doc"+str(iterator)]) + 1 f.close()存在函數(shù)為：def existence(tok, diction, iteration): if tok in diction: temp = "Doc"+str(iteration) if temp in diction: return 1 else: return 0 else: return 0結(jié)果有點像這樣。{'blunder': {'Doc1': 1}, 'by': {'Doc50': 1}, 'anton': {'Doc27': 1}, 'chekhov': {'Doc27': 1}, 'an': {'Doc50': 1}, 'illustration': {'Doc48': 1}, 'story': {'Doc48': 1}, 'author': {'Doc48': 1}, 'portrait'...
1 回答
收到一只叮咚
TA貢獻1821條經(jīng)驗 獲得超5個贊
您想知道每個單詞在每個文件中出現(xiàn)的次數(shù)嗎？這可以通過 a defaultdictof Counters輕松完成，由 collections 模塊提供。
我認(rèn)為您的想法是正確的，循環(huán)遍歷文件，逐行閱讀并拆分成單詞。這是您需要幫助的計數(shù)部分。
from collections import defaultdict, Counter
from string import punctuation
fnames = ['1.txt', '2.txt', '3.txt', '4.txt', '5.txt']
word_counter = defaultdict(Counter)
for fname in fnames:
with open(fname, 'r') as txt:
for line in txt:
words = line.lower().strip().split()
for word in words:
word = word.strip(punctuation)
if word:
word_counter[word][fname] += 1
里面的數(shù)據(jù)看起來像這樣word_counter：
{
'within': {
'1.txt': 2,
},
'we': {
'1.txt': 3,
'2.txt': 2,
'3.txt': 2,
'4.txt': 2,
'5.txt': 4,
},
'do': {
'1.txt': 7,
'2.txt': 8,
'3.txt': 8,
'4.txt': 6,
'5.txt': 5,
},
...
}
添加回答
舉報
0/150
提交
取消