我有一個包含 1000 個帶有相應(yīng)關(guān)鍵字的文本文檔的數(shù)據(jù)框。我想通過查找與列表中最相似的文檔對應(yīng)的關(guān)鍵字來提取新文檔的關(guān)鍵字。
1 回答
幕布斯6054654
TA貢獻1876條經(jīng)驗 獲得超7個贊
首先將您的 csv 保存到數(shù)據(jù)幀 df 并使用以下函數(shù)進行余弦相似度計算。def get_cosine(vec1, vec2): intersection = set(vec1.keys()) & set(vec2.keys()) numerator = sum([vec1[x] * vec2[x] for x in intersection])
sum1 = sum([vec1[x]**2 for x in vec1.keys()])
sum2 = sum([vec2[x]**2 for x in vec2.keys()])
denominator = math.sqrt(sum1) * math.sqrt(sum2)
if not denominator:
return 0.0
else:
return float(numerator) / denominator
def text_to_vector(text):
word = re.compile(r'\w+')
words = word.findall(text)
return Counter(words)
def get_result(content_a, content_b):
text1 = content_a
text2 = content_b
vector1 = text_to_vector(text1)
vector2 = text_to_vector(text2)
cosine_result = get_cosine(vector1, vector2)
return cosine_result
然后遍歷 df 并調(diào)用如下函數(shù)：
similarity=[]
for ind in df.index:
#my_doc="new document should go in here"
#prev_doc= "previous document for each index should go in here"
cos=get_result(my_doc, prev_doc)
similarity.append(cos)
max_ind= similarity.index(max(similarity))
您將獲得最相似文檔的索引位置
添加回答
舉報
0/150
提交
取消