我的 python 代碼不起作用。它應(yīng)該可以工作，但我不知道為什么......它只是給了 []url = "https://www.naver.com/"req = urllib.request.urlopen(url)res = req.read()soup = BeautifulSoup(res, 'html.parser')keywords = soup.find_all('span', class_='ah_k')keywords = [each_line.get_text().strip() for each_line in keywords[:20]]print(keywords)
2 回答
炎炎設(shè)計(jì)
TA貢獻(xiàn)1808條經(jīng)驗(yàn) 獲得超4個(gè)贊
如果您轉(zhuǎn)到network Tab下方，XHR您將獲得以下返回的鏈接keywords in json format。您不需要硒來獲取關(guān)鍵字。
https://www.naver.com/srchrank?frm=main&ag=all&gr=1&ma=-2&si=0&en=0&sp=0
代碼：
import requests
url='https://www.naver.com/srchrank?frm=main&ag=all&gr=1&ma=-2&si=0&en=0&sp=0'
res=requests.get(url).json()
print(res['data'])
for item in res['data']:
print(item['keyword'])
在控制臺上，第一次打印將返回 json 數(shù)據(jù)。
[{'rank': 1, 'keyword': '???', 'keyword_synonyms': []}, {'rank': 2, 'keyword': '???', 'keyword_synonyms': ['??? ???', '?? ???', '2? ??', '?? ?? 2? ??', '6?? ???', '??', '?????', '5?? ???', '3?? ???', '?? ??? ???', '??? ???? ???', '?? ??? ???? ???', '?????', '?? ???? ???', '?? ????', '?? ???', '??? ??', '??? ???', '???', '?? ?? ?? ???', '????? ???', '?? ?? ???']}, {'rank': 3, 'keyword': '??? ???', 'keyword_synonyms': []}, {'rank': 4, 'keyword': '???', 'keyword_synonyms': []}, {'rank': 5, 'keyword': '???', 'keyword_synonyms': []}, {'rank': 6, 'keyword': '500???', 'keyword_synonyms': []}, {'rank': 7, 'keyword': '2020 ??????', 'keyword_synonyms': []}, {'rank': 8, 'keyword': '????????? ???', 'keyword_synonyms': []}, {'rank': 9, 'keyword': '???', 'keyword_synonyms': []}, {'rank': 10, 'keyword': '???', 'keyword_synonyms': []}, {'rank': 11, 'keyword': 'who ????', 'keyword_synonyms': []}, {'rank': 12, 'keyword': '???', 'keyword_synonyms': []}, {'rank': 13, 'keyword': '???', 'keyword_synonyms': []}, {'rank': 14, 'keyword': '?? clip clop', 'keyword_synonyms': []}, {'rank': 15, 'keyword': '??? ??', 'keyword_synonyms': []}, {'rank': 16, 'keyword': '????', 'keyword_synonyms': []}, {'rank': 17, 'keyword': '??', 'keyword_synonyms': ['?? ??']}, {'rank': 18, 'keyword': '????', 'keyword_synonyms': []}, {'rank': 19, 'keyword': '???', 'keyword_synonyms': []}, {'rank': 20, 'keyword': '???', 'keyword_synonyms': []}]
第二次打印將打印所有關(guān)鍵字。
???
???
??? ???
???
???
500???
2020 ??????
????????? ???
???
???
who ????
???
???
?? clip clop
??? ??
????
??
????
???
???
慕標(biāo)5832272
TA貢獻(xiàn)1966條經(jīng)驗(yàn) 獲得超4個(gè)贊
有一個(gè)解決方案需要安裝 pyppeteer 和 simple_scrapy。
from simplified_scrapy.simplified_doc import SimplifiedDoc
from simplified_html.request_render import RequestRender
# req = RequestRender({ 'executablePath': '/Applications/chrome.app/Contents/MacOS/Google Chrome'})
req = RequestRender({ 'executablePath': 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe'})
def callback(html,url,extr):
doc = SimplifiedDoc(html)
keywords = doc.selects('span.ah_k')
keywords = [each_line.text for each_line in keywords[:20]]
print(keywords)
req.get("https://www.naver.com/", callback,selectorOrFunctionOrTimeout='//span[@class="ah_k"]')
結(jié)果：
['????', '??? ??', '?????3', '???', '500???', '???', '???', '???', 'who ????', '??? ??? ???
?', '?? ???', '??', '?????', '???', '???', '???', '??', '???', '??? ???', '???']
分享
編輯
跟隨
添加回答
舉報(bào)
0/150
提交
取消