所以我試圖從 Steam 上的免費游戲網(wǎng)站獲取所有網(wǎng)址，但它一直返回空。我不知道我在這里做錯了什么，下圖顯示了路徑result = requests.get("https://steamdb.info/upcoming/free/")src = result.contentsoup = BeautifulSoup(src, 'lxml')urls = []for td_tag in soup.find_all('td'): a_tag = td_tag.find('a') urls.append(a_tag.attrs['href'])print(urls)
1 回答
BIG陽
TA貢獻1859條經(jīng)驗 獲得超6個贊
您必須使用標(biāo)頭User-Agent，它不能很短Mozilla/5.0，但是來自真實網(wǎng)絡(luò)瀏覽器的完整字符串
import requests
from bs4 import BeautifulSoup
headers = {
"User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0",
}
result = requests.get("https://steamdb.info/upcoming/free/", headers=headers)
soup = BeautifulSoup(result.content, 'lxml')
#print(result.content)
urls = []
for td_tag in soup.find_all('td'):
a_tag = td_tag.find('a')
if a_tag:
urls.append(a_tag.attrs['href'])
print(urls)
添加回答
舉報
0/150
提交
取消