我嘗試創(chuàng)建一個程序來提取 Spotify 中 Daily Mix 1 中的所有歌曲。我知道我必須使用的邏輯，但我無法獲得整個源代碼。這是我寫的代碼：import requests from bs4 import BeautifulSoupheaders = {"User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36"}result = requests.get("https://open.spotify.com/playlist/37i9dQZF1E38L6D2gtQHWw", headers=headers)src = result.contentsoup = BeautifulSoup(src, 'lxml')print(soup.prettify())這是我得到的輸出：我使用的標(biāo)題適用于亞馬遜和維基百科等其他網(wǎng)站，所以我認(rèn)為這不是問題。我也不認(rèn)為問題與 javascript 有關(guān)，因為在其他用于抓取網(wǎng)站（例如亞馬遜（也包含很多<script>標(biāo)簽））的程序中，代碼顯示得非常好。請告訴問題是什么。PS - 請不要在您的解決方案中推薦 selenium 或 scrapy。
1 回答
隔江千里
TA貢獻(xiàn)1906條經(jīng)驗 獲得超10個贊
您嘗試抓取的日期是由 Javascript 填充的，因此您不會在頁面的源代碼中找到它，但您可以通過網(wǎng)站正在使用的 api 獲取它：
import json , requests
from bs4 import BeautifulSoup as bs
base_url = 'https://open.spotify.com/playlist/37i9dQZF1E38L6D2gtQHWw'
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36"}
# Getting the access token first to send it with the header to the api endpoint
page = requests.get(base_url,headers=headers)
soup = bs(page.text,'html.parser')
access_token_tag = soup.find('script',{'id':'config'})
json_obj = json.loads(access_token_tag.text)
access_token_text = json_obj['accessToken']
endpoint = "https://api.spotify.com/v1/playlists/37i9dQZF1E38L6D2gtQHWw"
headers.update({"authorization": f"Bearer {access_token_text}",
'referer': base_url,
'accept': 'application/json',
'app-platform': 'WebPlayer'})
url_paramters = {'type': 'track,episode','market': 'EG'}
data = requests.get(endpoint, params=url_paramters, headers=headers).json()
tracks = data['tracks']['items']
for index , track in enumerate(tracks,1):
print(f'{index } - ' , track['track']['name'] )
輸出：
1 - Tu Hi Haqeeqat
2 - Hasi - Female Version
3 - Kabhi Jo Baadal Barse
4 - Tere Bin Nahi Laage (Male Version)
5 - Dekhte Dekhte (Rahat Fateh Ali Khan Version) [From "Batti Gul Meter
Chalu"]
6 - Panchhi Bole
7 - Jame Raho
8 - Banjaara (From "Ek Villain")
9 - Mitwa
10 - Agar Tu Hota (From "Baaghi")
11 - Aasan Nahin Yahan
12 - Jiyo Re Bahubali
13 - Pyaar Manga Hai
14 - Kaun Hain Voh
15 - Mamta Se Bhari
16 - Zehnaseeb
17 - Dil Ibaadat
18 - Tu Hi Tu (Reprise)
19 - Haule Haule
20 - Manohari
21 - Ilahi (From "Yeh Jawaani Hai Deewani")
22 - Humsafar (From "Badrinath Ki Dulhania")
23 - Kiya Kiya
24 - Sunn Raha Hai (Female)
25 - Phir Le Aya Dil
26 - Tere Naal Nachna (From "Nawabzaade")
27 - Galliyan (From "Ek Villain")
28 - Valentine's Mashup 2019(Remix By Kedrock,Sd Style)
29 - Halka Halka
30 - Raabta (From "Agent Vinod")
31 - Mere Bina - Unplugged
32 - Agar Tum Saath Ho-Maahi Ve
33 - Swapn Sunehere
34 - Radha
35 - Behti Hawa Sa Tha Woh
36 - Mere Rashke Qamar
37 - Kehta Hai Pal Pal
38 - Maana Ke Hum Yaar Nahin
39 - Khoya Hain
40 - O Re Piya
41 - Jal Rahin Hain
42 - Zero Hour Mashup 2015(Remix By Dj Kiran Kamath)
43 - Aashiq Banaya Aapne
44 - Bikhri Bikhri
45 - Maula Mere Lele Meri Jaan
46 - Yadaan Teriyaan (Version 2)
47 - Tujh Mein Rab Dikhta Hai
48 - Veeron Ke Veer Aa
49 - Bolo Har Har Har (feat. Mohit Chauhan, Sukhwinder Singh, Badshah, Megha Sriram Dalton, Anugrah, Sandeep Shrivastava)
50 - Main Agar
- 1 回答
- 0 關(guān)注
- 184 瀏覽
添加回答
舉報
0/150
提交
取消