我需要從這個亞馬遜頁面上抓取一些信息：https://www.amazon.com/dp/B07Q6H83VY/ref=sspa_dk_detail_6?pd_rd_i=B07Q6H83VY&pd_rd_w=n4cqh&pf_rd_p=48d372c1-f7e1-4b8b-9d02-4bd86f5158c5&pd_rd_wg=8d6Pd&pf_rd_r=AES6X22PPPPREK5DD60G&pd_rd_r=2a4ff4e6-f8ce-4d62-8106-cffd53838b9e&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUEyTTZUQzQ0Q05TOVZJJmVuY3J5cHRlZElkPUEwMDU2MjE0Q05HOUFSMkdQTkhPJmVuY3J5cHRlZEFkSWQ9QTA4NTIyNzAxOVZYM1dISEVBUk1DJndpZGdldE5hbWU9c3BfZGV0YWlsJmFjdGlvbj1jbGlja1JlZGlyZWN0JmRvTm90TG9nQ2xpY2s9dHJ1ZQ&th=1具體來說，我會對這些領(lǐng)域感興趣：Author | Star | Date | Title | Review例如： Gi1.0 out of 5 stars Unacceptable Launch State for PS4Reviewed in the United States on September 14, 2019Platform: PlayStation 4Edition: Super DeluxeVerified Purchase因為我以前從來沒有這樣做過，所以我想知道我是否可以用 Scrapy/BeautifulSoup/Selenium 來做這件事，或者我是否需要一個 API，盡管這些信息來自Author under <span class="a-profile-name">Gi</span>Rating <span class="a-icon-alt">1.0 out of 5 stars</span>Review <div data-hook="review-collapsed" aria-expanded="false" class="a-expander-content a-expander-partial-collapse-content" style="padding-bottom: 19px;"> ...TEXT...</div>
2 回答
慕姐8265434
TA貢獻1813條經(jīng)驗 獲得超2個贊
Scrapy 將是完成此任務(wù)的不錯選擇。這將是一個非常簡單的蜘蛛，它將能夠收集所需的信息。
import scrapy
class TestSpider(scrapy.Spider):
name = 'test'
start_urls = ['https://www.amazon.com/dp/B07Q6H83VY']
def parse(self, response):
for row in response.css('div.review'):
item = {}
item['author'] = row.css('span.a-profile-name::text').extract_first()
rating = row.css('i.review-rating > span::text').extract_first().strip().split(' ')[0]
item['rating'] = int(float(rating.strip().replace(',', '.')))
item['title'] = row.css('span.review-title > span::text').extract_first()
created_date = row.css('span.review-date::text').extract_first().strip()
item['created_date'] = created_date
review_content = row.css('div.reviewText ::text').extract()
review_content = [rc.strip() for rc in review_content if rc.strip()]
item['content'] = ', '.join(review_content)
yield item
輸出示例：
{
"author": "Jhona Diaz",
"rating": 4,
"title": "Recomendable solo si eres fan ya que si está algo caro",
"created_date": "Reviewed in Mexico on November 23, 2019",
"content": "Buena calidad y pues muy completo"
},
{
"author": "MANUEL MENDOZA OLVERA",
"rating": 5,
"title": "Perfecto Estado",
"created_date": "Reviewed in Mexico on September 28, 2019",
"content": "excelente, la edición es de caja metálica y llegó intacta"
},
神不在的星期二
TA貢獻1963條經(jīng)驗 獲得超6個贊
首先做 pip install selenium
第二個使用 Python 庫 dryscrape 來抓取 javascript 驅(qū)動的網(wǎng)站。在這個網(wǎng)址https://phantomjs.org/download.html
from selenium import webdriver
#the path below from dryscrape folder from step2
driver = webdriver.PhantomJS(executable_path='C:\\Users\\nayef\\Desktop\\New folder\\phantomjs-2.1.1-windows\\bin\\phantomjs')
driver.get('https://www.amazon.com/dp/B07Q6H83VY')
p_element = driver.find_element_by_id('deliveryMessageMirId')
driver.get(my_url)
p_element = driver.find_element_by_id(id_='intro-text')
print(p_element.text)
# result:
Arrives: Friday, Aug 7 Details
添加回答
舉報
0/150
提交
取消