我正在嘗試從電子商務(wù)網(wǎng)站上抓取頂級(jí)產(chǎn)品的名稱列表。然而結(jié)果是空的。想知道缺少什么。輸出為： 訪問： https ://www.amazon.in/gp/bestsellers/electronics/ref=zg_bs_nav_0/ 抓取結(jié)束： https ://www.amazon.in/gp/bestsellers/electronics/ref=zg_bs_nav_0/代碼：package mainimport ( "encoding/csv" "fmt" "log" "os" "github.com/gocolly/colly")func main() { fetchURL := "https://www.amazon.in/gp/bestsellers/electronics/ref=zg_bs_nav_0/" fileName := "results.csv" file, err := os.Create(fileName) if err != nil { log.Fatal("ERROR: Could not create file %q: %s\n", fileName, err) return } defer file.Close() writer := csv.NewWriter(file) defer writer.Flush() writer.Write([]string{"Sl. No."}) c := colly.NewCollector() c.OnRequest(func(r *colly.Request) { fmt.Println("Visiting: ", r.URL) }) c.OnHTML(`.a-section a-spacing-none aok-relative`, func(e *colly.HTMLElement) { number := e.ChildText(".zg-badge-text") name := e.ChildText(".p13n-sc-truncated") writer.Write([]string{ number, name, }) c.Visit(fetchURL) fmt.Println("End of scraping: ", fetchURL)}
1 回答
慕哥6287543
TA貢獻(xiàn)1831條經(jīng)驗(yàn) 獲得超10個(gè)贊
您需要添加 User-Agent 標(biāo)頭才能返回?cái)?shù)據(jù)。它似乎p13n-sc-truncated也是一個(gè)生成的類名。您可以使用以下示例：
package main
import (
"log"
"strings"
"github.com/gocolly/colly"
)
type AmazonData struct {
Index int
Link string
Title string
}
func main() {
c := colly.NewCollector()
var data []AmazonData
count := 1
c.OnHTML(`#zg-ordered-list`, func(e *colly.HTMLElement) {
e.ForEach("li .zg-item", func(_ int, elem *colly.HTMLElement) {
link := elem.DOM.Find("a")
linkHref, _ := link.Attr("href")
data = append(data, AmazonData{
Index: count,
Link: linkHref,
Title: strings.TrimSpace(link.Find("div").Text()),
})
count++
})
log.Println(data)
})
c.OnRequest(func(r *colly.Request) {
r.Headers.Set("User-Agent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36")
})
c.Visit("https://www.amazon.in/gp/bestsellers/electronics/ref=zg_bs_nav_0/")
}
- 1 回答
- 0 關(guān)注
- 126 瀏覽
添加回答
舉報(bào)
0/150
提交
取消