python3爬蟲實(shí)例源碼 https://github.com/fifths/python_baike_spider.git
2016-01-03
print('第三種方法')
cj = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
urllib.request.install_opener(opener)
response3 = urllib.request.urlopen(url)
print(response3.getcode())
print(cj)
print(response3.read())
cj = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
urllib.request.install_opener(opener)
response3 = urllib.request.urlopen(url)
print(response3.getcode())
print(cj)
print(response3.read())
2016-01-03
print('第二種方法')
req = urllib.request.Request(url)
req.add_header('user-agent', 'Mozilla/5.0')
response2 = urllib.request.urlopen(req)
print(response2.getcode())
print(len(response2.read()))
req = urllib.request.Request(url)
req.add_header('user-agent', 'Mozilla/5.0')
response2 = urllib.request.urlopen(req)
print(response2.getcode())
print(len(response2.read()))
2016-01-03
python3
import urllib.request
url = "http://www.baidu.com"
print('第一種方法')
response1=urllib.request.urlopen(url)
print(response1.getcode())
print(len(response1.read()))
import urllib.request
url = "http://www.baidu.com"
print('第一種方法')
response1=urllib.request.urlopen(url)
print(response1.getcode())
print(len(response1.read()))
2016-01-03
已采納回答 / 戴暉
仔細(xì)看看你的代碼哪里寫的有問(wèn)題，估計(jì)是不仔細(xì)?；蛘呤悄闩赖木W(wǎng)頁(yè)有問(wèn)題，換個(gè)東西爬爬看
2016-01-02
import sys
...
type = sys.getfilesystemencoding()
...
fout.write("<td>%s</td>" % data['title'].encode(type))
...
type = sys.getfilesystemencoding()
...
fout.write("<td>%s</td>" % data['title'].encode(type))
2016-01-02
outputer在data['title'].encode('utf-8')后，內(nèi)容亂碼，怎么辦
2016-01-02
最新回答 / 小楠仔子
你說(shuō)的js頁(yè)面應(yīng)該是指動(dòng)態(tài)加載數(shù)據(jù)的js方法，而這些js調(diào)用方法一般會(huì)調(diào)用特定的API返回json數(shù)據(jù)，所以直接訪問(wèn)api然后解析返回的json數(shù)據(jù)是一種解決方案。我也是初學(xué)，有不對(duì)的地方見(jiàn)諒。
2016-01-02