執(zhí)行douban_spider出現(xiàn)問(wèn)題
"UnicodeEncodeError",python3編碼是utf-8,嘗試用virtualenv解決:
(project_1) [root@hamster douban]# python3 -m scrapy crawl douban_spider
...
2018-08-28 22:59:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://movie.douban.com/top250> (referer: None)
2018-08-28 22:59:33 [scrapy.core.scraper] ERROR: Spider error processing <GET https://movie.douban.com/top250> (referer: None)
Traceback (most recent call last):
? File "/root/project_1/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
? ? current.result = callback(current.result, *args, **kw)
? File "/root/douban/douban/spiders/douban_spider.py", line 14, in parse
? ? print(response.text)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 349-352: ordinal not in range(128)
...
2018-08-28 22:59:33 [scrapy.core.engine] INFO: Spider closed (finished)
還是報(bào)錯(cuò),檢查了一下:
(project_1) [root@hamster douban]# python3
Python 3.6.5 (default, Aug 28 2018, 22:34:18)?
[GCC 4.4.7 20120313 (Red Hat 4.4.7-23)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getdefaultencoding()
'utf-8'
懷疑是"douban_spider.py"代碼有誤,但不知道錯(cuò)在哪,希望老師幫我解答一下,謝謝!?
2018-09-01
你用virtualenv的時(shí)候,運(yùn)行文件時(shí),是否workon切換到正確的解釋器了