求助,爬蟲運(yùn)行失敗,錯(cuò)誤提示如下
C:\Python27\python.exe C:/Users/Administrator/PycharmProjects/baike/baike/spider_main.py
craw 1 : https://baike.baidu.com/item/Python/407313?fr=aladdin
Traceback (most recent call last):
? File "C:/Users/Administrator/PycharmProjects/baike/baike/spider_main.py", line 34, in <module>
? ? obj_spider.craw(root_url)
? File "C:/Users/Administrator/PycharmProjects/baike/baike/spider_main.py", line 19, in craw
? ? new_urls,new_data = self.parser.parse(new_url,html_cont)
TypeError: 'NoneType' object is not iterable
Process finished with exit code 1
2017-08-13
在獲取節(jié)點(diǎn)的地方,加上try : ? except: ?處理一下
比如:
summary_node = soup.find('div',class_='lemma-summary')
try:
? ?res_data['summary'] = summary_node.get_text()
except:
? ?res_data['summary'] = ''