def?parse(self,pageUrl,htmlCont):
????if?pageUrl?is?None?or?htmlCont?is?None:
????????return
????soup?=?BeautifulSoup(htmlCont,'html_parser',from_encoding='utf-8')
????new_urls?=?self._get_new_urls(pageUrl,soup)
????new_data?=?self._get_new_data(pageUrl,soup)
????return?new_urls,?new_data
這個方法里
soup?=?BeautifulSoup(htmlCont,'html_parser',from_encoding='utf-8')這一句代碼執(zhí)行完,?parse?方法?就直接返回了,導致new_urls?,new_data?都沒有值,main模塊中的的try就出現(xiàn)異常,然后就退出了,是哪個環(huán)節(jié)出錯導致的呀,求大神指點
解析第一個url就craw failed 了,請大神幫忙排查原因
weibo_小PENG大志_0
2018-07-08 16:52:52