newtask_queque逐漸達(dá)到上限隨后fetcher2processor、scheduler2fetcher也相繼達(dá)到上限導(dǎo)致隊(duì)列堵塞已開啟多個(gè)fetcher和processor,增大queue_size至10000,仍無法解決
pyspider queue隊(duì)列堵塞
慕尼黑5688855
2019-03-25 11:24:51