我有以下代碼：def task1(): for url in splitarr[0]: print(url) #these are supposed to be scrape_induvidual_page() . print is just for debuggingdef task2(): for url in splitarr[1]: print(url)def task3(): for url in splitarr[2]: print(url)def task4(): for url in splitarr[3]: print(url)def task5(): for url in splitarr[4]: print(url)def task6(): for url in splitarr[5]: print(url)def task7(): for url in splitarr[6]: print(url) def task8(): for url in splitarr[7]: print(url) splitarr=np.array_split(urllist, 8)t1 = threading.Thread(target=task1, name='t1') t2 = threading.Thread(target=task2, name='t2') t3 = threading.Thread(target=task3, name='t3')t4 = threading.Thread(target=task4, name='t4') t5 = threading.Thread(target=task5, name='t5')t6 = threading.Thread(target=task6, name='t6')t7 = threading.Thread(target=task7, name='t7')t8 = threading.Thread(target=task8, name='t8')t1.start() t2.start()t3.start() t4.start() t5.start()t6.start() t7.start()t8.start() t1.join()t2.join()t3.join()t4.join()t5.join()t6.join()t7.join() t8.join() 它確實(shí)有所需的輸出，沒有重復(fù)或任何東西https://kickasstorrents.to/big-buck-bunny-1080p-h264-aac-5-1-tntvillage-t115783.htmlhttps://kickasstorrents.to/big-buck-bunny-4k-uhd-hfr-60fps-eng-flac-webdl-2160p-x264-zmachine-t1041079.htmlhttps://kickasstorrents.to/big-buck-bunny-4k-uhd-hfr-60-fps-flac-webrip-2160p-x265-zmachine-t1041689.htmlhttps://kickasstorrents.to/big-buck-bunny-2008-720p-bluray-x264-don-no-rars-t11623.htmlhttps://kickasstorrents.to/tkillaahh-big-buck-bunny-dvd-720p-2lions-team-t87503.htmlhttps://kickasstorrents.to/big-buck-bunny-2008-720p-bluray-nhd-x264-nhanc3-t127050.htmlhttps://kickasstorrents.to/big-buck-bunny-2008-brrip-720p-x264-mitzep-t172753.html但是，我覺得代碼對于所有重復(fù)的def taskx() 來說有點(diǎn)多余： 所以我嘗試使用單個(gè)任務(wù)來壓縮代碼：x=0def task1(): global x for url in splitarr[x]: print(url) x=x+1如何在多線程程序中正確地使 x 遞增？
1 回答
慕的地10843
TA貢獻(xiàn)1785條經(jīng)驗(yàn) 獲得超8個(gè)贊
for url in splitarr[x]:為 中的序列創(chuàng)建一個(gè)迭代器splitarr[x]。稍后增加 x 并不重要 - 迭代器已經(jīng)構(gòu)建好了。由于其中有打印內(nèi)容，因此所有線程很可能會(huì)x在其仍為零時(shí)抓取并迭代相同的序列。
args一種解決方案是通過中的參數(shù)將遞增值傳遞給 task1 threading.Thread。但線程池更容易。
from multiprocessing.pool import ThreadPool
# generate test array
splitarr = []
for i in range(8):
splitarr.append([f"url_{i}_{j}" for j in range(4)])
def task(splitarr_column):
for url in splitarr_column:
print(url)
with ThreadPool(len(splitarr)) as pool:
result = pool.map(task, splitarr)
在此示例中，len(splitarr)用于為 中的每個(gè)序列創(chuàng)建一個(gè)線程splitarr。然后將每個(gè)序列映射到該task函數(shù)。由于我們創(chuàng)建了正確數(shù)量的線程來處理所有序列，因此它們都會(huì)同時(shí)運(yùn)行。當(dāng)映射完成時(shí)，該with子句退出并且池關(guān)閉，從而終止線程。
添加回答
舉報(bào)
0/150
提交
取消