2 回答

TA貢獻(xiàn)1795條經(jīng)驗(yàn) 獲得超7個(gè)贊
這是您現(xiàn)有代碼的地圖版本。請(qǐng)注意,回調(diào)現(xiàn)在接受一個(gè)元組作為參數(shù)。我在回調(diào)中添加了一個(gè) try\except,因此結(jié)果不會(huì)拋出錯(cuò)誤。結(jié)果根據(jù)輸入列表排序。
from concurrent.futures import ThreadPoolExecutor
import urllib.request
URLS = ['http://www.foxnews.com/',
'http://www.cnn.com/',
'http://www.wsj.com/',
'http://www.bbc.co.uk/',
'http://some-made-up-domain.com/']
# Retrieve a single page and report the url and contents
def load_url(tt): # (url,timeout)
url, timeout = tt
try:
with urllib.request.urlopen(url, timeout=timeout) as conn:
return (url, conn.read())
except Exception as ex:
print("Error:", url, ex)
return(url,"") # error, return empty string
with ThreadPoolExecutor(max_workers=5) as executor:
results = executor.map(load_url, [(u,60) for u in URLS]) # pass url and timeout as tuple to callback
executor.shutdown(wait=True) # wait for all complete
print("Results:")
for r in results: # ordered results, will throw exception here if not handled in callback
print(' %r page is %d bytes' % (r[0], len(r[1])))
輸出
Error: http://www.wsj.com/ HTTP Error 404: Not Found
Results:
'http://www.foxnews.com/' page is 320028 bytes
'http://www.cnn.com/' page is 1144916 bytes
'http://www.wsj.com/' page is 0 bytes
'http://www.bbc.co.uk/' page is 279418 bytes
'http://some-made-up-domain.com/' page is 64668 bytes

TA貢獻(xiàn)1876條經(jīng)驗(yàn) 獲得超7個(gè)贊
在不使用該方法的情況下,您不僅map可以使用URL 作為值,還可以使用它們?cè)诹斜碇械乃饕齺?lái)enumerate構(gòu)建dict。然后,您可以使用索引作為鍵從調(diào)用返回的對(duì)象future_to_url構(gòu)建一個(gè)字典,這樣您就可以在字典的長(zhǎng)度上迭代索引,以按照與原始列表中相應(yīng)項(xiàng)目相同的順序讀取字典:futureconcurrent.futures.as_completed(future_to_url)
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
# Start the load operations and mark each future with its URL
future_to_url = {
executor.submit(load_url, url, 60): (i, url) for i, url in enumerate(URLS)
}
futures = {}
for future in concurrent.futures.as_completed(future_to_url):
i, url = future_to_url[future]
futures[i] = url, future
for i in range(len(futures)):
url, future = futures[i]
try:
data = future.result()
except Exception as exc:
print('%r generated an exception: %s' % (url, exc))
else:
print('%r page is %d bytes' % (url, len(data)))
添加回答
舉報(bào)