關(guān)于以下示例:import osNUM_CPUS = None # defaults to all availabledef worker(f1, f2): os.system("run program x on f1 and f2") <--- Big command, cannot run more that K in paralleldef test_run(pool): filelist = os.listdir(files_dir) for f1 in filelist: for f2 in filelist: pool.apply_async(worker, args=(f1, f2))if __name__ == "__main__": import multiprocessing as mp pool = mp.Pool(NUM_CPUS) test_run(pool) pool.close() pool.join()每次os.system調(diào)用都會(huì)消耗大量資源,因此我無(wú)法并行調(diào)度超過(guò) K( 5) 的資源。不幸的是,即使NUM_POOLS=5每個(gè)設(shè)置都會(huì)pool.apply_async立即返回。如何指定python不讓超過(guò)5個(gè)worker并行運(yùn)行?
python與os系統(tǒng)多線程,不超過(guò)K個(gè)并行命令
天涯盡頭無(wú)女友
2023-10-31 19:19:30
