python - how to optimize multiprocessing in this code? -
results big variable , do:
results = sum(results, []) print 'writing results json %s' % res_file open(res_file, 'w') f: json.dump(results, f, sort_keys=true, indent=4)
for it. how store big variable in generate process?
pool = mp.pool(8) results = pool.map(coco_results_one_category_kernel, data_pack) pool.close() pool.join()
this code cannot allocation memory multiprocessing:
oserror: [errno 12] cannot allocate memory
results = [] in range(len(data_pack)): print i,'/',len(data_pack) results.append(coco_results_one_category_kernel(data_pack[i])) results = sum(results, [])
this code kiilled when i=36 because of result big. how can this? donot care time, result variable..
Comments
Post a Comment