如何在Python中使用线程本地存储?
例如,如果您有一个线程工作池,并且每个线程都需要访问其自己的资源(例如网络或数据库连接),则线程本地存储很有用。请注意,该threading模块使用常规的线程概念(可以访问进程全局数据),但是由于全局解释器锁定,它们并不是太有用。不同的multiprocessing模块会为每个模块创建一个新的子流程,因此任何全局变量都将是线程局部的。
threading
multiprocessing
这是一个简单的示例:
import threading from threading import current_thread threadLocal = threading.local() def hi(): initialized = getattr(threadLocal, 'initialized', None) if initialized is None: print("Nice to meet you", current_thread().name) threadLocal.initialized = True else: print("Welcome back", current_thread().name) hi(); hi()
这将打印出:
Nice to meet you MainThread Welcome back MainThread
一件很容易被忽略的重要事情:一个threading.local()对象只需要创建一次,而不是每个线程一次或每个函数调用一次。的global或class水平的理想地点。
threading.local()
global
class
这就是为什么:threading.local()每次调用它时都会实际上创建一个新实例(就像任何工厂或类调用一样),因此threading.local()多次调用会不断覆盖原始对象,这很可能不是您想要的。当任何线程访问现有threadLocal变量(或任何被调用的变量)时,它将获得该变量的私有视图。
threadLocal
这将无法正常工作:
import threading from threading import current_thread def wont_work(): threadLocal = threading.local() #oops, this creates a new dict each time! initialized = getattr(threadLocal, 'initialized', None) if initialized is None: print("First time for", current_thread().name) threadLocal.initialized = True else: print("Welcome back", current_thread().name) wont_work(); wont_work()
将产生以下输出:
First time for MainThread First time for MainThread
因为multiprocessing模块为每个线程创建一个新进程,所以所有全局变量都是线程局部的。
考虑以下示例,其中processed计数器是线程本地存储的示例:
processed
from multiprocessing import Pool from random import random from time import sleep import os processed=0 def f(x): sleep(random()) global processed processed += 1 print("Processed by %s: %s" % (os.getpid(), processed)) return x*x if __name__ == '__main__': pool = Pool(processes=4) print(pool.map(f, range(10)))
它将输出如下内容:
Processed by 7636: 1 Processed by 9144: 1 Processed by 5252: 1 Processed by 7636: 2 Processed by 6248: 1 Processed by 5252: 2 Processed by 6248: 2 Processed by 9144: 2 Processed by 7636: 3 Processed by 5252: 3 [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
…当然,线程ID以及每个线程ID和每个命令的计数会因运行而异。