This is the best article on thread synchronization in Python that I have come across: Thread Synchronization Mechanisms in Python
It is very clearly explained and just helped me solve a major concurrency issues I was having.
Especially interesting is the list of thread safe operations in Python.
Here are some thread-safe operations:
- reading or replacing a single instance attribute
- reading or replacing a single global variable
- fetching an item from a list
- modifying a list in place (e.g. adding an item using append)
- fetching an item from a dictionary
- modifying a dictionary in place (e.g. adding an item, or calling the clear method)
Here is an explanation of global value mutation that are thread safe:
http://effbot.org/pyfaq/what-kinds-of-global-value-mutation-are-thread-safe.htm
Update: Now I am confused. Doesn't this example in the article contradict bullet point 2 above?
Update Update: See comments below for better explanation and info.
5 comments:
Even if those actions are "thread safe" due to current implementation - I would *not* go the route of doing them within appropriate locking mechanisms, relying on an implementation detail for shared data is a bad idea.
For the performance tests I've been working on, I have definitely moved to "less shared, more queues" for object passing and sharing. For configuration data, I pass it in read-only to the children.
Additional good reads:
http://utcc.utoronto.ca/~cks/space/blog/python/BuiltinsConcurrencyGuarantee?showcomments#comments
http://utcc.utoronto.ca/~cks/space/blog/python/BuiltinsConcurrencyAdvantage?showcomments#comments
http://utcc.utoronto.ca/~cks/space/blog/python/SetdefaultAsLockingPrimitive
http://utcc.utoronto.ca/~cks/space/blog/programming/PracticalLanguageGuarantees?showcomments#comments
Just call me paranoid about using "atomic" operations rather than doing explicit locking around data access.
thanks for the links.
You are right that I shouldn't rely on CPython implementation details :)
yeah, I am also working on some queues to make concurrent code sequential in some way.
There is no contradiction. The problem with the example is that the counter does both reads AND modifies, whereas the bulletpoint 2 refers to read OR modify.
Basically in between the read and the modification another thread may modify the value of the counter.
The apparent contradiction is mentioned immediately after the bullet list:
"Note that as mentioned earlier, operations that read a variable or attribute, modifies it, and then writes it back are not thread-safe. Another thread may update the variable after it’s been read by the current thread, but before it’s been updated."
Post a Comment