gh-114940: Add a Per-Interpreter Lock For the List of Thread States

[ericsnowcurrently]

The new lock may be used instead of the global "HEAD" lock. That global lock guards the list of interpreters, but unfortunately in some situations (like for interp->threads.head) it has been used as a generic "lock everything". This change helps us move away from that situation.

Note that there are 3 places in pystate.c where we have left HEAD_LOCK()/HEAD_UNLOCK() in place:

• interpreter_clear()
• new_threadstate()
• tstate_delete_common()

Otherwise we end up breaking a constraint on free-threaded builds.

Co-authored-by: RUANG (James Roy) [email protected]

• Issue: Add a Per-Interpreter "HEAD" Lock #114940

[ericsnowcurrently]

There's a bit of code in tstate_delete_common() to avoid a deadlock, but I'm sure it isn't right. It probably makes sense to use a recursive lock here. I'm going to look more closely.

[ericsnowcurrently]

Ideally we could build on top of gh-127121 for the recursive lock.