gh-117680: make _PyInstructionSequence a PyObject and use it in tests

[iritkatriel]

• Issue: Give _PyInstructionSequence a python interface and use it in compiler tests #117680

[markshannon]

Overall, this looks like a nice improvement, both in terms of separation of compiler passes and in debugability and introspection.

What is the purpose of the nested instruction sequences?
AFAICT they are only used in the tests.

[iritkatriel]

What is the purpose of the nested instruction sequences?

For tests, and also if you want to display (dis-like) the codegen output, then you need the nested instructions. Otherwise for a function you only get the top level function definition, not its body. And then again for nested functions.

[iritkatriel]

What is the purpose of the nested instruction sequences?

For tests, and also if you want to display (dis-like) the codegen output, then you need the nested instructions. Otherwise for a function you only get the top level function definition, not its body. And then again for nested functions.

I have another PR coming where codegen populates the nested instruction sequences (optional - not happening in regular compilation).

[markshannon]

For tests, and also if you want to display (dis-like) the codegen output, then you need the nested instructions.

I have another PR coming where codegen populates the nested instruction sequences (optional - not happening in regular compilation).

I'm a bit wary of tests that diverge from the things they are testing.

Would it make sense to create the nested sequences in the normal compiler and generated nested code objected directly from nested sequences in the assembler?

[iritkatriel]

I'm a bit wary of tests that diverge from the things they are testing.

That's not what happens. The test is testing codegen output, and the test mode simply saves the nested sequences, whereas in normal compilation those sequences are discarded as soon as the codeobject is created. Nothing else is changing in test mode. The code objects are created in the same way.

Anyway, it's not in the PR so we can look at it in the next PR.

Would it make sense to create the nested sequences in the normal compiler and generated nested code objected directly from nested sequences in the assembler?

There are two issues with this. First, I am concerned it could consume more memory during normal compilation (for no benefit).

Second, the code object creation currently happens with a certain "compiler unit state". If we delay it to the end, the compiler unit is in a different state (or gone altogether). So we need to make sure we capture enough of this state for later. It's not a trivial change (and again, there doesn't seem to be any concrete benefit).