Can someone explain this #Python import behavior

[bmispelon@mastodon.social]

@treyhunner Tagging you on this since it might qualify as a #Pythonoddity

[alexgmin@mastodon.social]

@bmispelon Wtf

[_chrismay@fosstodon.org]

@bmispelon My thought would be Python:
- runs c.py and immediately starts to import a.py
- sets `A` to 1
- then starts to import b.py
- module a(.py) is already in memory, so it bumps A to 2
- finished importing b and a
- returns to c.py and prints 2

[bmispelon@mastodon.social]

@_chrismay I think you're on to something. In my mental model Python would only "cache" an import after it's complete, but that doesn't appear correct.

Bonus questions for you then, what if `a.py` contains this: `A = 1; from b import *; A+=1`?
Or even this `A = 1; from b import *; A+=1;from b import *` ?

[ehmatthes@fosstodon.org]

@bmispelon My initial guess was 2. c first imports a.A, which is 1. But the call to import from a loads a.py. That includes the call to import * from b, which imports from a. So at that point, A is 1. b then adds one to A, which sets A at 2. Then execution returns to c, with A at 2. So I think the value of A in c comes from b.

I tried to verify this in a pdb session, but stepping through at a low enough level to see this was bringing me into even lower level Python functions.

[jonafato@mastodon.social]

@bmispelon I *think* what's happening is basically sleight of hand (though I guessed wrong at first and would not advise that people use this behavior because it can confuse us). `a.A` gets reassigned when it imports `b.A` (via `b.*`).

Autoformatting these imports would cause an error, and obviously we want these files structured this way, so setting `__all__ = []` in `b.py` is the correct fix here.

[ehmatthes@fosstodon.org]

@bmispelon Then I ran c.py in a VS Codium debugger session, watching A.

- A starts as undefined (everything does).
- After the first line of a.py, A is 1, but I think that VS Codium is actually reporting a.A.
- The import in a is hit, and A goes to undefined. I think VSC is showing b.A.
- b's import runs, and A is 1. I think that's b.A.
- The second line of b is run, and A is 2.
- Execution goes back to c, where the value of A is 2.

[bmispelon@mastodon.social]

@jonafato Interesting suggestion for a fix! What happens then if all the `from ... import *` are replaced by `from ... import A`?

[ehmatthes@fosstodon.org]

@bmispelon Here's my VSCodium session:

[ehmatthes@fosstodon.org]

@bmispelon Can you share the real-world motivation for this question at some point?

[bmispelon@mastodon.social]

@ehmatthes A very old Django project whose multiple settings files were importing from each other, leaving me very confused for a bit

I definitely would not recommend writing actual code that looks like this!

[ehmatthes@fosstodon.org]

@bmispelon

> whose multiple settings files were importing from each other

You are not the only one who would be confused, please do not mention this in office hours

[pawamoy@fosstodon.org]

@bmispelon

Got it! Did it in my head then verified with an interpreter

There's nothing weird here. Python executes stuff sequentially, so:

- in from a import A
- in a: A = 1
- in a: from b import *
- in b: from a import * (so we have A = 1 in b)
- in b: A += 1 (so we have A = 2 in b)
- in a: finishing previous import, so we now have A = 2 in a
- in finishing previous import, so we now have A = 2 in C
- in print(A) -> 2!

[jonafato@mastodon.social]

@bmispelon That would result in the same original behavior, since `__all__` controls the import behavior of `*` but not of individual variables (though I think I have seen projects that allow you to turn that kind of thing into an error via name mangling or some other hack under the hood).

[pawamoy@fosstodon.org]

@bmispelon by the way I'm not sure to understand why the circular import works. I think Python has special handling for some cases where it's able to tell the circular import is "safe" somehow (like "a is almost finished, there's only * to import from b", meaning b can import from a again, and when b is finisehd a is updated again with any symbols declared in b). Tried to find an actual answer in the past but didn't find anything. Maybe should read the sources!

[pawamoy@fosstodon.org]

@bmispelon OK no it's much simpler, module a is simply partially initialized. By the time b imports it, it's not re-executed since it exists in sys.modules, and b imports every existing (yet) symbols within A. from a import A would work too.

[treyhunner@mastodon.social]

@bmispelon This is absolutely a Python oddity. I guessed incorrectly. I understand why I guessed incorrectly now that I look back at the code... I'm not sure any Python oddity has stress testeded my mental model of Python's import system as much as this one.

[stylus@social.afront.org]

@bmispelon

$ echo 'A = 1; print("A1"); from b import A; print("A2")' > a.py
$ echo 'print("B1"); from a import A; print("B2"); A += 1' > b.py
$ python -c 'from a import A; print(A)'
A1
B1
B2
A2
2

I added several prints so that it's possible to tell what order code is executed, and changed import * to import A because I think it improves clarity without changing the behavior.

• The main program runs
• It encounters an import of a so it starts executing the content of a.py in a newly created a module
• It sets A.a=1 via the assignment statement in a.py
• It encounters an import of b so it starts executing the content of b.py in a newly created b module
• It sets b.A=1 by from...import
• It adds 1 to b.A so that b.A is now equal to 2
• Execution reaches the end of b.py so it returns to a.py
• a.py sets a.A to 2 by from...import
• Execution reaches the end of a.py so it returns to the main program.
• The main program sets __main__.A to 2 by from ...import
• The value of A is printed (2)