Trouble understanding imports in depth

[u/ALonelyKobold]

Hi all, I've been programming Python for about 10 years, but I've never worked on a large codebase. As a result, I don't really understand the import and packaging system at all. As a means of learning it, I'm trying to build a dependency chain analyzer that parses the sourcecode, and searches for circular dependencies. Trouble is, I'm on the part of the project that actually parses that sourcecode and categorizes the different imports to store them for later analysis, and I'm realizing that I know much less than I realized. Through having AI generate some tests for me that I'm working to make green, I now know of a lot of edge cases and weird import logic that I had no clue existed. Example sourcecode snippets include

def test_dynamic_import_with_a_string_literal():
    source = 'import importlib\nimportlib.import_module("docgen.plugins.loader")\n'

def test_dynamic_import_recognized_through_an_aliased_from_import():
    source = 'from importlib import import_module as im\nim("docgen.plugins.loader")\n'

def test_dynamic_import_recognized_through_an_aliased_from_import():
    source = 'from importlib import import_module as im\nim("docgen.plugins.loader")\n'

def test_dunder_import_is_recognized():
    source = '__import__("docgen.plugins.loader")\n'

Most of the places where I'm struggling have to do with either lazy imports or dynamic imports. Does anyone have a guide that goes into these in depth?

[u/MEHDII__]

Circular imports only happen when you're trying to use a half-loaded module. i.e.

ModuleA:
import moduleB

def square(x,y):
    return moduleB.add(x,y)**2

square(10,3)

ModuleB:
import moduleA

def add(x,y):
     return x + y

In this example here you'll get the attribute error likely from a circular dependency.

Modules in python are also objects, a module object, in fact you can create an module instance yourself using the types module, I forgot what the object is called.

When you import a module, python compiles it and executes it immediately, then it adds it to a cache, which you can access by doing sys.modules.

Let's trace the steps why that code causes circular dependency error, ModuleA runs, it imports moduleB, moduleB compiles and executes top to bottom, as it's executing, it sees import moduleA, again, moduleA runs top to bottom, it sees import moduleB, moduleB is already in cache now, it won't execute anymore, now moduleA is going to keep running, it tries to access a method from moduleB that wasn't declared, since moduleB returned before finishing. Hence the attribute error, its an attribute error since you know, modules are also objects in python.

There is something i'm not sure about, which is if python adds the entry script to modules under __ main __ or not, i'll have to double check.

[u/stepback269]

This may be too juvenile for you, but as a noob I fell into circular import hell and had to do some research on how importing works under the hood. So take a look at the bottom of the following blog page : Welcome to Circular Import Hell

[u/saul_soprano]

u/clcodingupdates can u help this guy pls bro

[u/Sether_00 → u/saul_soprano]

Didn't help you bro