Python import ModuleNotFoundError

ghz 2days ago ⋅ 10 views

I have a Python project that I am working on my local PC using PyCharm, and I execute the code from Google Colab server (because I need the GPU).

I get my project to Colab using GitHub (I upload the package to GitHub and than clone it in Colab).

However, I have problem with imports.

The package structure is like this:

a/
->b/
  ->c/
    -> script1.py
    -> script2.py
  ->d/
    -> script3.py

script1.py imports script2.py and script3.py.

I try import like that:

import script2
from b.d import script3

This is my test cases:

  • Locally on my PC, the main script is in folder a
  • Remotely on Google Colab, where the main script is the notebook (which is outside of folder a obviously).

In both this cases I get ModuleNotFoundError for my imports.

But if I use relative imports like so:

from . import script2
from ..d import script3

It works great in both of the above cases, but PyCharm complains about relative imports and the script would only execute when the main is outside folder b.

I tried adding __init__.py to folder d, but to be honest I still don't fully understand the purpose of this files. And from what I read they are redundant in Python 3.3+ (I use Python 3.7).

So my question is: How can I make the imports work anytime no matter the location of the main script?

Answer

To make your imports work consistently regardless of the location of the main script, you need to properly set up your project structure and manage the Python import paths. Here's how you can address your issue:

Solution 1: Set PYTHONPATH

The most robust and portable way is to set the PYTHONPATH environment variable so Python knows where to look for your modules.

Steps:

  1. Update PYTHONPATH:

    • Locally: Set the PYTHONPATH to the root of your project (a/ folder).
    • On Colab: Do the same programmatically.

    Example:

    export PYTHONPATH="/path/to/a:$PYTHONPATH"

    On Colab, add this to the first cell:

    import sys
sys.path.append('/path/to/a')

  2. Use Absolute Imports: Inside script1.py, use:

    from b.c import script2
from b.d import script3

This approach ensures that your scripts will work no matter where they are executed, as long as the PYTHONPATH is correctly set.

Solution 2: Use __init__.py for Packages

Even though __init__.py files are optional in Python 3.3+, they are useful for marking directories as Python packages and enabling package-level imports. Add empty __init__.py files in b/, b/c/, and b/d/.

Steps:

  1. Add __init__.py files:

    a/
├── b/
│   ├── c/
│   │   ├── __init__.py
│   │   ├── script1.py
│   │   ├── script2.py
│   ├── d/
│       ├── __init__.py
│       ├── script3.py
│   ├── __init__.py

  2. Use Absolute Imports: Inside script1.py, do:

    from b.c import script2
from b.d import script3

  3. Ensure you execute the script with the root (a/) in your Python path, similar to Solution 1.

Solution 3: Use Dynamic Path Adjustments

If you want to avoid setting environment variables, you can dynamically adjust the sys.path in your scripts. For example, in script1.py:

import sys
import os

# Add the root of the project to the path
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

# Now you can use absolute imports
from b.c import script2
from b.d import script3

This method ensures your imports work regardless of the execution location, but it's less clean since it modifies sys.path programmatically.

Solution 4: Use Relative Imports

Relative imports, like from . import script2, work if you execute the script as part of the package (e.g., python -m b.c.script1 from the a/ root). However, they won't work if you directly execute script1.py (python script1.py). Relative imports are also discouraged for standalone scripts because they depend on the execution context.

Best Practice

  1. Use absolute imports for clarity and maintainability.
  2. Ensure your project root is added to PYTHONPATH in all environments (local and Colab).
  3. Use __init__.py files to structure your packages.

Example Colab Setup

# Clone your repo and set the path
!git clone https://github.com/your-repo.git
import sys
sys.path.append('/content/your-repo/a')

# Now your imports will work
from b.c import script2
from b.d import script3
#py