Python Project Structure Template

This document describes a modern Python project structure with best practices for tooling, testing, and code quality. Use this as a template for generating any Python project.

Project Overview

Package Management: Poetry Python Version: 3.10+ Key Tools: pytest, ruff, mypy, poethepoet

Directory Structure

project-root/
├── .git/                          # Git repository
├── .mypy_cache/                   # MyPy type checker cache (gitignored)
├── .pytest_cache/                 # Pytest cache (gitignored)
├── .ruff_cache/                   # Ruff linter/formatter cache (gitignored)
├── data/                          # Data files (gitignored if contains secrets/large files)
├── examples/                      # Example usage scripts
│   └── sample_usage.py           # Example of how to use the package
├── scripts/                       # Utility scripts (e.g., data processing, setup)
│   └── helper_script.py          # Helper scripts for development
├── sessions/                      # Session notes/documentation (gitignored)
│   └── 2025-11-01-notes.md       # Development session logs
├── src/                          # Source code (src layout for packages)
│   └── package_name/             # Main package
│       ├── __init__.py           # Package initialization, version
│       ├── config.py             # Configuration management
│       ├── core.py               # Core business logic
│       └── utils.py              # Utility functions
├── tests/                        # Test suite (mirrors src structure)
│   ├── __init__.py               # Makes tests a package
│   ├── conftest.py               # Pytest fixtures and configuration
│   ├── test_config.py            # Configuration tests
│   ├── test_core.py              # Core logic tests
│   └── test_utils.py             # Utility function tests
├── .coverage                     # Coverage report (gitignored)
├── .env                          # Environment variables (gitignored, required)
├── .env.example                  # Environment variable template (committed)
├── .gitignore                    # Git ignore rules
├── LICENSE                       # License file
├── poetry.lock                   # Poetry dependency lock file (committed)
├── pyproject.toml                # Poetry config, dependencies, tool configs
├── README.md                     # Main documentation
└── ruff.toml                     # Ruff linter/formatter configuration

Key Files and Their Purposes

Configuration Files

`pyproject.toml`

The central configuration file for the project. Contains:

Project Metadata:

[tool.poetry]
name = "project-name"
version = "0.1.0"
description = "Project description"
authors = ["Author Name <[email protected]>"]
readme = "README.md"
packages = [{include = "package_name", from = "src"}]

[tool.poetry.dependencies]
python = "^3.10"
# Add your production dependencies here
# requests = "^2.31.0"
# fastapi = "^0.104.0"
# sqlalchemy = "^2.0.0"

[tool.poetry.group.dev.dependencies]
pytest = "^8.3.0"
pytest-cov = "^6.0.0"
pytest-mock = "^3.14.0"
ruff = "^0.8.0"
mypy = "^1.13.0"
poethepoet = "^0.35.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Task Runner (Poe the Poet):

[tool.poe.tasks]
# Application tasks
start = {cmd = "python -m package_name", help = "Start the application"}

# Testing tasks
test = {cmd = "pytest tests/ -v --cov=src/package_name --cov-report=term-missing", help = "Run tests with coverage"}
test-fast = {cmd = "pytest tests/ -v", help = "Run tests without coverage"}
test-watch = {cmd = "pytest tests/ -v --lf", help = "Run only failed tests"}

# Code quality tasks
lint = {cmd = "ruff check src tests", help = "Run linter on source and tests"}
format = {cmd = "ruff format src tests", help = "Format code with Ruff"}
format-check = {cmd = "ruff format --check src tests", help = "Check code formatting"}
typecheck = {cmd = "mypy src", help = "Run static type checking"}

# Combined quality check
check-all = ["format-check", "lint", "typecheck", "test"]

Pytest Configuration:

[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
addopts = [
    "-v",
    "--strict-markers",
    "--strict-config",
]
markers = [
    "unit: Unit tests",
    "integration: Integration tests",
    "slow: Slow-running tests",
]

Ruff Configuration:

[tool.ruff]
line-length = 100
target-version = "py310"

[tool.ruff.lint]
select = [
    "E",   # pycodestyle errors
    "W",   # pycodestyle warnings
    "F",   # pyflakes
    "I",   # isort
    "N",   # pep8-naming
    "UP",  # pyupgrade
    "B",   # flake8-bugbear
    "C4",  # flake8-comprehensions
    "PTH", # flake8-use-pathlib
]
ignore = []

[tool.ruff.lint.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests

MyPy Configuration:

[tool.mypy]
python_version = "3.10"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
no_implicit_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
strict_equality = true

# Add overrides for third-party packages without type hints
[[tool.mypy.overrides]]
module = ["some_package.*"]
ignore_missing_imports = true

Coverage Configuration:

[tool.coverage.run]
source = ["src"]
omit = ["*/tests/*", "*/__init__.py"]

[tool.coverage.report]
exclude_lines = [
    "pragma: no cover",
    "def __repr__",
    "raise AssertionError",
    "raise NotImplementedError",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
]

`ruff.toml`

Ruff linter and formatter configuration (can also be in pyproject.toml):

line-length = 100
target-version = "py310"

[lint]
select = [
    "E",   # pycodestyle errors
    "W",   # pycodestyle warnings
    "F",   # pyflakes
    "I",   # isort (import sorting)
    "N",   # pep8-naming
    "UP",  # pyupgrade (Python version upgrades)
    "B",   # flake8-bugbear (common bugs)
    "C4",  # flake8-comprehensions
    "PTH", # flake8-use-pathlib (prefer pathlib)
]
ignore = []

[lint.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert statements in tests

`.env.example`

Template for environment variables (committed to version control):

# Application Configuration
APP_ENV=development
APP_DEBUG=true
LOG_LEVEL=INFO

# Database (example)
# DATABASE_URL=postgresql://user:password@localhost:5432/dbname

# API Keys (example)
# API_KEY=your-api-key-here
# SECRET_KEY=your-secret-key-here

# Paths
DATA_DIR=./data

`.gitignore`

Standard Python gitignore:

# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg

# Virtual environments
.venv/
venv/
ENV/
env/

# IDEs
.vscode/
.idea/
*.swp
*.swo
*~

# Testing
.pytest_cache/
.coverage
htmlcov/
.tox/

# Type checking
.mypy_cache/
.pytype/
.pyre/

# Linting
.ruff_cache/

# Environment
.env

# Project-specific
data/
sessions/
*.log

# OS
.DS_Store
Thumbs.db

Source Code Organization

`src/` Layout

Use the src/ layout (not flat layout) for better separation and import testing:

# src/package_name/__init__.py
"""Package description."""

__version__ = "0.1.0"

# Optionally expose main functionality
from package_name.core import main_function

__all__ = ["main_function"]

Configuration Module Pattern

# src/package_name/config.py
"""Configuration management."""

from pathlib import Path
from dotenv import load_dotenv
import os

load_dotenv()

class Config:
    """Application configuration from environment variables."""

    # Application settings
    APP_ENV: str = os.getenv("APP_ENV", "development")
    DEBUG: bool = os.getenv("APP_DEBUG", "false").lower() == "true"
    LOG_LEVEL: str = os.getenv("LOG_LEVEL", "INFO")

    # Paths
    DATA_DIR: Path = Path(os.getenv("DATA_DIR", "./data"))

    @classmethod
    def validate(cls) -> None:
        """Validate required configuration."""
        # Add validation logic
        if not cls.DATA_DIR.exists():
            cls.DATA_DIR.mkdir(parents=True, exist_ok=True)

Module Organization Pattern

Main modules should be runnable:

# src/package_name/module.py
"""Module description."""

from typing import Any

def main() -> None:
    """Main entry point."""
    # Implementation
    pass

if __name__ == "__main__":
    main()

Testing Structure

`conftest.py` - Shared Fixtures

"""Pytest configuration and shared fixtures."""

import tempfile
from collections.abc import Generator
from pathlib import Path
import pytest

@pytest.fixture
def temp_dir() -> Generator[Path, None, None]:
    """Create a temporary directory for tests."""
    with tempfile.TemporaryDirectory() as tmp_dir:
        yield Path(tmp_dir)

@pytest.fixture
def sample_data() -> list[dict[str, Any]]:
    """Return sample test data."""
    return [
        {"id": 1, "name": "test1", "value": 100},
        {"id": 2, "name": "test2", "value": 200},
    ]

@pytest.fixture
def mock_config(monkeypatch: pytest.MonkeyPatch) -> None:
    """Mock configuration for tests."""
    from package_name import config as config_module
    monkeypatch.setattr(config_module.Config, "DEBUG", True)
    monkeypatch.setattr(config_module.Config, "LOG_LEVEL", "DEBUG")

Test File Pattern

"""Tests for module_name."""

import pytest
from package_name.module import function_to_test

def test_function_basic() -> None:
    """Test basic functionality.

    Arrange-Act-Assert pattern.
    """
    # Arrange
    input_data = "test input"
    expected = "expected output"

    # Act
    result = function_to_test(input_data)

    # Assert
    assert result == expected

@pytest.mark.parametrize("input_val,expected", [
    ("input1", "output1"),
    ("input2", "output2"),
    ("", "default"),
])
def test_function_parametrized(input_val: str, expected: str) -> None:
    """Test with multiple parameter combinations."""
    result = function_to_test(input_val)
    assert result == expected

def test_function_with_mocks(mocker) -> None:
    """Test with mocked external dependencies."""
    # Mock external function call
    mock_external = mocker.patch("package_name.module.external_function")
    mock_external.return_value = {"status": "success"}

    result = function_to_test()

    assert result is not None
    mock_external.assert_called_once()

def test_function_raises_error() -> None:
    """Test that function raises expected error."""
    with pytest.raises(ValueError, match="Invalid input"):
        function_to_test(invalid_input="bad")

Development Workflow

Initial Setup

# 1. Install Poetry
curl -sSL https://install.python-poetry.org | python3 -

# 2. Create new project
poetry new --src project-name
cd project-name

# 3. Install dependencies
poetry install

# 4. Configure environment
cp .env.example .env
# Edit .env with your configuration

# 5. Verify setup
poetry run poe check-all

Daily Development Commands

# Run tests during development
poetry run poe test-fast

# Format code
poetry run poe format

# Run all checks before commit
poetry run poe check-all

# Run specific module
poetry run python -m package_name.module

# Start application
poetry run poe start

Git Workflow

# Standard workflow
git status
git add .
git commit -m "Description of changes"
git push

# Before committing, always run:
poetry run poe check-all

Code Quality Standards

Type Hints (Required)

from pathlib import Path
from collections.abc import Generator
from typing import Any

def process_items(
    items: list[str],
    limit: int = 10,
    filter_fn: callable[[str], bool] | None = None
) -> list[str]:
    """Process list of items."""
    results: list[str] = []
    # Implementation
    return results

def read_file(file_path: Path) -> Generator[str, None, None]:
    """Generate lines from file."""
    with file_path.open() as f:
        for line in f:
            yield line.strip()

Docstrings (Required for Public APIs)

def function_name(param1: str, param2: int = 5) -> bool:
    """Brief description of what the function does.

    More detailed description if needed. Can span multiple
    lines to explain complex behavior.

    Args:
        param1: Description of param1.
        param2: Description of param2. Defaults to 5.

    Returns:
        Description of return value.

    Raises:
        ValueError: When param1 is empty.
        TypeError: When param2 is not an integer.

    Example:
        >>> function_name("test", 10)
        True
    """
    if not param1:
        raise ValueError("param1 cannot be empty")
    return True

Import Organization (Enforced by Ruff)

# Standard library imports
import os
import sys
from pathlib import Path
from typing import Any

# Third-party imports
import pytest
import requests
from fastapi import FastAPI

# Local imports
from package_name.config import Config
from package_name.utils import helper_function

README.md Structure

A comprehensive README should include:

# Project Name

Brief description of what the project does.

## Features

- Feature 1
- Feature 2
- Feature 3

## Prerequisites

- Python 3.10 or higher
- Any other requirements

## Installation

### 1. Clone the repository

\`\`\`bash
git clone <repository-url>
cd project-name
\`\`\`

### 2. Install Poetry

\`\`\`bash
curl -sSL https://install.python-poetry.org | python3 -
\`\`\`

### 3. Install dependencies

\`\`\`bash
poetry install
\`\`\`

### 4. Configure environment

\`\`\`bash
cp .env.example .env
# Edit .env with your configuration
\`\`\`

## Usage

### Basic usage

\`\`\`bash
poetry run python -m package_name
\`\`\`

### Using as a library

\`\`\`python
from package_name import main_function

result = main_function("input")
print(result)
\`\`\`

## Development

### Running Tests

\`\`\`bash
# Run all tests with coverage
poetry run poe test

# Run tests without coverage (faster)
poetry run poe test-fast
\`\`\`

### Code Quality

\`\`\`bash
# Format code
poetry run poe format

# Check formatting
poetry run poe format-check

# Lint code
poetry run poe lint

# Type checking
poetry run poe typecheck

# Run all checks
poetry run poe check-all
\`\`\`

## Project Structure

\`\`\`
project-name/
├── src/package_name/     # Source code
├── tests/                # Test suite
├── examples/             # Usage examples
├── pyproject.toml        # Project configuration
└── README.md
\`\`\`

## Configuration

Configuration is done via environment variables in `.env`:

\`\`\`bash
APP_ENV=development
DEBUG=true
\`\`\`

## Contributing

1. Run `poe check-all` before committing
2. Add tests for new features
3. Update documentation as needed

## License

[Your license here]

Poetry Commands Reference

# Dependency management
poetry add package-name              # Add production dependency
poetry add --group dev package-name  # Add dev dependency
poetry remove package-name           # Remove dependency
poetry update                        # Update dependencies
poetry update package-name           # Update specific package
poetry show                          # List dependencies
poetry show --tree                   # Show dependency tree
poetry show package-name             # Show package details

# Environment management
poetry env info                      # Show environment info
poetry env list                      # List environments
poetry env remove python             # Remove environment
poetry shell                         # Activate virtual environment

# Running commands
poetry run python script.py          # Run Python script
poetry run pytest                    # Run tests
poetry run poe task-name            # Run Poe task

# Building and publishing
poetry build                         # Build package
poetry publish                       # Publish to PyPI
poetry version                       # Show current version
poetry version patch                 # Bump patch version
poetry version minor                 # Bump minor version
poetry version major                 # Bump major version

Best Practices Summary

Code Quality

Use type hints for all functions
Write docstrings for public APIs
Maintain 80%+ test coverage
Run poe check-all before committing
Use src/ layout for packages
Follow AAA pattern in tests (Arrange-Act-Assert)
Keep line length to 100 characters

Dependencies

Lock dependencies with poetry.lock (commit to version control)
Use ^ for compatible versions (^1.2.0 = >=1.2.0 <2.0.0)
Separate dev dependencies from production
Document required environment variables in .env.example
Pin dependencies for libraries, use ranges for applications

Testing

Mock external APIs and dependencies
Use fixtures for reusable test data
Parametrize tests for multiple cases
Test both success and error paths
Use descriptive test names (test_function_does_specific_thing)
One assertion per test (when possible)

Git Practices

Write clear, descriptive commit messages
Keep .gitignore comprehensive
Don't commit .env, cache files, or build artifacts
Do commit .env.example, poetry.lock, and config files
Create feature branches for new work

Project Organization

One module = one responsibility
Keep modules runnable with if __name__ == "__main__"
Configuration in separate module
Tests mirror source structure
Examples in separate directory
Use absolute imports from package root

Project Generation Checklist

When creating a new project based on this structure:

Common Project Types

CLI Application

[tool.poetry.scripts]
myapp = "package_name.cli:main"

# src/package_name/cli.py
import argparse

def main() -> None:
    parser = argparse.ArgumentParser(description="Application description")
    parser.add_argument("input", help="Input file")
    parser.add_argument("--verbose", action="store_true")
    args = parser.parse_args()
    # Implementation

Web API (FastAPI example)

# src/package_name/api.py
from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def read_root():
    return {"message": "Hello World"}

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

Library Package

# src/package_name/__init__.py
"""Library description."""

__version__ = "0.1.0"

from package_name.core import main_function, helper_function

__all__ = ["main_function", "helper_function"]

This structure provides a solid foundation for any modern Python project with comprehensive testing, code quality tools, and professional development practices.

mtomcal/PYTHON_STRUCTURE.md