Manage Secrets and Config in Python Apps

By Khimananda Oli | Last reviewed: August 2026

Hardcoded credentials remain the single most common cause of data breaches in Python applications I audit. To properly manage secrets and config in Python apps, you must decouple sensitive values from source code while maintaining strict type safety and validation at startup. This guide covers the production-grade pattern using pydantic-settings, local development workflows with .env files, and secure integration with cloud secret managers.

How Do You Manage Secrets and Config in Python Apps Using Pydantic-Settings?

The standard library os.getenv() is insufficient for production systems because it returns unvalidated strings and fails silently when variables are missing. In 2026, pydantic-settings (v2.x) is the definitive solution to manage secrets and config in Python apps. It provides automatic environment variable loading, type coercion, nested structure support, and immediate validation failures if required configuration is absent.

Install the library and create a centralized settings module. This approach ensures that misconfigurations crash the app at startup rather than causing silent failures hours later in production.

# requirements.txt
pydantic-settings==2.8.0
python-dotenv==1.1.0

# config.py
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import field_validator

class AppSettings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",
        env_file_encoding="utf-8",
        case_sensitive=False,
        extra="forbid"  # Reject undefined vars to catch typos
    )

    # Required fields - app fails fast if missing
    database_url: str
    api_secret_key: str
    debug: bool = False
    
    # Nested configuration
    redis_host: str = "localhost"
    redis_port: int = 6379

    @field_validator("database_url")
    @classmethod
    def validate_db_url(cls, v: str) -> str:
        if not v.startswith(("postgresql://", "mysql://")):
            raise ValueError("database_url must be postgresql:// or mysql://")
        return v

# Singleton instance - import this everywhere
settings = AppSettings()

This pattern aligns with the principles discussed in the twelve-factor app revisited, where configuration is strictly separated from code. The extra="forbid" setting is critical: it prevents typos in environment variable names from going unnoticed, a common issue I see during SOC 2 audits.

How Should You Handle Environment Variables and .env Files Safely?

Local development requires convenience without sacrificing security discipline. The .env file bridges this gap but must be handled correctly to avoid accidental credential leakage.

Setting Up Local Development Configuration

1. Create a .env.example file committed to Git with placeholder values only
2. Add .env to your .gitignore immediately after project initialization
3. Copy the example file locally: cp .env.example .env
4. Populate real secrets in .env (never share this file)
5. Use pre-commit hooks with tools like gitleaks or detect-secrets to scan staged changes

# .env.example (COMMIT THIS)
DATABASE_URL=postgresql://user:password@localhost:5432/mydb
API_SECRET_KEY=change-me-to-a-real-secret
DEBUG=true
REDIS_HOST=localhost
REDIS_PORT=6379

# .gitignore (CRITICAL)
.env
.env.local
.env.*.local
!.env.example

A common mistake is assuming .gitignore retroactively removes tracked files. If you accidentally commit a .env file, run git rm --cached .env immediately and rotate every secret contained within it. For teams working across Nepal and global offices, consider that developers may clone repositories over insecure networks; never assume a leaked secret hasn't been intercepted.

Environment Variable Precedence

Understanding precedence prevents debugging nightmares. pydantic-settings resolves configuration in this order (highest priority first):

• Constructor arguments passed directly to AppSettings()
• System environment variables
• Variables defined in .env file
• Default values in the model definition

This means production environment variables always override local .env files, which is exactly the behavior you want for safe deployments.

How Do You Integrate Cloud Secret Managers With Python Applications?

For production environments, storing secrets in environment variables alone is insufficient for compliance frameworks like SOC 2 or ISO 27001. Dedicated secret managers provide encryption at rest, access logging, automatic rotation, and fine-grained IAM policies. When you manage secrets and config in Python apps at scale, integrating with AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault becomes mandatory.

Custom Settings Source for AWS Secrets Manager

pydantic-settings supports custom sources. Here's a production-ready implementation that fetches secrets from AWS Secrets Manager and merges them with environment variables:

# aws_settings_source.py
import json
import boto3
from pydantic_settings import PydanticBaseSettingsSource
from botocore.exceptions import ClientError

class AWSSecretsManagerSource(PydanticBaseSettingsSource):
    def __init__(self, settings_cls, secret_name: str):
        super().__init__(settings_cls)
        self.secret_name = secret_name
        self._secrets_cache = None

    def _load_secrets(self) -> dict:
        if self._secrets_cache is not None:
            return self._secrets_cache
        
        client = boto3.client("secretsmanager")
        try:
            response = client.get_secret_value(SecretId=self.secret_name)
            self._secrets_cache = json.loads(response["SecretString"])
        except ClientError as e:
            raise RuntimeError(f"Failed to load secret {self.secret_name}: {e}")
        
        return self._secrets_cache

    def get_field_value(self, field, field_name):
        secrets = self._load_secrets()
        return secrets.get(field_name), field_name in secrets

    def __call__(self):
        return self._load_secrets()

# Usage in config.py
class ProductionSettings(AppSettings):
    @classmethod
    def settings_customise_sources(cls, settings_cls, init_settings, 
                                    env_settings, dotenv_settings, file_secret_settings):
        return (
            init_settings,
            env_settings,
            AWSSecretsManagerSource(settings_cls, "prod/myapp/secrets"),
            dotenv_settings,
        )

This approach keeps your settings interface identical across environments. Developers use .env locally; production pulls from AWS Secrets Manager transparently. For Kubernetes deployments, see Kubernetes secrets management done right for cluster-specific patterns that complement this application-level strategy.

What Are the Security Risks of Mismanaging Python Configuration?

Understanding failure modes helps you build defenses. These are the most common vulnerabilities I encounter when auditing Python applications:

Logging is particularly dangerous. Even if you never explicitly log secrets, exception tracebacks can include them. Configure your logging framework to redact known sensitive field names. For comprehensive guidance on safe observability, refer to structured logging best practices.

Validating Secrets Before Use

Never trust that a fetched secret is valid. Add runtime checks for critical credentials:

# Validate database connectivity at startup
from sqlalchemy import create_engine, text

def verify_database_connection(url: str) -> None:
    engine = create_engine(url, pool_pre_ping=True)
    with engine.connect() as conn:
        conn.execute(text("SELECT 1"))
    print("✓ Database connection verified")

# Call during app initialization, before accepting traffic
verify_database_connection(settings.database_url)

This fail-fast approach prevents deploying broken configurations. In my experience managing infrastructure for fintech companies in Nepal handling eSewa and Khalti integrations, startup validation has prevented more incidents than any amount of test coverage.

Manage Secrets and Config in Python Apps: Production Checklist

Securing Python configuration is not optional—it's foundational to trustworthy software. Implement these controls before your next production deployment:

• Replace all os.getenv() calls with pydantic-settings models
• Add .env to .gitignore and commit only .env.example
• Enable extra="forbid" to catch configuration typos immediately
• Integrate with a cloud secret manager for production workloads
• Implement pre-commit secret scanning with gitleaks
• Add startup validation for critical external dependencies
• Configure log redaction for sensitive field names
• Document secret rotation procedures and test them quarterly

If your team needs help implementing secure configuration patterns, passing SOC 2 audits, or migrating legacy Python apps to modern secret management, reach out to discuss your specific requirements. Secure configuration is the foundation everything else rests on—get it right first.