Viper Environment Variable Expansion Solution

A working solution for Viper Issue #315 - Environment variable expansion in configuration files.

The Problem

While working with gofiber and using viper to load env variables from .env or .env.* (depending on environment such as .env.testing when testing), I noticed viper does not handle dynamic env variables properly. A simple googling of this problem showed me that since 2017, developers have needed Viper to support environment variable expansion in .env files. For example:

QUERY_DB_HOST=${QUERY_DB_HOST:-127.0.0.1}
QUERY_DB_PORT=${QUERY_DB_PORT:-3306}

Viper would incorrectly parse these as :-127.0.0.1} instead of expanding them to 127.0.0.1.

The Solution

I implemented a post-processing approach that fixes Viper's incomplete parsing and properly expands environment variables.

1. Environment Variable Expansion Function

func expandEnvVars(value string) string {
    // Handle ${VAR} and ${VAR:-default} formats
    re1 := regexp.MustCompile(`\$\{([^}]+)\}`)
    value = re1.ReplaceAllStringFunc(value, func(match string) string {
        content := match[2 : len(match)-1]

        // Check if it contains default value syntax (:-default)
        if strings.Contains(content, ":-") {
            parts := strings.SplitN(content, ":-", 2)
            varName := parts[0]
            defaultValue := parts[1]

            envValue := os.Getenv(varName)
            if envValue == "" {
                return defaultValue
            }
            return envValue
        }

        // Simple variable expansion
        return os.Getenv(content)
    })

    // Handle $VAR format
    re2 := regexp.MustCompile(`\$([A-Za-z_][A-Za-z0-9_]*)`)
    value = re2.ReplaceAllStringFunc(value, func(match string) string {
        varName := match[1:]
        return os.Getenv(varName)
    })

    return value
}

2. Post-Processing Viper Values

func LoadConfig() (Config, error) {
    // ... viper setup code ...

    if err := viper.ReadInConfig(); err != nil {
        return Config{}, fmt.Errorf("error reading config file, %w", err)
    }

    // Fix Viper's incomplete env variable parsing
    allKeys := viper.AllKeys()
    for _, key := range allKeys {
        value := viper.GetString(key)
        if strings.HasPrefix(value, ":-") && strings.HasSuffix(value, "}") {
            // Reconstruct the original ${VAR:-default} pattern
            originalValue := "${" + strings.ToUpper(key) + value
            expandedValue := expandEnvVars(originalValue)
            viper.Set(key, expandedValue)
        }
    }

    // ... rest of config unmarshaling ...
}

3. Decode Hook for Additional Processing

func trimAndExpandHook() viper.DecoderConfigOption {
    return func(c *mapstructure.DecoderConfig) {
        c.DecodeHook = mapstructure.ComposeDecodeHookFunc(
            func(f reflect.Type, t reflect.Type, data interface{}) (interface{}, error) {
                if t.Kind() == reflect.String && f.Kind() == reflect.String {
                    if str, ok := data.(string); ok {
                        trimmed := strings.TrimSpace(str)
                        expanded := expandEnvVars(trimmed)
                        return expanded, nil
                    }
                }
                return data, nil
            },
            mapstructure.StringToSliceHookFunc(","),
        )
    }
}

How It Works

1. Viper reads the config file but incorrectly parses ${VAR:-default} as :-default}
2. Post-processing detects truncated values (starting with :- and ending with })
3. Reconstruction rebuilds the original pattern: ${KEY:-default}
4. Expansion applies proper environment variable substitution
5. Decode hook handles any remaining expansions during unmarshaling

Result

Environment variables are now properly expanded:

# Before (broken)
"Host": ":-127.0.0.1}"

# After (working)
"Host": "127.0.0.1"

Supported Syntax

• ${VAR} - Simple variable expansion
• ${VAR:-default} - Variable with default value
• $VAR - Shorthand variable expansion

This solution has been tested and successfully handles all common environment variable expansion patterns.

I hope this helps you and I hope some smart person would expand on this solution hopefully create a PR to goviper for this 😄