Enhance Configuration Management by Utilizing Environment Variables for Sensitive Data

Description:

Currently, OpenIM manages its configuration, including sensitive data like database credentials, through a configuration file. This approach, while functional, poses certain risks and lacks the flexibility required for different deployment environments (like Docker, Kubernetes, or traditional server setups). This issue proposes a shift from file-based configuration to environment variable-based configuration for sensitive data, aligning with best practices observed in top-tier open-source projects.

Proposed Solution:

1. Transition to Environment Variables:

• Modify the application to read sensitive configuration details (like database credentials) from environment variables instead of a static configuration file.

2. Secure Management of Sensitive Information:

• For local development, introduce a .env file to store these variables. This file should not be committed to the version control system (Git), and a .env.example file should be provided as a template.
• For Docker-based deployments, utilize the env_file directive in the docker-compose.yml to inject these variables into the container.
• For Kubernetes deployments, leverage ConfigMaps for non-sensitive data and Secrets for sensitive data. The environment variables will be injected into the Pods based on these resources.

3. Code Changes:

• Use Go's standard os.Getenv function to read environment variables.
• Implement a fallback mechanism to read from the configuration file if the environment variable is not set, ensuring backward compatibility during the transition phase.

4. Documentation and Guidelines:

• Update the project documentation to instruct developers and users on how to set up their environment variables in different scenarios (local development, Docker, Kubernetes).
• Provide guidelines on the creation and management of .env files, Docker Compose configurations, and Kubernetes manifests.

5. Security Considerations:

• Emphasize the non-inclusion of .env files in version control to prevent accidental exposure of sensitive data.
• In Kubernetes setup, recommend using encrypted channels (like TLS) for transmitting sensitive data and proper access control on Secrets.

6. Sample Implementation (Go Code Snippet):

dbUser := os.Getenv("DB_USER")
if dbUser == "" {
    dbUser = defaultConfig.DBUser // Fallback to default configuration
}
// Similar approach for other sensitive variables...

7. Benefits:

• Security: Reduces the risk of sensitive data exposure.
• Flexibility: Easier to manage different configurations for various deployment environments.
• Best Practices: Aligns with industry standards for configuration management.

Action Items:

• Update application code to read environment variables.
• Create .env.example file.
• Update Docker Compose and Kubernetes configuration samples.
• Revise and enhance documentation.
• Communicate changes to the team and contributors.

This change aims to enhance the security and flexibility of OpenIM's configuration management, aligning it with best practices in the industry.

Proposed Solution:

For Source Code Deployment:

1. Environment Variables Setup:
   • Users should set the required environment variables in their local environment.
   • This includes DB_USER, DB_PASSWORD, DB_HOST, and DB_NAME.
   • The application will directly read these environment variables.

For Docker Deployment:

1. Utilizing .env File:

   • Provide a .env.example file in the repository, which users can copy to create their own .env file.
   • The .env file should include variables like DB_USER, DB_PASSWORD, DB_HOST, and DB_NAME.

2. Docker Compose Configuration:

   • Update the docker-compose.yml to include services for both the MySQL database and the Go application (goapp).
   • The environment variables from the .env file should be mapped to both the MySQL service and the Go application service in docker-compose.yml.

3. Sample docker-compose.yml:

   version: '3'
   services:
     mysql:
       image: mysql:5.7
       ports:
         - "3306:3306"
       environment:
         MYSQL_ROOT_PASSWORD: ${DB_PASSWORD}
         MYSQL_DATABASE: ${DB_NAME}
         MYSQL_USER: ${DB_USER}
         MYSQL_PASSWORD: ${DB_PASSWORD}
       volumes:
         - db-data:/var/lib/mysql
   
     goapp:
       build: .
       environment:
         DB_USER: ${DB_USER}
         DB_PASSWORD: ${DB_PASSWORD}
         DB_HOST: mysql
         DB_NAME: ${DB_NAME}
       depends_on:
         - mysql
   
   volumes:
     db-data:
   

Initialization Script:

• Implement a make init script (or enhance the existing one) to generate both config.yaml and .env files, facilitating the configuration process.
• This script should help generate and populate the .env file with default values, which users can then modify as needed.

For Upgrades:

• Instruct users to use docker-compose down to stop the current running containers.
• After stopping, users can modify the .env file with new account information or other configurations.
• Then, users can proceed with the upgrade process, possibly using docker-compose up -d to start the services with the new configurations.

Benefits:

• This approach ensures a unified and secure method of configuring environment variables for different deployment scenarios.
• Enhances the flexibility and scalability of the application deployment, making it easier for users to manage and update configurations.
• Aligns with industry best practices for environment variable management in both development and production environments.

Action Items:

• Update docker-compose.yml to reflect the new configuration strategy.
• Modify or create a make init script for easy initialization of configuration files.
• Update documentation to guide users through the new configuration process.
• Communicate these changes clearly to the existing user base to ensure a smooth transition.

This issue aims to streamline the configuration process for both source code and Docker deployments, enhancing user experience and project maintainability.