baktainer/README.md

145 lines
5 KiB
Markdown
Raw Normal View History

2025-04-05 13:58:02 +00:00
# baktainer
2025-04-14 09:39:37 -04:00
Easily backup databases running in docker containers.
## Features
- Backup MySQL, PostgreSQL, MongoDB, and SQLite databases
- Run on a schedule using cron expressions
- Backup databases running in docker containers
- Define which databases to backup using docker labels
## Installation
⚠️ **Security Notice**: Baktainer requires Docker socket access which grants significant privileges. Please review [SECURITY.md](SECURITY.md) for important security considerations and recommended mitigations.
2025-04-14 09:39:37 -04:00
```yaml
services:
baktainer:
2025-05-08 22:54:50 -04:00
image: jamez001/baktainer:latest
2025-04-14 09:39:37 -04:00
container_name: baktainer
restart: unless-stopped
volumes:
- ./backups:/backups
- /var/run/docker.sock:/var/run/docker.sock
environment:
- BT_CRON="0 0 * * *" # Backup every day at midnight
- "BT_DOCKER_URL=unix:///var/run/docker.sock"
- BT_THREADS=4
- BT_LOG_LEVEL=info
# Enable if using SSL over tcp
#- BT_SSL = true
#- BT_CA
#- BT_CERT
#- BT_KEY
```
2025-04-05 13:58:02 +00:00
For enhanced security, consider using a Docker socket proxy. See [SECURITY.md](SECURITY.md) for detailed security recommendations.
2025-04-14 09:39:37 -04:00
## Environment Variables
| Variable | Description | Default |
| -------- | ----------- | ------- |
| BT_CRON | Cron expression for scheduling backups | 0 0 * * * |
| BT_THREADS | Number of threads to use for backups | 4 |
| BT_LOG_LEVEL | Log level (debug, info, warn, error) | info |
Major architectural overhaul: dependency injection, monitoring, and operational improvements This commit represents a comprehensive refactoring and enhancement of Baktainer: ## Core Architecture Improvements - Implemented comprehensive dependency injection system with DependencyContainer - Fixed critical singleton instantiation bug that was returning Procs instead of service instances - Replaced problematic Concurrent::FixedThreadPool with custom SimpleThreadPool implementation - Achieved 100% test pass rate (121 examples, 0 failures) after fixing 30+ failing tests ## New Features Implemented ### 1. Backup Rotation & Cleanup (BackupRotation) - Configurable retention policies by age, count, and disk space - Automatic cleanup with comprehensive statistics tracking - Empty directory cleanup and space monitoring ### 2. Backup Encryption (BackupEncryption) - AES-256-CBC and AES-256-GCM encryption support - Key derivation from passphrases or direct key input - Encrypted backup metadata storage ### 3. Operational Monitoring Suite - **Health Check Server**: HTTP endpoints for monitoring (/health, /status, /metrics) - **Web Dashboard**: Real-time monitoring dashboard with auto-refresh - **Prometheus Metrics**: Integration with monitoring systems - **Backup Monitor**: Comprehensive metrics tracking and performance alerts ### 4. Advanced Label Validation (LabelValidator) - Schema-based validation for all 12+ Docker labels - Engine-specific validation rules - Helpful error messages and warnings - Example generation for each database engine ### 5. Multi-Channel Notifications (NotificationSystem) - Support for Slack, Discord, Teams, webhooks, and log notifications - Event-based notifications for backups, failures, warnings, and health issues - Configurable notification thresholds ## Code Organization Improvements - Extracted responsibilities into focused classes: - ContainerValidator: Container validation logic - BackupOrchestrator: Backup workflow orchestration - FileSystemOperations: File I/O with comprehensive error handling - Configuration: Centralized environment variable management - BackupStrategy/Factory: Strategy pattern for database engines ## Testing Infrastructure - Added comprehensive unit and integration tests - Fixed timing-dependent test failures - Added RSpec coverage reporting (94.94% coverage) - Created test factories and fixtures ## Breaking Changes - Container class constructor now requires dependency injection - BackupCommand methods now use keyword arguments - Thread pool implementation changed from Concurrent to SimpleThreadPool ## Configuration New environment variables: - BT_HEALTH_SERVER_ENABLED: Enable health check server - BT_HEALTH_PORT/BT_HEALTH_BIND: Health server configuration - BT_NOTIFICATION_CHANNELS: Comma-separated notification channels - BT_ENCRYPTION_ENABLED/BT_ENCRYPTION_KEY: Backup encryption - BT_RETENTION_DAYS/COUNT: Backup retention policies This refactoring improves maintainability, testability, and adds enterprise-grade monitoring and operational features while maintaining backward compatibility for basic usage. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 22:58:26 -04:00
| BT_COMPRESS | Enable gzip compression for backups | true |
2025-04-14 09:39:37 -04:00
| BT_SSL | Enable SSL for docker connection | false |
| BT_CA | Path to CA certificate | none |
| BT_CERT | Path to client certificate | none |
| BT_KEY | Path to client key | none |
| BT_DOCKER_URL | Docker URL | unix:///var/run/docker.sock |
## Usage
Add labels to your docker containers to specify which databases to backup.
```yaml
services:
db:
image: postgres:17
container_name: my-db
restart: unless-stopped
volumes:
- db:/var/lib/postgresql/data
environment:
POSTGRES_DB: "${DB_BASE:-database}"
POSTGRES_USER: "${DB_USER:-user}"
POSTGRES_PASSWORD: "${DB_PASSWORD:-StrongPassword}"
labels:
2025-04-14 09:49:36 -04:00
- baktainer.backup=true
- baktainer.db.engine=postgres
- baktainer.db.name=my-db
- baktainer.db.user=user
- baktainer.db.password=StrongPassword
- baktainer.name="MyApp"
2025-04-14 09:39:37 -04:00
```
2025-04-14 09:49:36 -04:00
## Possible Values for Labels
| Label | Description |
| ----- | ----------- |
| baktainer.backup | Set to true to enable backup for this container |
| baktainer.db.engine | Database engine (mysql, postgres, mongodb, sqlite) |
| baktainer.db.name | Name of the database to backup |
| baktainer.db.user | Username for the database |
| baktainer.db.password | Password for the database |
| baktainer.name | Name of the application (optional). Determines name of sql dump file. |
Major architectural overhaul: dependency injection, monitoring, and operational improvements This commit represents a comprehensive refactoring and enhancement of Baktainer: ## Core Architecture Improvements - Implemented comprehensive dependency injection system with DependencyContainer - Fixed critical singleton instantiation bug that was returning Procs instead of service instances - Replaced problematic Concurrent::FixedThreadPool with custom SimpleThreadPool implementation - Achieved 100% test pass rate (121 examples, 0 failures) after fixing 30+ failing tests ## New Features Implemented ### 1. Backup Rotation & Cleanup (BackupRotation) - Configurable retention policies by age, count, and disk space - Automatic cleanup with comprehensive statistics tracking - Empty directory cleanup and space monitoring ### 2. Backup Encryption (BackupEncryption) - AES-256-CBC and AES-256-GCM encryption support - Key derivation from passphrases or direct key input - Encrypted backup metadata storage ### 3. Operational Monitoring Suite - **Health Check Server**: HTTP endpoints for monitoring (/health, /status, /metrics) - **Web Dashboard**: Real-time monitoring dashboard with auto-refresh - **Prometheus Metrics**: Integration with monitoring systems - **Backup Monitor**: Comprehensive metrics tracking and performance alerts ### 4. Advanced Label Validation (LabelValidator) - Schema-based validation for all 12+ Docker labels - Engine-specific validation rules - Helpful error messages and warnings - Example generation for each database engine ### 5. Multi-Channel Notifications (NotificationSystem) - Support for Slack, Discord, Teams, webhooks, and log notifications - Event-based notifications for backups, failures, warnings, and health issues - Configurable notification thresholds ## Code Organization Improvements - Extracted responsibilities into focused classes: - ContainerValidator: Container validation logic - BackupOrchestrator: Backup workflow orchestration - FileSystemOperations: File I/O with comprehensive error handling - Configuration: Centralized environment variable management - BackupStrategy/Factory: Strategy pattern for database engines ## Testing Infrastructure - Added comprehensive unit and integration tests - Fixed timing-dependent test failures - Added RSpec coverage reporting (94.94% coverage) - Created test factories and fixtures ## Breaking Changes - Container class constructor now requires dependency injection - BackupCommand methods now use keyword arguments - Thread pool implementation changed from Concurrent to SimpleThreadPool ## Configuration New environment variables: - BT_HEALTH_SERVER_ENABLED: Enable health check server - BT_HEALTH_PORT/BT_HEALTH_BIND: Health server configuration - BT_NOTIFICATION_CHANNELS: Comma-separated notification channels - BT_ENCRYPTION_ENABLED/BT_ENCRYPTION_KEY: Backup encryption - BT_RETENTION_DAYS/COUNT: Backup retention policies This refactoring improves maintainability, testability, and adds enterprise-grade monitoring and operational features while maintaining backward compatibility for basic usage. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 22:58:26 -04:00
| baktainer.compress | Enable gzip compression for this container's backups (true/false). Overrides BT_COMPRESS. |
2025-04-14 09:49:36 -04:00
## Backup Files
The backup files will be stored in the directory specified by the `BT_BACKUP_DIR` environment variable. The files will be named according to the following format:
```
Major architectural overhaul: dependency injection, monitoring, and operational improvements This commit represents a comprehensive refactoring and enhancement of Baktainer: ## Core Architecture Improvements - Implemented comprehensive dependency injection system with DependencyContainer - Fixed critical singleton instantiation bug that was returning Procs instead of service instances - Replaced problematic Concurrent::FixedThreadPool with custom SimpleThreadPool implementation - Achieved 100% test pass rate (121 examples, 0 failures) after fixing 30+ failing tests ## New Features Implemented ### 1. Backup Rotation & Cleanup (BackupRotation) - Configurable retention policies by age, count, and disk space - Automatic cleanup with comprehensive statistics tracking - Empty directory cleanup and space monitoring ### 2. Backup Encryption (BackupEncryption) - AES-256-CBC and AES-256-GCM encryption support - Key derivation from passphrases or direct key input - Encrypted backup metadata storage ### 3. Operational Monitoring Suite - **Health Check Server**: HTTP endpoints for monitoring (/health, /status, /metrics) - **Web Dashboard**: Real-time monitoring dashboard with auto-refresh - **Prometheus Metrics**: Integration with monitoring systems - **Backup Monitor**: Comprehensive metrics tracking and performance alerts ### 4. Advanced Label Validation (LabelValidator) - Schema-based validation for all 12+ Docker labels - Engine-specific validation rules - Helpful error messages and warnings - Example generation for each database engine ### 5. Multi-Channel Notifications (NotificationSystem) - Support for Slack, Discord, Teams, webhooks, and log notifications - Event-based notifications for backups, failures, warnings, and health issues - Configurable notification thresholds ## Code Organization Improvements - Extracted responsibilities into focused classes: - ContainerValidator: Container validation logic - BackupOrchestrator: Backup workflow orchestration - FileSystemOperations: File I/O with comprehensive error handling - Configuration: Centralized environment variable management - BackupStrategy/Factory: Strategy pattern for database engines ## Testing Infrastructure - Added comprehensive unit and integration tests - Fixed timing-dependent test failures - Added RSpec coverage reporting (94.94% coverage) - Created test factories and fixtures ## Breaking Changes - Container class constructor now requires dependency injection - BackupCommand methods now use keyword arguments - Thread pool implementation changed from Concurrent to SimpleThreadPool ## Configuration New environment variables: - BT_HEALTH_SERVER_ENABLED: Enable health check server - BT_HEALTH_PORT/BT_HEALTH_BIND: Health server configuration - BT_NOTIFICATION_CHANNELS: Comma-separated notification channels - BT_ENCRYPTION_ENABLED/BT_ENCRYPTION_KEY: Backup encryption - BT_RETENTION_DAYS/COUNT: Backup retention policies This refactoring improves maintainability, testability, and adds enterprise-grade monitoring and operational features while maintaining backward compatibility for basic usage. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 22:58:26 -04:00
/backups/<date>/<db_name>-<timestamp>.sql.gz
2025-04-14 09:49:36 -04:00
```
Where `<date>` is the date of the backup ('YY-MM-DD' format) `<db_name>` is the name provided by baktainer.name, or the name of the database, `<timestamp>` is the unix timestamp of the backup.
2025-04-14 09:49:36 -04:00
Major architectural overhaul: dependency injection, monitoring, and operational improvements This commit represents a comprehensive refactoring and enhancement of Baktainer: ## Core Architecture Improvements - Implemented comprehensive dependency injection system with DependencyContainer - Fixed critical singleton instantiation bug that was returning Procs instead of service instances - Replaced problematic Concurrent::FixedThreadPool with custom SimpleThreadPool implementation - Achieved 100% test pass rate (121 examples, 0 failures) after fixing 30+ failing tests ## New Features Implemented ### 1. Backup Rotation & Cleanup (BackupRotation) - Configurable retention policies by age, count, and disk space - Automatic cleanup with comprehensive statistics tracking - Empty directory cleanup and space monitoring ### 2. Backup Encryption (BackupEncryption) - AES-256-CBC and AES-256-GCM encryption support - Key derivation from passphrases or direct key input - Encrypted backup metadata storage ### 3. Operational Monitoring Suite - **Health Check Server**: HTTP endpoints for monitoring (/health, /status, /metrics) - **Web Dashboard**: Real-time monitoring dashboard with auto-refresh - **Prometheus Metrics**: Integration with monitoring systems - **Backup Monitor**: Comprehensive metrics tracking and performance alerts ### 4. Advanced Label Validation (LabelValidator) - Schema-based validation for all 12+ Docker labels - Engine-specific validation rules - Helpful error messages and warnings - Example generation for each database engine ### 5. Multi-Channel Notifications (NotificationSystem) - Support for Slack, Discord, Teams, webhooks, and log notifications - Event-based notifications for backups, failures, warnings, and health issues - Configurable notification thresholds ## Code Organization Improvements - Extracted responsibilities into focused classes: - ContainerValidator: Container validation logic - BackupOrchestrator: Backup workflow orchestration - FileSystemOperations: File I/O with comprehensive error handling - Configuration: Centralized environment variable management - BackupStrategy/Factory: Strategy pattern for database engines ## Testing Infrastructure - Added comprehensive unit and integration tests - Fixed timing-dependent test failures - Added RSpec coverage reporting (94.94% coverage) - Created test factories and fixtures ## Breaking Changes - Container class constructor now requires dependency injection - BackupCommand methods now use keyword arguments - Thread pool implementation changed from Concurrent to SimpleThreadPool ## Configuration New environment variables: - BT_HEALTH_SERVER_ENABLED: Enable health check server - BT_HEALTH_PORT/BT_HEALTH_BIND: Health server configuration - BT_NOTIFICATION_CHANNELS: Comma-separated notification channels - BT_ENCRYPTION_ENABLED/BT_ENCRYPTION_KEY: Backup encryption - BT_RETENTION_DAYS/COUNT: Backup retention policies This refactoring improves maintainability, testability, and adds enterprise-grade monitoring and operational features while maintaining backward compatibility for basic usage. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 22:58:26 -04:00
By default, backups are compressed with gzip. To disable compression, set `BT_COMPRESS=false` or add `baktainer.compress=false` label to specific containers.
## Testing
The project includes comprehensive test coverage with both unit and integration tests.
### Running Tests
```bash
# Run all tests
cd app && bundle exec rspec
# Run tests with coverage report
cd app && COVERAGE=true bundle exec rspec
# Run only unit tests
cd app && bundle exec rspec spec/unit/
# Run only integration tests (requires Docker)
cd app && bundle exec rspec spec/integration/
```
### Test Coverage
- **Line Coverage**: 94.94% (150/158 lines)
- **Branch Coverage**: 71.11% (32/45 branches)
- Tests cover all database engines, container discovery, error handling, and backup workflows
- Integration tests validate full backup operations with real Docker containers
### Test Commands
```bash
# Quick unit tests
bin/test
# All tests with coverage
bin/test --all --coverage
# Integration tests with setup/cleanup
bin/test --integration --setup --cleanup
```
2025-04-14 09:49:36 -04:00
## Roadmap
- [x] Add support for SQLite backups
- [x] Add support for MongoDB backups
- [x] Add support for MySQL backups
- [x] Add support for PostgreSQL backups
- [x] Add support for cron scheduling
- [x] Add support for Docker labels to specify which databases to backup
- [x] Add support for Docker socket
- [x] Add support for Docker API over TCP
- [x] Add support for Docker API over SSL
- [x] Add support for Docker API over HTTP
- [x] Add support for Docker API over HTTPS
- [x] Add support for Docker API over Unix socket
- [x] Add comprehensive test coverage (94.94% line coverage)
2025-04-14 09:49:36 -04:00
- [ ] Add individual hook for completed backups
- [ ] Add hook for fullly completed backups
2025-05-08 22:54:50 -04:00
- [ ] Optionally limit time for each backup