desk-moloni/PHP_MIGRATION_ROLLBACK_STRATEGY.md

🔙 PHP MIGRATION ROLLBACK STRATEGY

desk-moloni: Emergency Recovery and Rollback Plan

Document Owner: System Development Agent
Critical Priority: IMMEDIATE ACCESS during deployment
Last Updated: 2025-09-12
Emergency Contacts: See escalation matrix

---

🚨 EMERGENCY ROLLBACK OVERVIEW

When to Execute Rollback

IMMEDIATE ROLLBACK TRIGGERS:

• 🔴 Critical System Failure - Core functionality broken
• 🔴 Security Breach - New vulnerabilities detected
• 🔴 Data Corruption - Database integrity compromised
• 🔴 Performance Degradation - >50% performance loss
• 🔴 API Integration Failure - External services not accessible
• 🔴 User-Blocking Issues - System unusable for end users

Rollback Decision Matrix

Issue Severity	Response Time	Approval Required	Rollback Type
CRITICAL	<15 minutes	CTO Only	Full Rollback
HIGH	<30 minutes	Technical Director	Partial Rollback
MEDIUM	<60 minutes	Development Lead	Code Rollback Only
LOW	<4 hours	Team Consensus	Fix Forward

---

⚡ IMMEDIATE ROLLBACK (< 15 minutes)

Emergency Rollback Command Sequence

#!/bin/bash
# EMERGENCY_ROLLBACK.sh - Execute immediately on critical failure

echo "🚨 EMERGENCY ROLLBACK INITIATED $(date)"
echo "User: $(whoami) | Host: $(hostname)"

# 1. IMMEDIATE GIT ROLLBACK (30 seconds)
echo "Step 1: Git rollback..."
git checkout main
git reset --hard HEAD~1  # Or specific commit hash
git push origin main --force

# 2. DATABASE ROLLBACK (2 minutes)
echo "Step 2: Database rollback..."
mysql -u root -p desk_moloni < /backups/pre_migration_backup_$(date +%Y%m%d).sql

# 3. SERVICE RESTART (1 minute)
echo "Step 3: Service restart..."
systemctl restart apache2
systemctl restart php8.0-fpm  # Rollback to PHP 8.0
systemctl restart mysql

# 4. IMMEDIATE VERIFICATION (30 seconds)
echo "Step 4: Quick verification..."
curl -f http://localhost/desk_moloni/health_check.php || echo "❌ HEALTH CHECK FAILED"

echo "✅ EMERGENCY ROLLBACK COMPLETED $(date)"
echo "🔍 CHECK LOGS: tail -f /var/log/apache2/error.log"

Critical File Rollback Locations

# Keep these paths updated with actual backup locations
BACKUP_ROOT="/backups/php_migration_$(date +%Y%m%d)"
CODE_BACKUP="$BACKUP_ROOT/codebase_backup.tar.gz"
DB_BACKUP="$BACKUP_ROOT/database_backup.sql"  
CONFIG_BACKUP="$BACKUP_ROOT/config_backup.tar.gz"

---

🗂️ ROLLBACK PREPARATION CHECKLIST

Pre-Migration Backup Creation (Day 17 - Before Production)

1. Complete Codebase Backup

#!/bin/bash
# create_codebase_backup.sh

BACKUP_DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/backups/php_migration_$BACKUP_DATE"
mkdir -p $BACKUP_DIR

echo "Creating complete codebase backup..."

# Full project backup
tar -czf "$BACKUP_DIR/codebase_backup.tar.gz" \
    --exclude='.git' \
    --exclude='vendor' \
    --exclude='node_modules' \
    --exclude='tests/.phpunit.cache' \
    /path/to/desk-moloni/

# Configuration files
tar -czf "$BACKUP_DIR/config_backup.tar.gz" \
    /etc/apache2/sites-available/desk-moloni.conf \
    /etc/php/*/apache2/php.ini \
    /etc/mysql/mysql.conf.d/mysqld.cnf

# Document backup details
echo "Backup created: $BACKUP_DATE" > "$BACKUP_DIR/backup_info.txt"
echo "PHP Version: $(php -v | head -n 1)" >> "$BACKUP_DIR/backup_info.txt"
echo "Commit Hash: $(git rev-parse HEAD)" >> "$BACKUP_DIR/backup_info.txt"

echo "✅ Codebase backup completed: $BACKUP_DIR"

2. Database Backup with Verification

#!/bin/bash  
# create_database_backup.sh

BACKUP_DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/backups/php_migration_$BACKUP_DATE"
DB_NAME="desk_moloni"

echo "Creating database backup with verification..."

# Full database backup
mysqldump -u root -p \
    --single-transaction \
    --lock-tables=false \
    --routines \
    --triggers \
    --events \
    $DB_NAME > "$BACKUP_DIR/database_backup.sql"

# Verify backup integrity
echo "Verifying backup integrity..."
mysql -u root -p -e "CREATE DATABASE test_restore_$$"
mysql -u root -p test_restore_$$ < "$BACKUP_DIR/database_backup.sql"

if [ $? -eq 0 ]; then
    echo "✅ Database backup verified successfully"
    mysql -u root -p -e "DROP DATABASE test_restore_$$"
else
    echo "❌ Database backup verification FAILED"
    exit 1
fi

# Create backup metadata
echo "Database: $DB_NAME" > "$BACKUP_DIR/database_info.txt"
echo "Backup Date: $BACKUP_DATE" >> "$BACKUP_DIR/database_info.txt"
echo "Table Count: $(mysql -u root -p $DB_NAME -e 'SHOW TABLES' | wc -l)" >> "$BACKUP_DIR/database_info.txt"
echo "Data Size: $(du -sh $BACKUP_DIR/database_backup.sql)" >> "$BACKUP_DIR/database_info.txt"

echo "✅ Database backup completed and verified"

3. System Configuration Backup

#!/bin/bash
# create_system_backup.sh

BACKUP_DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="/backups/php_migration_$BACKUP_DATE"

echo "Creating system configuration backup..."

# Apache configuration
mkdir -p "$BACKUP_DIR/apache"
cp -r /etc/apache2/sites-available/ "$BACKUP_DIR/apache/"
cp -r /etc/apache2/sites-enabled/ "$BACKUP_DIR/apache/"

# PHP configuration (for all versions)
mkdir -p "$BACKUP_DIR/php"
cp -r /etc/php/ "$BACKUP_DIR/php/"

# MySQL configuration
mkdir -p "$BACKUP_DIR/mysql"
cp /etc/mysql/mysql.conf.d/mysqld.cnf "$BACKUP_DIR/mysql/"

# Cron jobs
crontab -l > "$BACKUP_DIR/crontab_backup.txt" 2>/dev/null || echo "No crontab found"

# System info snapshot
echo "System backup created: $BACKUP_DATE" > "$BACKUP_DIR/system_info.txt"
echo "PHP Version: $(php -v | head -n 1)" >> "$BACKUP_DIR/system_info.txt"
echo "Apache Version: $(apache2 -v | head -n 1)" >> "$BACKUP_DIR/system_info.txt"
echo "MySQL Version: $(mysql --version)" >> "$BACKUP_DIR/system_info.txt"
echo "OS Info: $(lsb_release -a)" >> "$BACKUP_DIR/system_info.txt"

echo "✅ System configuration backup completed"

---

🔄 ROLLBACK EXECUTION PROCEDURES

Level 1: Code-Only Rollback (5-10 minutes)

Use When: Code issues, no database changes affected

#!/bin/bash
# code_rollback.sh

echo "🔄 Executing Code-Only Rollback..."

# 1. Git rollback to last stable commit
git checkout main
git log --oneline -5  # Show recent commits
read -p "Enter commit hash to rollback to: " COMMIT_HASH

git reset --hard $COMMIT_HASH
git clean -fd  # Remove untracked files

# 2. Restore vendor dependencies
composer install --no-dev --optimize-autoloader

# 3. Clear caches
rm -rf tests/.phpunit.cache
php -r "opcache_reset();" 2>/dev/null || true

# 4. Restart services
systemctl reload apache2

echo "✅ Code rollback completed"

Level 2: Partial System Rollback (15-20 minutes)

Use When: Configuration changes need reverting

#!/bin/bash
# partial_rollback.sh

BACKUP_DIR="/backups/php_migration_$(date +%Y%m%d)"

echo "🔄 Executing Partial System Rollback..."

# 1. Code rollback (from Level 1)
./code_rollback.sh

# 2. Restore PHP configuration
systemctl stop apache2
tar -xzf "$BACKUP_DIR/config_backup.tar.gz" -C /

# 3. Switch PHP version back (if needed)
a2dismod php8.4
a2enmod php8.0
systemctl start apache2

# 4. Verify services
systemctl status apache2
systemctl status mysql

echo "✅ Partial rollback completed"

Level 3: Full System Rollback (30-45 minutes)

Use When: Database changes need reverting

#!/bin/bash
# full_rollback.sh

BACKUP_DIR="/backups/php_migration_$(date +%Y%m%d)"

echo "🔄 Executing Full System Rollback..."

# 1. Stop all services
systemctl stop apache2
systemctl stop mysql

# 2. Database rollback
systemctl start mysql
mysql -u root -p -e "DROP DATABASE IF EXISTS desk_moloni_rollback_temp"
mysql -u root -p -e "CREATE DATABASE desk_moloni_rollback_temp"
mysql -u root -p desk_moloni_rollback_temp < "$BACKUP_DIR/database_backup.sql"

# Verify database restore
if mysql -u root -p desk_moloni_rollback_temp -e "SELECT COUNT(*) FROM tbldeskmoloni_config" >/dev/null 2>&1; then
    mysql -u root -p -e "DROP DATABASE desk_moloni"
    mysql -u root -p -e "RENAME DATABASE desk_moloni_rollback_temp TO desk_moloni" 2>/dev/null || {
        mysql -u root -p -e "CREATE DATABASE desk_moloni"
        mysqldump -u root -p desk_moloni_rollback_temp | mysql -u root -p desk_moloni
        mysql -u root -p -e "DROP DATABASE desk_moloni_rollback_temp"
    }
    echo "✅ Database rollback successful"
else
    echo "❌ Database rollback verification failed"
    exit 1
fi

# 3. Code and configuration rollback
./partial_rollback.sh

# 4. Full service restart
systemctl start mysql
systemctl start apache2

echo "✅ Full system rollback completed"

---

📊 ROLLBACK VERIFICATION PROCEDURES

Immediate Health Checks (< 2 minutes)

#!/bin/bash
# rollback_verification.sh

echo "🔍 Running rollback verification checks..."

# 1. Basic connectivity
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost/desk_moloni/)
if [ "$HTTP_STATUS" = "200" ]; then
    echo "✅ HTTP connectivity working"
else
    echo "❌ HTTP connectivity failed: $HTTP_STATUS"
fi

# 2. Database connectivity  
mysql -u root -p desk_moloni -e "SELECT 1" >/dev/null 2>&1
if [ $? -eq 0 ]; then
    echo "✅ Database connectivity working"
else
    echo "❌ Database connectivity failed"
fi

# 3. PHP functionality
php -r "echo 'PHP working: ' . PHP_VERSION . PHP_EOL;"

# 4. Critical tables exist
TABLES=(
    "tbldeskmoloni_config"
    "tbldeskmoloni_sync_log"
    "tbldeskmoloni_mapping"
    "tbldeskmoloni_sync_queue"
)

for table in "${TABLES[@]}"; do
    if mysql -u root -p desk_moloni -e "SELECT 1 FROM $table LIMIT 1" >/dev/null 2>&1; then
        echo "✅ Table $table exists and accessible"
    else
        echo "❌ Table $table missing or inaccessible"
    fi
done

echo "🔍 Verification completed"

Extended Functionality Testing (10-15 minutes)

#!/bin/bash
# extended_verification.sh

echo "🔍 Running extended functionality verification..."

# 1. Admin dashboard access
curl -s "http://localhost/desk_moloni/admin" | grep -q "Dashboard" && echo "✅ Admin dashboard accessible" || echo "❌ Admin dashboard failed"

# 2. API endpoints
curl -s "http://localhost/desk_moloni/api/health" | grep -q "ok" && echo "✅ API endpoints responding" || echo "❌ API endpoints failed"

# 3. Database operations
php -r "
    \$ci = &get_instance();
    \$ci->load->model('desk_moloni/desk_moloni_config_model');
    \$config = \$ci->desk_moloni_config_model->get('api_base_url');
    echo \$config ? '✅ Database operations working' : '❌ Database operations failed';
    echo PHP_EOL;
"

# 4. Queue processing
php modules/desk_moloni/tests/contract/test_queue_processing.php --verify-only

echo "🔍 Extended verification completed"

---

📞 COMMUNICATION PLAN DURING ROLLBACK

Immediate Notifications (< 5 minutes)

# Stakeholder notification script
echo "🚨 ROLLBACK INITIATED: $(date)" | mail -s "CRITICAL: PHP Migration Rollback" stakeholders@company.com

Communication Template

SUBJECT: [CRITICAL] PHP Migration Rollback in Progress - desk-moloni

TIMELINE: $(date)
STATUS: Rollback in progress
ESTIMATED RESTORATION: [15/30/45] minutes
CAUSE: [Brief description]

ACTIONS TAKEN:
- System backed up
- Rollback initiated
- Services being restored

NEXT UPDATE: In [15] minutes

Technical Team

Post-Rollback Communication

SUBJECT: [RESOLVED] PHP Migration Rollback Completed - desk-moloni  

STATUS: System restored to previous stable state
ROLLBACK COMPLETED: $(date)
SERVICES: All operational
IMPACT: [Duration] of service disruption

ROOT CAUSE: [Detailed explanation]
LESSONS LEARNED: [Key findings]
NEXT STEPS: [Future plan]

Technical Team

---

🔍 POST-ROLLBACK ANALYSIS

Immediate Investigation Checklist

• Identify Root Cause - What triggered the rollback?
• Log Analysis - Review all error logs during migration
• Performance Metrics - Compare before/during/after metrics
• Data Integrity Check - Verify no data corruption occurred
• Security Assessment - Ensure no security vulnerabilities introduced
• Stakeholder Impact - Document business impact and user effect

Rollback Report Template

# PHP Migration Rollback Report

## Incident Summary
- **Date/Time:** [DateTime]
- **Duration:** [Duration]
- **Rollback Type:** [Code/Partial/Full]
- **Root Cause:** [Cause]

## Timeline
- Migration Start: [Time]
- Issue Detected: [Time]  
- Rollback Initiated: [Time]
- System Restored: [Time]

## Impact Assessment
- **Users Affected:** [Number]
- **Services Impacted:** [List]
- **Data Loss:** [None/Description]
- **Business Impact:** [Description]

## Technical Analysis
- **Failed Components:** [List]
- **Error Messages:** [Key errors]
- **Performance Impact:** [Metrics]

## Actions Taken
- [List all actions during rollback]

## Lessons Learned
- [Key findings]
- [Improvement opportunities]

## Recommendations
- [Future prevention measures]
- [Process improvements]

## Approval
Technical Director: ________________
Date: ________________

---

🛡️ ROLLBACK PREVENTION STRATEGIES

Pre-Deployment Prevention

1. Comprehensive Testing - Ensure all tests pass
2. Staging Validation - Complete staging environment testing
3. Gradual Rollout - Consider blue-green deployment
4. Monitoring Setup - Comprehensive monitoring before deployment
5. Team Readiness - Ensure all team members are available

Early Detection Systems

# monitoring_setup.sh - Deploy before migration
echo "Setting up migration monitoring..."

# 1. Performance monitoring
echo "* * * * * curl -s http://localhost/desk_moloni/health_check.php || echo 'ALERT: Health check failed' | mail -s 'Migration Alert' admin@company.com" | crontab

# 2. Error log monitoring  
tail -f /var/log/apache2/error.log | grep -E "(FATAL|ERROR)" --line-buffered | while read line; do
    echo "MIGRATION ERROR: $line" | mail -s "Migration Error Alert" admin@company.com
done &

# 3. Database monitoring
mysql -u root -p desk_moloni -e "SELECT COUNT(*) FROM tbldeskmoloni_config" || echo "DB Alert" | mail admin@company.com

---

🔧 ROLLBACK TESTING PLAN

Pre-Migration Rollback Testing (Day 16)

#!/bin/bash
# test_rollback_procedures.sh

echo "🧪 Testing rollback procedures..."

# 1. Create test backup
./create_codebase_backup.sh
./create_database_backup.sh

# 2. Make small test change
echo "<!-- Test change $(date) -->" >> index.html

# 3. Test code rollback
./code_rollback.sh

# 4. Verify rollback worked
if ! grep -q "Test change" index.html; then
    echo "✅ Code rollback test passed"
else
    echo "❌ Code rollback test failed"
fi

# 5. Test database rollback with temporary database
mysql -u root -p -e "CREATE DATABASE rollback_test"
mysql -u root -p rollback_test < /backups/php_migration_*/database_backup.sql
mysql -u root -p -e "DROP DATABASE rollback_test"

echo "✅ Rollback procedures tested successfully"

---

📋 ROLLBACK DECISION FLOWCHART

Migration Issue Detected
           |
           ▼
    Is it Critical?
      /         \
   YES           NO
    |             |
    ▼             ▼
Immediate      Attempt Fix
Rollback        (30 min)
    |             |
    ▼             ▼
Execute        Fix Success?
Level 3          /      \
Rollback      YES      NO
    |          |        |
    ▼          ▼        ▼
Notify     Continue   Rollback
Team       Monitoring  Level 1/2
    |          |        |
    ▼          ▼        ▼
Post-Rollback Success  Verify
Analysis   Report    Success

---

📚 ROLLBACK KNOWLEDGE BASE

Common Issues and Solutions

Issue: "Database connection failed after rollback"

Solution:

# Check MySQL service
systemctl status mysql
systemctl restart mysql

# Verify database exists
mysql -u root -p -e "SHOW DATABASES;" | grep desk_moloni

# Test connection with application user
mysql -u desk_moloni_user -p desk_moloni -e "SELECT 1"

Issue: "PHP version mismatch after rollback"

Solution:

# Check current PHP version
php -v

# Switch PHP version
a2dismod php8.4
a2enmod php8.0
systemctl restart apache2

# Verify correct version is active
php -v

Issue: "Composer dependencies conflict"

Solution:

# Clear composer cache
composer clear-cache

# Remove vendor directory
rm -rf vendor/

# Restore from backup
tar -xzf /backups/*/codebase_backup.tar.gz

# Or reinstall fresh
composer install --no-dev --optimize-autoloader

Emergency Contact Information

PRIMARY CONTACTS:
- Development Lead: [Phone] / [Email]
- DevOps Engineer: [Phone] / [Email]  
- Database Administrator: [Phone] / [Email]

ESCALATION CONTACTS:
- Technical Director: [Phone] / [Email]
- CTO: [Phone] / [Email]
- Infrastructure Team: [Phone] / [Email]

VENDOR SUPPORT:
- Hosting Provider: [Support Number]
- Database Support: [Support Number]

---

✅ FINAL ROLLBACK CHECKLIST

Pre-Migration Preparation

• Rollback scripts tested and verified
• Complete backups created and validated
• Emergency contact list updated
• Team trained on rollback procedures
• Monitoring systems configured
• Communication templates prepared

During Migration Monitoring

• Rollback scripts accessible and ready
• Team members on standby
• Monitoring dashboards active
• Communication channels open
• Decision makers available

Post-Rollback Actions

• System functionality verified
• Performance metrics restored
• All services operational
• Users notified of restoration
• Incident documentation completed
• Lessons learned captured

---

REMEMBER: The best rollback is the one you never need to use. Thorough testing and preparation are your best defense against needing emergency rollbacks.

Document Status: READY FOR USE
Last Tested: [Update after testing]
Next Review: [Update quarterly]

Keep this document easily accessible during migration deployment.