# MyCloudVault System Overview

Complete documentation index and system architecture overview for MyCloudVault backup system.

## 📚 Documentation Index

### Core Documentation

1. **[README.md](README.md)** - Project overview and quick start guide
2. **[SERVER_SETUP.md](SERVER_SETUP.md)** - Development server setup instructions
3. **[DEPLOYMENT_GUIDE.md](DEPLOYMENT_GUIDE.md)** - Production deployment guide

### API Documentation

4. **[MOBILE_API_REFERENCE.md](MOBILE_API_REFERENCE.md)** - Complete REST API reference
5. **[BACKUP_UPLOAD_API.md](BACKUP_UPLOAD_API.md)** - Backup upload endpoints
6. **[BACKUP_RESTORE_API.md](BACKUP_RESTORE_API.md)** - Backup restore endpoints
7. **[COMMUNICATION_API.md](COMMUNICATION_API.md)** - Device communication API

### Mobile Integration

8. **[ANDROID_INTEGRATION.md](ANDROID_INTEGRATION.md)** - Android client guide (Kotlin)
9. **[IOS_INTEGRATION.md](IOS_INTEGRATION.md)** - iOS client guide (Swift)

### Technical Guides

10. **[BACKUP_ENCRYPTION.md](BACKUP_ENCRYPTION.md)** - Encryption implementation
11. **[BACKUP_VERIFICATION.md](BACKUP_VERIFICATION.md)** - Integrity verification

---

## 🏗️ System Architecture

### High-Level Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                     Mobile Clients                          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐     │
│  │   Android    │  │     iOS      │  │   Desktop    │     │
│  │   (Kotlin)   │  │   (Swift)    │  │  (Electron)  │     │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘     │
└─────────┼──────────────────┼──────────────────┼────────────┘
          │                  │                  │
          │    HTTPS/REST API (TLS 1.2+)       │
          │                  │                  │
┌─────────▼──────────────────▼──────────────────▼────────────┐
│                    Nginx Reverse Proxy                      │
│  • SSL/TLS Termination                                      │
│  • Rate Limiting (10 req/s API, 2 req/s uploads)           │
│  • Load Balancing                                           │
│  • Security Headers                                         │
└─────────┬───────────────────────────────────────────────────┘
          │
┌─────────▼───────────────────────────────────────────────────┐
│              Node.js Application (Express)                  │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  API Routes (vite-plugin-api)                        │  │
│  │  • /api/devices/* - Device management                │  │
│  │  • /api/backup/* - Backup operations                 │  │
│  │  • /api/encryption/* - Key management                │  │
│  │  • /api/communication/* - Device messaging           │  │
│  └──────────────────────────────────────────────────────┘  │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  Business Logic                                       │  │
│  │  • Encryption (AES-256-GCM)                          │  │
│  │  • Verification (SHA-256 + CRC32)                    │  │
│  │  • Chunked Upload (5MB chunks)                       │  │
│  └──────────────────────────────────────────────────────┘  │
└─────────┬───────────────────────────────────────────────────┘
          │
          ├─────────────────┬─────────────────┐
          │                 │                 │
┌─────────▼─────┐  ┌────────▼────────┐  ┌────▼──────────┐
│  MySQL 8.0    │  │  File Storage   │  │  PM2 Process  │
│  (Drizzle ORM)│  │  /var/mycloudvault│  │  Manager      │
│               │  │  • /uploads     │  │  • Clustering │
│  • devices    │  │  • /backups     │  │  • Auto-restart│
│  • backups    │  │                 │  │  • Monitoring │
│  • servers    │  │                 │  │               │
└───────────────┘  └─────────────────┘  └───────────────┘
```

### Component Breakdown

#### 1. Frontend (React + Vite)

**Technology Stack:**
- React 19 with TypeScript
- Vite build tool
- Tailwind CSS + shadcn UI
- React Router for navigation

**Key Features:**
- Dashboard for backup management
- Device pairing interface
- Backup history and status
- Real-time progress tracking

**File Structure:**
```
src/
├── pages/
│   ├── index.tsx - Landing page
│   └── dashboard/ - Dashboard views
├── components/ui/ - shadcn components
├── layouts/ - Page layouts
└── lib/ - Utilities and API client
```

#### 2. Backend (Node.js + Express)

**Technology Stack:**
- Node.js 20.x LTS
- Express.js framework
- vite-plugin-api for routing
- Drizzle ORM for database
- Multer for file uploads

**API Structure:**
```
src/server/api/
├── devices/
│   ├── GET.ts - List devices
│   ├── pair/POST.ts - Pair new device
│   └── [deviceId]/GET.ts - Get device details
├── backup/
│   ├── start/POST.ts - Start backup session
│   ├── upload-chunk/POST.ts - Upload chunk
│   ├── upload-encrypted/POST.ts - Upload encrypted
│   ├── verify/POST.ts - Verify backup
│   ├── list/GET.ts - List backups
│   ├── download/[backupId]/GET.ts - Download
│   └── delete/[backupId]/DELETE.ts - Delete
├── encryption/
│   └── generate-key/POST.ts - Generate key
└── communication/
    ├── send/POST.ts - Send message
    └── messages/GET.ts - Get messages
```

#### 3. Database Schema

**Tables:**

```sql
-- Devices table
CREATE TABLE devices (
  id INT PRIMARY KEY AUTO_INCREMENT,
  device_id VARCHAR(255) UNIQUE NOT NULL,
  device_name VARCHAR(255) NOT NULL,
  device_model VARCHAR(255),
  os_version VARCHAR(50),
  status ENUM('active', 'inactive', 'pending') DEFAULT 'pending',
  api_key VARCHAR(255) UNIQUE,
  last_seen TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Backups table
CREATE TABLE backups (
  id INT PRIMARY KEY AUTO_INCREMENT,
  backup_id VARCHAR(255) UNIQUE NOT NULL,
  device_id VARCHAR(255) NOT NULL,
  backup_type VARCHAR(50) NOT NULL,
  status ENUM('pending', 'in_progress', 'completed', 'failed') DEFAULT 'pending',
  size BIGINT DEFAULT 0,
  uploaded_size BIGINT DEFAULT 0,
  file_count INT DEFAULT 0,
  file_path TEXT,
  encryption_enabled BOOLEAN DEFAULT FALSE,
  encryption_algorithm VARCHAR(50),
  encryption_iv VARCHAR(255),
  encryption_auth_tag VARCHAR(255),
  verification_status ENUM('pending', 'passed', 'failed'),
  verification_checksum VARCHAR(255),
  verification_crc32 VARCHAR(50),
  started_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  completed_at TIMESTAMP NULL,
  FOREIGN KEY (device_id) REFERENCES devices(device_id)
);

-- Servers table (for multi-server deployments)
CREATE TABLE servers (
  id INT PRIMARY KEY AUTO_INCREMENT,
  server_id VARCHAR(255) UNIQUE NOT NULL,
  server_name VARCHAR(255) NOT NULL,
  server_url VARCHAR(255) NOT NULL,
  status ENUM('active', 'inactive') DEFAULT 'active',
  last_heartbeat TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
```

#### 4. Security Features

**Authentication:**
- Device pairing with 6-digit codes
- API key authentication (X-API-Key header)
- Device ID validation (X-Device-ID header)

**Encryption:**
- Algorithm: AES-256-GCM
- Key size: 256 bits
- Random IV per backup
- Authentication tags for integrity

**Verification:**
- SHA-256 checksums for file integrity
- CRC32 for quick corruption detection
- Automatic verification after upload

**Network Security:**
- TLS 1.2+ required
- Rate limiting (10 req/s API, 2 req/s uploads)
- CORS protection
- Security headers (HSTS, X-Frame-Options, etc.)

---

## 🔄 Data Flow

### Backup Upload Flow

```
1. Device Pairing
   Mobile App → POST /api/devices/pair
   ├─ Generate device_id
   ├─ Validate pairing code
   ├─ Create API key
   └─ Return credentials

2. Start Backup Session
   Mobile App → POST /api/backup/start
   ├─ Authenticate device
   ├─ Create backup record
   ├─ Generate backup_id
   └─ Return backup session

3. Upload Data (Two Methods)
   
   A. Chunked Upload (Large Files)
      Mobile App → POST /api/backup/upload-chunk (multiple)
      ├─ Upload 5MB chunks
      ├─ Track progress
      ├─ Reassemble on server
      └─ Update uploaded_size
   
   B. Encrypted Upload (Secure)
      Mobile App → POST /api/backup/upload-encrypted
      ├─ Encrypt locally (AES-256-GCM)
      ├─ Upload encrypted data
      ├─ Store encryption metadata
      └─ Save to /var/mycloudvault/backups

4. Verify Backup
   Mobile App → POST /api/backup/verify
   ├─ Calculate SHA-256 checksum
   ├─ Calculate CRC32
   ├─ Compare with client values
   ├─ Update verification_status
   └─ Return verification result

5. Complete
   ├─ Update status to 'completed'
   ├─ Set completed_at timestamp
   └─ Send success response
```

### Backup Restore Flow

```
1. List Available Backups
   Mobile App → GET /api/backup/list?deviceId=xxx
   └─ Return backup list with metadata

2. Download Backup
   Mobile App → GET /api/backup/download/:backupId
   ├─ Authenticate device
   ├─ Verify ownership
   ├─ Stream file from storage
   └─ Return encrypted/plain data

3. Decrypt Locally (if encrypted)
   Mobile App
   ├─ Retrieve encryption key
   ├─ Decrypt using AES-256-GCM
   ├─ Verify authentication tag
   └─ Extract files

4. Restore Files
   Mobile App
   └─ Write files to device storage
```

---

## 📊 Performance Characteristics

### Upload Performance

**Chunked Upload:**
- Chunk size: 5MB
- Parallel uploads: 1 (sequential for reliability)
- Network efficiency: ~85-90% (overhead from HTTP)
- Resume capability: Yes (track chunk index)

**Encrypted Upload:**
- Encryption overhead: ~5-10% CPU
- Memory usage: ~50MB per upload
- Network efficiency: ~95% (single upload)
- Resume capability: No (single file)

### Storage Efficiency

**Compression:**
- Optional gzip compression before encryption
- Average compression ratio: 2:1 for text, 1.1:1 for media
- CPU overhead: ~10-15%

**Deduplication:**
- Not currently implemented
- Future enhancement: Block-level deduplication

### Database Performance

**Query Optimization:**
- Indexed columns: device_id, backup_id, status
- Connection pooling: 2-10 connections
- Query cache: Enabled for read-heavy operations

**Scaling Limits:**
- Single server: ~100 concurrent devices
- With load balancer: ~1000+ devices
- Database replication: Read replicas for scaling

---

## 🔐 Security Model

### Threat Model

**Protected Against:**
- ✅ Man-in-the-middle attacks (TLS 1.2+)
- ✅ Unauthorized access (API key authentication)
- ✅ Data tampering (SHA-256 + CRC32 verification)
- ✅ Brute force attacks (rate limiting)
- ✅ Data breaches (AES-256-GCM encryption)

**Not Protected Against:**
- ❌ Compromised client devices (client-side security)
- ❌ Stolen API keys (rotate keys regularly)
- ❌ Physical server access (secure data center required)

### Key Management

**Encryption Keys:**
- Generated client-side (mobile app)
- Never transmitted to server
- Stored in device secure storage (Keychain/Keystore)
- User responsible for key backup

**API Keys:**
- Generated server-side during pairing
- Stored in database (hashed)
- Transmitted once during pairing
- Stored in device secure storage

### Compliance Considerations

**GDPR:**
- User data encryption at rest
- Right to deletion (DELETE /api/backup/:backupId)
- Data portability (download backups)

**HIPAA (if applicable):**
- End-to-end encryption
- Audit logging (implement separately)
- Access controls

---

## 🚀 Deployment Options

### Option 1: Single Server (Small Scale)

**Suitable for:**
- Personal use or small teams (< 10 devices)
- Development and testing
- Low budget deployments

**Architecture:**
```
Single VPS/Server
├─ Nginx (reverse proxy)
├─ Node.js (application)
├─ MySQL (database)
└─ Local storage (/var/mycloudvault)
```

**Estimated Cost:**
- VPS: $10-20/month (DigitalOcean, Linode)
- Storage: Included (100GB-500GB)
- Total: ~$15/month

### Option 2: Load Balanced (Medium Scale)

**Suitable for:**
- Small businesses (10-100 devices)
- Production deployments
- High availability requirements

**Architecture:**
```
Load Balancer (Nginx)
├─ App Server 1 (Node.js)
├─ App Server 2 (Node.js)
└─ App Server 3 (Node.js)
    ↓
Shared MySQL Database
    ↓
Shared Storage (NFS/S3)
```

**Estimated Cost:**
- Load balancer: $10/month
- App servers: $30-60/month (3x $10-20)
- Database: $20-40/month (managed MySQL)
- Storage: $20-50/month (S3 or NFS)
- Total: ~$100-150/month

### Option 3: Cloud Native (Large Scale)

**Suitable for:**
- Enterprise deployments (100+ devices)
- Global distribution
- Maximum scalability

**Architecture:**
```
CloudFlare CDN
    ↓
AWS ALB (Load Balancer)
    ↓
ECS/EKS (Container Orchestration)
├─ App Container 1
├─ App Container 2
└─ App Container N (auto-scaling)
    ↓
RDS MySQL (Multi-AZ)
    ↓
S3 (Object Storage)
```

**Estimated Cost:**
- ALB: $20/month
- ECS/EKS: $50-200/month
- RDS: $50-150/month
- S3: $20-100/month (depends on usage)
- CloudFlare: $20/month
- Total: ~$200-500/month

---

## 📈 Monitoring & Observability

### Metrics to Track

**Application Metrics:**
- Request rate (req/s)
- Response time (p50, p95, p99)
- Error rate (%)
- Active connections

**Backup Metrics:**
- Backups per day
- Average backup size
- Upload success rate
- Verification pass rate

**System Metrics:**
- CPU usage (%)
- Memory usage (MB)
- Disk usage (GB)
- Network I/O (MB/s)

### Logging Strategy

**Log Levels:**
- ERROR: Application errors, failed operations
- WARN: Degraded performance, retries
- INFO: Successful operations, state changes
- DEBUG: Detailed debugging information

**Log Destinations:**
- Application logs: `/var/log/mycloudvault/app.log`
- Access logs: `/var/log/nginx/mycloudvault-access.log`
- Error logs: `/var/log/nginx/mycloudvault-error.log`
- Database logs: `/var/log/mysql/error.log`

### Alerting

**Critical Alerts:**
- Service down (health check fails)
- Disk space > 90%
- Error rate > 5%
- Database connection failures

**Warning Alerts:**
- Response time > 2s (p95)
- Memory usage > 80%
- Backup verification failures
- SSL certificate expiring (< 30 days)

---

## 🔧 Maintenance & Operations

### Regular Maintenance Tasks

**Daily:**
- [ ] Check application logs for errors
- [ ] Monitor disk usage
- [ ] Verify backups completed successfully
- [ ] Check system resource usage

**Weekly:**
- [ ] Review security logs
- [ ] Analyze performance metrics
- [ ] Check for failed backups
- [ ] Review rate limiting logs

**Monthly:**
- [ ] Update system packages
- [ ] Rotate and archive logs
- [ ] Test backup restoration
- [ ] Review SSL certificate expiry
- [ ] Database optimization (ANALYZE, OPTIMIZE)

**Quarterly:**
- [ ] Security audit
- [ ] Performance testing
- [ ] Disaster recovery drill
- [ ] Review and update documentation

### Backup Strategy

**Database Backups:**
- Frequency: Daily at 2 AM
- Retention: 30 days
- Method: mysqldump + gzip
- Location: `/var/mycloudvault/db-backups`
- Offsite: Sync to S3 daily

**File Backups:**
- Frequency: Daily at 3 AM
- Retention: 30 days
- Method: rsync incremental
- Location: `/backup/mycloudvault-files`
- Offsite: Sync to S3 daily

**Configuration Backups:**
- Frequency: On change
- Retention: Indefinite (version control)
- Method: Git repository
- Location: GitHub/GitLab

---

## 🎯 Future Enhancements

### Planned Features

**Short Term (1-3 months):**
- [ ] Incremental backups (delta sync)
- [ ] Backup scheduling (automatic backups)
- [ ] Email notifications
- [ ] Backup compression options
- [ ] Multi-language support

**Medium Term (3-6 months):**
- [ ] Block-level deduplication
- [ ] Backup versioning (multiple versions)
- [ ] Selective restore (choose specific files)
- [ ] Backup sharing (between devices)
- [ ] Web-based file browser

**Long Term (6-12 months):**
- [ ] End-to-end encrypted sync
- [ ] Real-time backup (continuous)
- [ ] AI-powered backup optimization
- [ ] Blockchain verification (optional)
- [ ] Distributed storage (IPFS integration)

### Scalability Roadmap

**Phase 1: Current (< 100 devices)**
- Single server deployment
- Local file storage
- Manual scaling

**Phase 2: Growth (100-1000 devices)**
- Load balanced deployment
- Shared storage (NFS/S3)
- Database replication
- Auto-scaling

**Phase 3: Enterprise (1000+ devices)**
- Container orchestration (Kubernetes)
- Multi-region deployment
- CDN integration
- Advanced monitoring (Prometheus/Grafana)

---

## 📞 Support & Community

### Getting Help

**Documentation:**
- System overview: This document
- API reference: MOBILE_API_REFERENCE.md
- Deployment guide: DEPLOYMENT_GUIDE.md
- Integration guides: ANDROID_INTEGRATION.md, IOS_INTEGRATION.md

**Community:**
- GitHub Issues: Report bugs and request features
- Discussions: Ask questions and share ideas
- Wiki: Community-contributed guides

**Commercial Support:**
- Email: support@mycloudvault.example.com
- Priority support: Available for enterprise customers
- Custom development: Contact for pricing

---

## 📄 License

MyCloudVault is open source software. See LICENSE file for details.

---

## 🙏 Acknowledgments

**Technologies Used:**
- Node.js & Express - Backend framework
- React & Vite - Frontend framework
- MySQL & Drizzle ORM - Database
- Nginx - Reverse proxy
- PM2 - Process manager
- CryptoKit/Crypto - Encryption

**Contributors:**
- See CONTRIBUTORS.md for full list

---

**Last Updated:** December 12, 2025
**Version:** 1.0.0
