myCRM/docs/GIT_PERFORMANCE_OPTIMIZATION.md

# Git Repository API Performance Optimization

## Problem
The `/api/git-repos/{id}/commits` endpoint was taking **66+ seconds** (66678ms) to fetch 50 commits from GitHub, making the application unusable.

## Root Causes Identified

1. **No GitHub Authentication Token**
   - Unauthenticated GitHub API requests: **60 requests/hour** rate limit
   - Authenticated requests: **5000 requests/hour**
   - Without a token, GitHub severely throttles requests causing massive delays

2. **No HTTP Timeout Configuration**
   - HTTP client waited indefinitely when GitHub throttled requests
   - No retry logic for failed or rate-limited requests
   - No maximum duration limits

3. **Cache Working But Not Enough**
   - 15-minute cache was configured but doesn't help first-time requests
   - GitHub API calls were still the bottleneck

## Solutions Implemented

### 1. HTTP Client Configuration (`config/packages/http_client.yaml`)

**Default timeout and retry logic:**
```yaml
default_options:
    timeout: 10          # Connection timeout: 10 seconds
    max_duration: 30     # Maximum request duration: 30 seconds
    
    retry_failed:
        max_retries: 2   # Retry failed requests twice
        delay: 1000      # Initial delay: 1 second
        multiplier: 2    # Exponential backoff
```

**Scoped clients for GitHub and Gitea:**
```yaml
scoped_clients:
    github.client:
        base_uri: 'https://api.github.com'
        timeout: 15
        max_duration: 45
        headers:
            Accept: 'application/vnd.github.v3+json'
            User-Agent: 'myCRM-App'
    
    gitea.client:
        scope: 'https?://.*'
        timeout: 15
        max_duration: 45
        headers:
            Accept: 'application/json'
            User-Agent: 'myCRM-App'
```

### 2. GitHub Token Configuration

**Added to `.env`:**
```env
###> app/git-services ###
# Optional: GitHub Personal Access Token for higher rate limits
# Create at: https://github.com/settings/tokens (needs 'public_repo' scope)
GITHUB_TOKEN=

# Optional: Gitea Access Token for private instances
GITEA_TOKEN=
###< app/git-services ###
```

**How to create a GitHub token:**
1. Go to: https://github.com/settings/tokens
2. Click "Generate new token (classic)"
3. Select scopes:
   - `public_repo` - For accessing public repositories
   - `repo` - For accessing private repositories (if needed)
4. Copy token and add to `.env.local`:
   ```
   GITHUB_TOKEN=ghp_your_token_here
   ```

### 3. Service Configuration Update

**Updated `config/services.yaml`:**
```yaml
App\Service\GitHubService:
    arguments:
        $httpClient: '@github.client'  # Use scoped client
        $githubToken: '%env(string:default::GITHUB_TOKEN)%'

App\Service\GiteaService:
    arguments:
        $httpClient: '@gitea.client'  # Use scoped client
        $giteaToken: '%env(string:default::GITEA_TOKEN)%'
```

## Expected Performance Improvements

| Scenario | Before | After | Improvement |
|----------|--------|-------|-------------|
| **First request (no cache)** | 66+ seconds | 2-5 seconds | **92-97% faster** |
| **Cached requests** | 66+ seconds | <50ms | **99.9% faster** |
| **With GitHub token** | 66+ seconds | 1-2 seconds | **98% faster** |
| **Timeout prevents hanging** | ∞ (infinite wait) | Max 45 seconds | Guaranteed response |

## Testing Performance

### Test without token:
```bash
# First request (cache miss)
curl "https://mycrm.test/api/git-repos/4/commits?branch=main&limit=50" -w "\nTime: %{time_total}s\n"

# Second request (cache hit - should be <50ms)
curl "https://mycrm.test/api/git-repos/4/commits?branch=main&limit=50" -w "\nTime: %{time_total}s\n"
```

### Test with token:
1. Add `GITHUB_TOKEN=ghp_xxx` to `.env.local`
2. Clear cache: `php bin/console cache:clear`
3. Clear API cache via UI or: `curl -X POST https://mycrm.test/api/git-repos/4/refresh-cache`
4. Test again - should be 1-2 seconds

### Monitor rate limits:
```bash
curl -I https://api.github.com/rate_limit
# Without token: X-RateLimit-Limit: 60
# With token: X-RateLimit-Limit: 5000
```

## Additional Optimizations

### 1. Reduce Cache Duration (if needed)
Current: 15 minutes (`900` seconds)
```php
// In GitRepositoryController.php line 39
$item->expiresAfter(900); // Reduce to 300 (5 min) for more frequent updates
```

### 2. Background Jobs (future enhancement)
For repositories with frequent updates, consider:
- Symfony Messenger to fetch commits in background
- Pre-warm cache on schedule (every 10 minutes)
- Webhook integration with GitHub/Gitea for instant updates

### 3. Partial Response Loading
Instead of loading all 50 commits at once:
- Load 10 commits initially (fast)
- Lazy-load more on scroll (pagination)

## Monitoring

### Check cache effectiveness:
```bash
# Monitor cache hit/miss ratio
php bin/console cache:pool:list
php bin/console cache:pool:prune  # Clean up old cache entries
```

### Check HTTP client metrics:
```bash
# Enable Symfony profiler and check HTTP client panel
# Shows: request duration, retry attempts, cache hits
```

## Rollback Plan

If issues occur, revert by:
1. Remove `config/packages/http_client.yaml`
2. Restore original `config/services.yaml` (remove `$httpClient` arguments)
3. Clear cache: `php bin/console cache:clear`

## Related Files
- `config/packages/http_client.yaml` - HTTP client configuration
- `config/services.yaml` - Service injection configuration
- `.env` - Token configuration (template)
- `.env.local` - Actual tokens (gitignored)
- `src/Controller/GitRepositoryController.php` - Caching logic
- `src/Service/GitHubService.php` - GitHub API integration
- `src/Service/GiteaService.php` - Gitea API integration