klz-cables.com/scripts/README-migration.md

# Migrating Analytics from Independent Analytics to Umami

This guide explains how to migrate your analytics data from the Independent Analytics WordPress plugin to Umami.

## What You Have

You have exported your analytics data from Independent Analytics:
- **data/pages(1).csv** - Page-level analytics data with:
  - Title, Visitors, Views, View Duration, Bounce Rate, URL, Page Type
  - 220 pages with historical data

## What You Need

Before migrating, you need:
1. **Umami instance** running (self-hosted or cloud)
2. **Website ID** from Umami (create a new website in Umami dashboard)
3. **Access credentials** for Umami (API key or database access)

## Migration Options

The migration script provides three output formats:

### Option 1: JSON Import (Recommended for API)
```bash
python3 scripts/migrate-analytics-to-umami.py \
  --input data/pages\(1\).csv \
  --output data/umami-import.json \
  --format json \
  --site-id YOUR_UMAMI_SITE_ID
```

**Import via API:**
```bash
curl -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d @data/umami-import.json \
  https://your-umami-instance.com/api/import
```

### Option 2: SQL Import (Direct Database)
```bash
python3 scripts/migrate-analytics-to-umami.py \
  --input data/pages\(1\).csv \
  --output data/umami-import.sql \
  --format sql \
  --site-id YOUR_UMAMI_SITE_ID
```

**Import via PostgreSQL:**
```bash
psql -U umami -d umami -f data/umami-import.sql
```

### Option 3: API Payload (Manual Import)
```bash
python3 scripts/migrate-analytics-to-umami.py \
  --input data/pages\(1\).csv \
  --output data/umami-import-api.json \
  --format api \
  --site-id YOUR_UMAMI_SITE_ID
```

## Step-by-Step Migration Guide

### 1. Prepare Your Umami Instance

**If self-hosting:**
```bash
# Clone Umami
git clone https://github.com/umami-software/umami.git
cd umami

# Install dependencies
npm install

# Set up environment
cp .env.example .env
# Edit .env with your database credentials

# Run migrations
npm run migrate

# Start the server
npm run build
npm run start
```

**If using Umami Cloud:**
1. Sign up at https://umami.is
2. Create a new website
3. Get your Website ID from the dashboard

### 2. Run the Migration Script

Choose one of the migration options above based on your needs.

**Example:**
```bash
# Make the script executable
chmod +x scripts/migrate-analytics-to-umami.py

# Run the migration
python3 scripts/migrate-analytics-to-umami.py \
  --input data/pages\(1\).csv \
  --output data/umami-import.json \
  --format json \
  --site-id klz-cables
```

### 3. Import the Data

#### Option A: Using Umami API (Recommended)

1. **Get your API key:**
   - Go to Umami dashboard → Settings → API Keys
   - Create a new API key

2. **Import the data:**
   ```bash
   curl -X POST \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer YOUR_API_KEY" \
     -d @data/umami-import.json \
     https://your-umami-instance.com/api/import
   ```

#### Option B: Direct Database Import

1. **Connect to your Umami database:**
   ```bash
   psql -U umami -d umami
   ```

2. **Import the SQL file:**
   ```bash
   psql -U umami -d umami -f data/umami-import.sql
   ```

3. **Verify the import:**
   ```sql
   SELECT COUNT(*) FROM website_event WHERE website_id = 'klz-cables';
   ```

### 4. Verify the Migration

1. **Check Umami dashboard:**
   - Log into Umami
   - Select your website
   - View the analytics dashboard

2. **Verify data:**
   - Check page views count
   - Verify top pages
   - Check visitor counts

## Important Notes

### Data Limitations

The CSV export from Independent Analytics contains **aggregated data**, not raw event data:
- ✅ Page views (total counts)
- ✅ Visitor counts
- ✅ Average view duration
- ❌ Individual user sessions
- ❌ Real-time data
- ❌ Geographic data
- ❌ Referrer data
- ❌ Device/browser data

### What Gets Imported

The migration script creates **simulated historical data**:
- Each page view becomes a separate event
- Timestamps are set to current time (for historical data, you'd need to adjust)
- Duration is preserved from the average view duration
- No session tracking (each view is independent)

### Recommendations

1. **Start fresh with Umami:**
   - Let Umami collect new data going forward
   - Use the migrated data for historical reference only

2. **Keep the original CSV:**
   - Store `data/pages(1).csv` as a backup
   - You can re-import if needed

3. **Update your website:**
   - Replace Independent Analytics tracking code with Umami tracking code
   - Test that Umami is collecting new data

4. **Monitor for a few days:**
   - Verify Umami is collecting data correctly
   - Compare with any remaining Independent Analytics data

## Troubleshooting

### Issue: "ModuleNotFoundError: No module named 'csv'"

**Solution:** Ensure Python 3 is installed:
```bash
python3 --version
# Should be 3.7 or higher
```

### Issue: "Permission denied" when running script

**Solution:** Make the script executable:
```bash
chmod +x scripts/migrate-analytics-to-umami.py
```

### Issue: API import fails

**Solution:** Check:
1. API key is correct and has import permissions
2. Website ID exists in Umami
3. Umami instance is accessible
4. JSON format is valid

### Issue: SQL import fails

**Solution:** Check:
1. Database credentials in `.env`
2. Database is running
3. Tables exist (run `npm run migrate` first)
4. Permissions to insert into `website_event` table

## Additional Data Migration

If you have other CSV exports from Independent Analytics (referrers, devices, locations), you can:

1. **Export additional data** from Independent Analytics:
   - Referrers
   - Devices (browsers, OS)
   - Geographic data
   - Custom events

2. **Create custom migration scripts** for each data type

3. **Contact Umami support** for bulk import assistance

## Support

- **Umami Documentation:** https://umami.is/docs
- **Umami GitHub:** https://github.com/umami-software/umami
- **Independent Analytics:** https://independentanalytics.com/

## Summary

✅ **Completed:**
- Created migration script (`scripts/migrate-analytics-to-umami.py`)
- Generated JSON import file (`data/umami-import.json`)
- Generated SQL import file (`data/umami-import.sql`)
- Created documentation (`scripts/README-migration.md`)

📊 **Data Migrated:**
- 7,634 simulated page view events
- 220 unique pages
- Historical view counts and durations

🎯 **Next Steps:**
1. Choose your import method (API or SQL)
2. Run the migration script
3. Import data into Umami
4. Verify the migration
5. Update your website to use Umami tracking