Migration — Memory Storage

EF Core SQL Migrations

All SQL schema changes are managed by EF Core migrations bundled in the SqlServerPlugin assembly. Migrations are applied automatically on startup:

// Automatic migration on startup (in SqlServerPlugin.OnStartAsync)
public async Task OnStartAsync(IServiceProvider sp, CancellationToken ct)
{
    var db = sp.GetRequiredService<OctopusDbContext>();
    // Applies all pending migrations from the plugin assembly
    await db.Database.MigrateAsync(ct);
}

// Developer workflow: add a new migration
dotnet ef migrations add AddProceduralVersion \
    --project BizFirst.Octopus.SqlServerPlugin \
    --startup-project BizFirst.Octopus.Api

// Review generated migration before deploying
// File: BizFirst.Octopus.SqlServerPlugin/Migrations/20250301_AddProceduralVersion.cs

// Apply manually (for production environments requiring controlled rollout)
dotnet ef database update \
    --project BizFirst.Octopus.SqlServerPlugin \
    --connection "Server=prod-sql;Database=OctopusDB;..."

Safe Migration Practices

Practice	Reason
Always add columns as nullable first	Allows zero-downtime deployments — old code ignores the new column
Never rename columns in a single migration	Split into: add new column → backfill data → drop old column (3 separate deployments)
Test migrations on a copy of production data	Large tables may take longer than expected; estimate and test first
Keep migration history intact	Never delete EF Core migration files after they are applied to any environment

Embedding Model Migration

Changing the embedding model requires re-indexing all documents because vectors from different models are incompatible (different dimensions and semantic spaces):

Create New Collection Create a new Qdrant collection with the new model's vector dimensions. Do not delete the old collection yet.

Re-index All Documents Run the document ingestion pipeline against all source documents using the new embedding model. Write to the new collection.

Update Agent Config Update each agent's EmbeddingModel config to point to the new model. This switches retrieval to use the new collection.

Validate Retrieval Quality Run a set of known test queries and verify expected documents are returned with good scores.

Delete Old Collection Only after validation passes and all agents are updated. This frees vector storage.

Migrating Between Vector Backends

To migrate from Qdrant to PGVector (or vice versa), export all points from the source and import them into the destination:

// Migration helper: export from Qdrant, import to PGVector
public class VectorBackendMigrator
{
    public async Task MigrateCollectionAsync(
        string collectionName,
        IQdrantClient source,
        IPgVectorStore destination,
        CancellationToken ct)
    {
        uint offset = 0;
        const int batchSize = 1000;

        while (true)
        {
            // Scroll through Qdrant points in batches
            var batch = await source.ScrollAsync(collectionName,
                limit: (uint)batchSize,
                offset: offset,
                withPayload: true,
                withVectors: true,
                cancellationToken: ct);

            if (!batch.Result.Any()) break;

            // Write each point to PGVector
            foreach (var point in batch.Result)
            {
                await destination.UpsertAsync(new MemoryRecord
                {
                    Id        = point.Id.ToString(),
                    Content   = point.Payload["content"].StringValue,
                    Embedding = point.Vectors.Vector.Data.ToArray(),
                    Metadata  = MapPayloadToMetadata(point.Payload)
                }, ct);
            }

            offset += (uint)batch.Result.Count;
        }
    }
}

Episode Embeddings Must Also Be Re-indexed

If you change the embedding model and use Semantic recall mode for episodic memory, the episode embeddings stored in Octopus_Episodes.Embedding (SQL) must also be recalculated. Run a backfill job that reads each episode summary, calls the new embedding model, and updates the row.

← Backup and Restore