Vector Storage

Collection Naming

Each agent has its own vector collection, named using a deterministic convention to enforce knowledge isolation:

// Collection name convention
string collectionName = $"agent_{agentId:N}";
// Example: "agent_7f3a8b2c4d5e6f7a8b9c0d1e2f3a4b5c"

// This means:
// - Agent A's knowledge is in collection "agent_AAAA..."
// - Agent B's knowledge is in collection "agent_BBBB..."
// - No cross-collection queries possible — complete isolation by design

Qdrant — Production Backend

Qdrant is the recommended vector backend for production deployments. It is a dedicated vector database with native support for filtering, snapshots, and distributed deployments:

// appsettings.json — Qdrant configuration
{
  "Octopus": {
    "VectorStore": {
      "Backend":   "Qdrant",
      "QdrantUrl": "http://qdrant:6333",
      "ApiKey":    null  // Set for cloud-hosted Qdrant
    },
    "Embedding": {
      "Provider":    "OpenAI",
      "Model":       "text-embedding-3-small",
      "Dimensions":  1536,
      "CredentialId": 42
    }
  }
}

// Collection creation (called automatically on first use):
PUT http://qdrant:6333/collections/agent_{agentId}
{
  "vectors": {
    "size":     1536,
    "distance": "Cosine"
  },
  "optimizers_config": {
    "indexing_threshold": 20000
  }
}

Qdrant Point Structure

Each indexed document chunk is stored as a Qdrant point with a vector and a payload:

{
  "id":     "550e8400-e29b-41d4-a716-446655440000",
  "vector": [0.012, -0.034, 0.091, ...],   // 1536 floats
  "payload": {
    "content":   "Annual leave entitlement is 20 days per calendar year...",
    "source":    "HR Policy 2025.pdf",
    "chunkIndex": 3,
    "pageNumber": 12,
    "category":  "Leave",
    "agentId":   "7f3a8b2c...",
    "tenantId":  "tenant_acme",
    "indexedAt": "2025-03-01T08:00:00Z"
  }
}

PGVector — PostgreSQL Backend

PGVector is a PostgreSQL extension that adds native vector similarity search. It uses the existing PostgreSQL instance — no additional infrastructure required:

-- Enable the extension (once per database)
CREATE EXTENSION IF NOT EXISTS vector;

-- Per-agent knowledge table
CREATE TABLE octopus_knowledge_agent_{agentId} (
    id          UUID           NOT NULL DEFAULT gen_random_uuid(),
    content     TEXT           NOT NULL,
    embedding   vector(1536)   NOT NULL,
    source      TEXT           NULL,
    chunk_index INT            NULL,
    category    TEXT           NULL,
    tenant_id   TEXT           NOT NULL,
    indexed_at  TIMESTAMPTZ    NOT NULL DEFAULT NOW(),

    PRIMARY KEY (id)
);

-- IVFFlat index for approximate nearest-neighbour search
CREATE INDEX ON octopus_knowledge_agent_{agentId}
    USING ivfflat (embedding vector_cosine_ops)
    WITH (lists = 100);

-- Search query
SELECT content, source, 1 - (embedding <=> $1::vector) AS score
FROM   octopus_knowledge_agent_{agentId}
WHERE  tenant_id = $2
  AND  1 - (embedding <=> $1::vector) >= $3  -- minScore
ORDER BY embedding <=> $1::vector
LIMIT  $4;

Backend Comparison