Adding a RAG assistant to a Laravel app without rewriting it

The usual advice for adding AI to an existing app is to stand up a separate Python service, a vector database, an orchestration framework and a message bus. For a mature Laravel application with a few hundred thousand documents, almost all of that is unnecessary.

You already have the two things that matter: a database and a queue. Everything else is a handful of classes.

Where the vectors live

If you are on Postgres, use pgvector and keep the chunks in a normal Eloquent-backed table. If you are on MySQL, either add a small Postgres instance purely for retrieval or use a hosted vector service — but keep the chunk metadata in MySQL alongside everything else.

Schema::create('document_chunks', function (Blueprint $table) {
    $table->id();
    $table->foreignId('document_id')->constrained()->cascadeOnDelete();
    $table->foreignId('tenant_id')->constrained();
    $table->text('content');
    $table->string('heading_path')->nullable();
    $table->vector('embedding', 1536);   // laravel-pgvector
    $table->timestamps();
});

cascadeOnDelete is doing quiet but important work. Delete a document and its chunks go with it, in the same transaction — so retrieval can never surface content from something the user has removed.

Indexing belongs on the queue

Chunking and embedding a document takes seconds and calls a third-party API. That is queue work, and it maps cleanly onto Laravel's job system.

class IndexDocument implements ShouldQueue
{
    public int $tries = 3;
    public array $backoff = [10, 60, 300];

    public function handle(Chunker $chunker, Embeddings $embeddings): void
    {
        $chunks = $chunker->split($this->document);

        // Batch the API call — one request per document, not per chunk
        $vectors = $embeddings->embedMany(
            $chunks->pluck('content')->all()
        );

        DB::transaction(function () use ($chunks, $vectors) {
            $this->document->chunks()->delete();
            $this->document->chunks()->createMany(
                $chunks->zip($vectors)->map(fn ($pair) => [
                    'content'      => $pair[0]['content'],
                    'heading_path' => $pair[0]['heading'],
                    'embedding'    => $pair[1],
                ])->all()
            );
        });
    }
}

Delete-then-insert inside a transaction keeps re-indexing idempotent. Run the job twice and you get one clean set of chunks, not two overlapping ones.

Retrieval is a scope

The multi-tenancy you already enforce everywhere else applies unchanged. This is the single biggest advantage of keeping retrieval inside your application: the assistant inherits your existing authorisation rather than needing a parallel copy of it.

public function retrieve(string $question, User $user, int $k = 20): Collection
{
    $vector = $this->embeddings->embed($question);

    return DocumentChunk::query()
        ->where('tenant_id', $user->tenant_id)
        ->whereHas('document', fn ($q) => $q->visibleTo($user))
        ->orderByRaw('embedding <=> ?', [$vector])
        ->limit($k)
        ->get();
}

A separate Python service would need its own copy of visibleTo. Every permission change would need making twice, and the day they drift is the day someone reads a document they should not.

Streaming the answer

Laravel supports streamed responses natively, so tokens can reach the browser as they are generated without introducing a websocket layer:

return response()->eventStream(function () use ($question, $context) {
    foreach ($this->llm->stream($question, $context) as $token) {
        yield $token;
    }
});

What you did not have to do

• No second service to deploy, monitor and secure.
• No duplicate authorisation logic.
• No sync problem between your database and a vector store.
• No new language in the stack for the team to maintain.

There is a point where this stops being the right answer — tens of millions of chunks, or GPU inference you want to host yourself. But that threshold is much further away than the standard architecture diagrams suggest, and you can cross it later with the retrieval interface already defined.