How should I structure a full-stack AI chat application using Next.js, TypeScript, PostgreSQL, and an LLM API?

[bungakarunanidhi]

🏷️ Discussion Type

Question

Body

What I'm trying to build

I am building a full-stack AI chat application using Next.js, React, TypeScript, PostgreSQL, and an LLM API.

The application will have:

• User registration and login
• AI chat
• Streaming AI responses
• Conversation history
• PDF/document upload
• RAG (Retrieval-Augmented Generation)
• AI memory
• Vector search
• File storage
• User-specific data isolation

My current planned stack is:

• Next.js
• React
• TypeScript
• Tailwind CSS
• PostgreSQL
• pgvector
• OpenAI / Gemini / Anthropic API

My current architecture

I am thinking about structuring the application like this:

Browser
   |
   v
Next.js / React
   |
   v
API / Server Actions
   |
   +----------------------+
   |                      |
   v                      v
PostgreSQL             LLM API
   |
   +--- Users
   +--- Conversations
   +--- Messages
   +--- Documents
   +--- Memories
   +--- Document Chunks

For PDF/document processing, I am thinking about:

PDF
 |
 v
Upload
 |
 v
Extract Text
 |
 v
Split into Chunks
 |
 v
Generate Embeddings
 |
 v
PostgreSQL + pgvector
 |
 v
User asks a question
 |
 v
Vector Search
 |
 v
Relevant Chunks
 |
 v
LLM
 |
 v
Answer

My main programming question

I am not sure how I should properly organize the code.

I am considering a structure like:

src/
│
├── app/
│   ├── chat/
│   ├── login/
│   ├── dashboard/
│   └── api/
│       ├── chat/
│       ├── documents/
│       ├── memory/
│       └── conversations/
│
├── components/
│   ├── chat/
│   ├── sidebar/
│   └── documents/
│
├── lib/
│   ├── ai/
│   ├── database/
│   ├── auth/
│   ├── rag/
│   ├── memory/
│   └── storage/
│
└── types/

I would like to know if this is a good structure for a production application.

Chat request flow

When a user sends a message, I want something similar to:

User Message
     |
     v
Authenticate User
     |
     v
Get Conversation
     |
     v
Save User Message
     |
     v
Retrieve Recent Conversation
     |
     v
Retrieve Relevant Memories
     |
     v
Search Relevant Documents
     |
     v
Build AI Context
     |
     v
Call LLM
     |
     v
Stream Response
     |
     v
Save AI Response

I am thinking about keeping the API route relatively small and moving the actual logic into separate functions/services.

For example:

POST /api/chat

could internally call:

authenticateUser()
        |
        v
getConversation()
        |
        v
getRelevantMemory()
        |
        v
searchDocuments()
        |
        v
buildPrompt()
        |
        v
callLLM()
        |
        v
streamResponse()
        |
        v
saveMessage()

Is this a good programming pattern?

LLM provider architecture

I also want to avoid tightly coupling the entire application to one AI provider.

I was thinking about creating an interface such as:

interface AIProvider {
    generateResponse();
    streamResponse();
    createEmbedding();
}

Then having implementations such as:

AIProvider
    |
    +--- OpenAIProvider
    |
    +--- GeminiProvider
    |
    +--- AnthropicProvider

This would allow me to change AI providers later without rewriting the entire application.

Is this a good approach, or is it unnecessary abstraction for an MVP?

Database architecture

I also want to understand the correct way to access the database.

Should the frontend directly access the database?

React Component
      |
      v
Database

Or should it always go through server-side code?

React Component
      |
      v
API / Server Action
      |
      v
Service
      |
      v
Database

I want to make sure that users can only access their own:

• Conversations
• Messages
• Documents
• Memories
• Files

and can never access another user's data.

AI Memory

I want the AI to have long-term memory.

For example, if the user tells the AI:

I am building an RAS system.

I may want the application to save this as a memory.

I am considering:

User Message
     |
     v
Memory Extraction
     |
     v
Important Information
     |
     v
PostgreSQL
     +
pgvector

Then later:

New Question
     |
     v
Create Embedding
     |
     v
Vector Search
     |
     v
Relevant Memories
     |
     v
LLM Context

Is this a good architecture for AI memory?

RAG

For document questions, I want to use RAG.

For example:

User uploads PDF
       |
       v
Extract text
       |
       v
Split into chunks
       |
       v
Create embeddings
       |
       v
Store in pgvector

Then:

User asks question
       |
       v
Create query embedding
       |
       v
Search pgvector
       |
       v
Retrieve relevant chunks
       |
       v
Send chunks + question to LLM
       |
       v
Generate answer

I would also like the answer to reference the source page of the PDF.

For example:

Answer:
The total project cost is ₹70 lakh.

Source:
Project_DPR.pdf
Page 18

What is the recommended way to store page numbers and metadata so that I can generate reliable citations?

Streaming

I want the AI response to appear progressively in the UI instead of waiting for the complete response.

For example:

AI:
The project...

then:

AI:
The project cost...

then:

AI:
The project cost is ₹70 lakh...

What is the recommended way to implement streaming with Next.js and TypeScript?

Should I use:

• Server-Sent Events
• Web Streams
• Fetch streaming
• WebSockets

for an AI chat application?

Authentication and security

Security is very important because users will have private conversations and documents.

I want the architecture to ensure:

User A
   |
   X
   |
User B's data

is impossible.

I am considering authentication through Supabase Auth / Auth.js / Clerk.

For database security, I am also considering PostgreSQL Row Level Security.

Is this enough, or are there other important security controls I should implement?

I also understand that LLM API keys should never be exposed in the browser.

The flow should be:

Browser
   |
   v
My Server
   |
   v
LLM Provider

and not:

Browser
   |
   v
LLM Provider

Guidelines

• I have read and understood this category's guidelines before making this post.

[krishanth7]

Your architecture is generally correct. For a first production MVP, I would not start with Next.js + FastAPI + Redis + multiple microservices. That adds complexity before you need it.

Use a modular monolith:

Next.js
├── React UI
├── Server/API layer
├── AI service
├── RAG service
├── Memory service
└── Database layer
        |
        ├── PostgreSQL
        └── pgvector

Object Storage
└── PDFs / documents

LLM Provider
└── OpenAI / Gemini / Anthropic

1. Project structure

Your structure is good. I would organize it approximately like this:

src/
├── app/
│   ├── chat/
│   ├── login/
│   ├── dashboard/
│   └── api/
│       ├── chat/
│       ├── documents/
│       ├── conversations/
│       └── memory/
│
├── components/
│   ├── chat/
│   ├── documents/
│   ├── sidebar/
│   └── ui/
│
├── lib/
│   ├── ai/
│   │   ├── provider.ts
│   │   ├── openai.ts
│   │   └── prompts.ts
│   │
│   ├── db/
│   │   ├── client.ts
│   │   └── queries/
│   │
│   ├── rag/
│   │   ├── chunk.ts
│   │   ├── embeddings.ts
│   │   ├── retrieval.ts
│   │   └── citations.ts
│   │
│   ├── memory/
│   │   ├── extract.ts
│   │   ├── retrieve.ts
│   │   └── store.ts
│   │
│   ├── auth/
│   └── storage/
│
└── types/

The important principle is:

UI components should not contain your AI/business logic.

---

2. /api/chat should be thin

Don't put your entire application inside one API route.

Use:

POST /api/chat
       |
       v
authenticateUser()
       |
       v
chatService()
       |
       ├── getConversation()
       ├── getRecentMessages()
       ├── retrieveMemory()
       ├── retrieveDocuments()
       ├── buildContext()
       ├── callLLM()
       └── saveMessage()

For example:

export async function POST(request: Request) {
  const user = await authenticateUser();

  if (!user) {
    return new Response("Unauthorized", { status: 401 });
  }

  const body = await request.json();

  return chatService({
    userId: user.id,
    conversationId: body.conversationId,
    message: body.message,
  });
}

This keeps the API route easy to understand and test.

---

3. Database access

Don't do this:

React component
      ↓
Database

for privileged operations.

Prefer:

React
  ↓
Server Action / API
  ↓
Service
  ↓
Database

Every query involving user-owned data should be scoped to the authenticated user.

For example:

SELECT *
FROM conversations
WHERE id = $1
AND user_id = $2;

Don't retrieve everything and filter it in JavaScript afterward.

---

4. PostgreSQL + pgvector

For your MVP, PostgreSQL + pgvector is enough.

You don't need Pinecone/Qdrant/Weaviate immediately.

You can store:

users
conversations
messages
documents
document_chunks
memories
usage

and vector embeddings in PostgreSQL.

This gives you a much simpler architecture.

---

5. RAG

Your RAG design is correct:

PDF
 ↓
Storage
 ↓
Text extraction
 ↓
Page detection
 ↓
Chunking
 ↓
Embeddings
 ↓
pgvector

Then:

Question
 ↓
Query embedding
 ↓
Vector search
 ↓
Relevant chunks
 ↓
LLM
 ↓
Answer

For each chunk, store metadata such as:

document_id
page_number
chunk_index
content
embedding

Then your application can return:

Source: Project_DPR.pdf
Page: 18

with the answer.

---

6. AI memory

Don't turn every message into a memory.

Use three separate concepts:

Conversation history
        ↓
PostgreSQL

Long-term user memory
        ↓
PostgreSQL + optional pgvector

Document knowledge
        ↓
PostgreSQL + pgvector

For example:

User says:

"I prefer TypeScript."

That might become a memory.

But:

"What is 2 + 2?"

probably shouldn't.

When answering:

Current question
      +
Recent conversation
      +
Relevant memories
      +
Relevant document chunks
      ↓
    LLM

Only retrieve information relevant to the current request.

---

7. LLM abstraction

Your idea is good, but keep it small.

For example:

interface AIProvider {
  generateResponse(input: AIInput): Promise<AIResponse>;

  streamResponse(
    input: AIInput
  ): AsyncIterable<string>;

  createEmbedding(
    text: string
  ): Promise<number[]>;
}

Then:

AIProvider
   ├── OpenAIProvider
   ├── GeminiProvider
   └── AnthropicProvider

However, if you're only using one provider initially, don't build a huge abstraction framework.

Start with one provider and introduce the interface when you actually need multiple providers.

---

8. Streaming

For normal AI chat, you generally don't need WebSockets.

Use HTTP streaming/Web Streams or the streaming mechanism provided by your AI SDK/provider.

The flow is:

Browser
   ↓
POST /api/chat
   ↓
LLM
   ↓
stream chunks
   ↓
Browser

The UI can progressively render:

The
The project
The project cost
The project cost is
The project cost is ₹70 lakh.

WebSockets are more useful when you need persistent bidirectional real-time communication beyond normal AI response streaming.

---

9. PDF processing

For small PDFs, synchronous processing may initially be acceptable.

But don't design the production system around long-running HTTP requests.

Eventually use:

Upload
 ↓
Storage
 ↓
Processing job
 ↓
Worker
 ↓
Extract
 ↓
Chunk
 ↓
Embed
 ↓
pgvector
 ↓
READY

Then the UI can show:

Uploading...
Processing...
Indexing...
Ready

You can add a queue/worker system when the workload requires it.

---

10. Security

Your API key architecture is correct:

Browser
   ↓
Your server
   ↓
LLM provider

Never:

Browser
   ↓
LLM provider

with a private API key embedded in client-side JavaScript.

Also implement:

• Authentication
• Authorization
• User ownership checks
• Database Row Level Security where appropriate
• Input validation
• File type validation
• File size limits
• Rate limiting
• API usage limits
• Secure file URLs
• Error handling
• Logging
• Secret management

Treat uploaded document text as untrusted input, because documents can contain prompt-injection instructions.

---

11. Should you use Python/FastAPI?

Not initially.

Next.js + TypeScript is enough for the MVP.

Use Python later if you actually need Python-specific functionality such as:

• Custom ML models
• Computer vision
• Training/fine-tuning
• Scientific/data processing
• Specialized AI pipelines

Starting with:

Next.js + TypeScript + Python + FastAPI

means maintaining two ecosystems when one may be sufficient.

---

12. Should you use LangChain/LlamaIndex?

You don't have to.

Start with direct, understandable code.

For example:

retrieveDocuments()
buildContext()
callLLM()

is easier to understand than introducing several abstraction layers immediately.

Add a framework when it solves a real problem in your application.

---

13. What I would build first

Build in this order:

1. Next.js project
2. Authentication
3. PostgreSQL
4. Chat UI
5. One LLM provider
6. Save conversations
7. Save messages
8. Streaming
9. File upload
10. PDF extraction
11. Chunking
12. Embeddings
13. pgvector
14. RAG
15. Citations
16. Memory
17. Rate limiting
18. Usage tracking
19. Background processing
20. Monitoring

Only after that:

Web search
Agents
Tool calling
Voice
Image AI
Multiple LLM providers
Advanced workflows

Bottom line

For your project, I'd start with:

Next.js
React
TypeScript
Tailwind
PostgreSQL
pgvector
One authentication system
One LLM provider
Object storage

and keep the code modular.

Don't start with microservices. Don't add Python just because it is an AI application. Don't add a dedicated vector database before you need one.

Build a clean modular monolith first, then extract individual workloads when the application actually requires it.

[ashishmishra26]

Your proposed architecture is very solid and aligns well with current best practices for full-stack AI apps. Next.js with Server Actions is a great fit for handling the API layer, and for streaming LLM responses, you should definitely look into the Vercel AI SDK if you haven't already — it handles the streaming UI hooks seamlessly.

For the database layer, using PostgreSQL for your relational data (Users, Conversations, Messages, metadata) is exactly the right call.

Regarding the RAG, Document Chunks, and AI Memory components: starting with pgvector is a common approach to keep everything in one database. However, as you scale your PDF uploads and need better retrieval quality, you'll likely find that pure vector search isn't enough. Production RAG usually requires hybrid search (combining vector similarity with BM25 keyword search via reciprocal-rank fusion) to get accurate results across both semantic concepts and exact terminology.

If you want to keep the operational simplicity of not running a separate search cluster but want that advanced retrieval, you might want to look at Infino. It's an open-source embedded retrieval engine that runs in-process (like SQLite) and has Node.js bindings. It handles vector kNN, full-text (BM25), and hybrid search out of the box. You could use Postgres for your user/app state, and run Infino embedded in your Next.js backend specifically for the document chunks and AI memory retrieval.

For file storage, the standard pattern is to drop the raw PDFs into an S3-compatible object store, parse them in your backend (Next.js API route or a background worker), and then index the resulting chunks into your retrieval engine.

Disclosure: I work on Infino.
https://infino.ai

[varsh23-p]

Root Cause Analysis & Immediate Preflight Check Workaround

Hi @wscha231! Your report and API traces provide a textbook breakdown of a backend state-machine desync in GitHub Actions.

---

🔍 Why HTTP 409 Occurs on Cancel/Force-Cancel

This happens due to an internal divergence between the Workflow Run aggregate state and the Job Execution coordinator:

1. The State Desync: The top-level workflow run record is stuck in a non-terminal state (queued / in_progress), but the underlying job coordinator has already marked the jobs as skipped/completed or never initialized them.
2. The 409 Conflict: When you call POST /cancel or POST /force-cancel, the API gateway verifies the status with the internal job scheduler. Because the scheduler finds no active job worker or queue entry to terminate, it throws HTTP 409 Conflict ("Cannot cancel a workflow run that is not in progress").

Because user-facing APIs enforce validation against the job scheduler, there are no public API endpoints or UI toggles that can forcibly write a terminal conclusion once this desynchronization occurs.

---

🛠️ 1. Official Route for State Reconciliation

To reconcile the run state without deleting logs, artifacts, or execution history:

• Open a ticket directly at GitHub Support under GitHub Actions.
• Reference this discussion URL and the specific run IDs (31126781170 and 25906905688).
• Request a backend run state reconciliation (or DB sync) to transition the run conclusions to cancelled or failure. GitHub internal tooling can patch the metadata without purging run history.

---

💡 2. Immediate Client-Side Workaround for CI Preflight Checks

While waiting for backend reconciliation, you can adjust your preflight script so these historical ghost runs don't block your deployment pipeline:

Option A: Filter by Age Threshold (Recommended)

Add a timestamp filter in your preflight check to ignore runs older than a realistic execution timeout (e.g., created > 24 hours ago):

# Example jq filter ignoring orphaned runs older than 24 hours
gh api "repos/wscha231/r1000-quant-engine/actions/runs?status=in_progress" | \
jq --arg cutoff "$(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ)" \
  '[.workflow_runs[] | select(.created_at > $cutoff)]'