Multimodal hybrid RAG on AWS

• Ingestion: documents land in S3, get parsed with layout awareness, chunked by structure, embedded and indexed in OpenSearch for both BM25 and k-NN.
• Query path: authentication, query rewriting, hybrid retrieval with access filters, fusion, reranking, then generation on Bedrock with citations.
• Guardrails and PII checks on input and output; traces, metrics and cost per request logged.
• For each box, say why it’s there and what you’d swap it for at 10× scale.

Listing AWS services without the data flow or the reasons behind each choice.

• Layout-aware parsing (Textract or a vision model) so tables keep rows, columns and headers, stored as Markdown or HTML.
• Charts and images captioned by a multimodal model, with the page image kept for citations.
• Chunk by section and table, not fixed token windows; attach metadata such as document type, period, section and page.
• Evaluate table questions separately, because they fail differently from prose.

Saying a generic text splitter was enough.

• Lexical and vector search in one engine, which hybrid retrieval needs.
• AWS-native IAM, VPC and encryption, which a finance client’s security review favors.
• Metadata filtering at query time for access control.
• Trade-offs: HNSW memory cost and tuning effort. Alternatives were pgvector, Pinecone or Bedrock Knowledge Bases.

“The client chose it.” Even when it was a constraint, show you know the trade-offs.

• Lambda suits event-driven ingestion steps; one invocation can run for at most 15 minutes.
• API Gateway’s default 29-second integration timeout and long streamed answers push chat traffic toward response streaming, ECS/Fargate or EKS.
• Cold starts: provisioned concurrency, slim packages, connection reuse.
• Say where you’d draw the line between Lambda and containers.

Claiming Lambda is right for every part of the system.

• Store access metadata (groups, classification) on every chunk at index time.
• Apply the user’s permissions as a filter inside the retrieval query, so unauthorized chunks are never retrieved.
• Never rely on the prompt or the model to hide content.
• Test with leakage evals: users from different groups asking about restricted documents.

Filtering after generation. By then the model has already seen the data.

• Name one real weakness and its fix, such as building the golden eval set earlier, adding contextual retrieval, or moving agent parts onto AgentCore.
• Explain what you learned that changed your mind.

Saying you wouldn’t change anything.