Design a Conversational AI Customer Support System

Why This Is Asked

Conversational AI support is one of the most common real-world LLM applications. This question tests your ability to combine multiple AI techniques into a coherent system with real product constraints.

Key Concepts to Cover

• Intent classification — routing queries to the right handler
• RAG for knowledge retrieval — grounding answers in company documentation
• Multi-turn context — maintaining conversation state across messages
• Human escalation — when and how to hand off to a human agent
• Guardrails — preventing the bot from making wrong commitments
• Feedback collection — measuring resolution rate and CSAT

How to Approach This

1. Clarify Requirements

• What's the domain? (e-commerce, SaaS, banking)
• What percentage of queries should resolve automatically?
• What's the escalation path?
• What data sources are available?
• Any compliance requirements?

2. High-Level Architecture

User Message → Intent Classifier → Router
                                    ├── FAQ/Info queries → RAG Pipeline → LLM Response
                                    ├── Account queries → API Integration → LLM Response
                                    ├── Complex/emotional → Human Agent Queue
                                    └── Out-of-scope → Deflection + Escalation

3. Multi-Turn Context

• Store conversation turns in Redis with TTL
• Include recent N turns in every LLM prompt
• Summarize old turns to fit context limits
• Preserve key entities (order numbers, account IDs) across turns

4. Guardrails

• Never let the bot commit to refunds, timelines, or policies it can't verify
• Use structured output to separate "facts" from "suggestions"
• Log all bot responses for audit

5. Measuring Success

• Containment rate: % resolved without human escalation
• CSAT: customer satisfaction score
• False resolution rate: cases where bot said "resolved" but customer came back

Common Follow-ups

1. "How would you handle an angry or distressed customer?" Sentiment detection to trigger immediate human escalation, empathetic response templates.

2. "How do you keep the bot's knowledge up to date?" Automated re-indexing on documentation updates, version-controlled prompt updates, A/B testing new knowledge base versions.

3. "What happens when the bot gives a wrong answer?" Logging all responses, a "report a problem" button, post-conversation review for flagged cases.