Stock Market RAG Pipeline
Kafka → Spark Structured Streaming → PostgreSQL → RAG Query Layer
kafka:9092 spark:streaming postgres:5432 ⏳ Loading...
LIVE
Price History — AAPL (last 30 ticks)
Pipeline Status
K
Kafka Producer
Initializing...
STREAM
S
Spark Streaming
micro-batch: 1s
OK
P
PostgreSQL
0 rows
OK
R
RAG Layer
Claude Sonnet 4
READY
Kafka Event Log
RAG Query Layer — Natural Language Interface to Live Pipeline Data
📡 Data: connecting to Yahoo Finance...
highest price? biggest gainer? biggest loser? market summary above average? pipeline stats
Pipeline initialized. I have live tick data for AAPL, MSFT, NVDA, GOOGL, AMZN ingested via Kafka. Ask me anything about the market data.
Built by Sai Krishna Pothana — Data Engineer · Kafka · Spark Structured Streaming · PostgreSQL · RAG · Claude AI