Real-time Event & Delivery Pipelines
"Streaming plumbing for growth: CleverTap webhooks into Kafka and ClickHouse across AbhiBus, ixigo trains and ConfirmTkt, plus a WhatsApp delivery-report service that absorbs crore-scale blasts without losing a single receipt."
Executive Summary
Growth decisions were waiting on yesterday's data, and our old automation tool collapsed during a festive blast. I built two pipelines. The first streams real-time search and booking events from CleverTap into Kafka and ClickHouse for three brands, so cross-sell and drop-off campaigns fire within minutes. The second replaced an overloaded workflow tool for WhatsApp delivery reports with a queue-backed service — RAM buffer, MySQL queue, parallel workers claiming batches safely — that keeps up with crore-scale sends.
Key Outcomes
- Search and booking events from three brands available in real time.
- WhatsApp delivery reports processed losslessly through festive peaks.
- Train-to-bus cross-sell triggered within minutes of a train search.
- Self-monitoring watchdog alerts on WhatsApp and Slack if anything stalls.
Next Case Study
AI-Graded B2B Prospect Engine
Scraped 84K local travel businesses from Google Maps, had an AI model grade every one by how likely it sells bus tickets, and turned it into a 4.6-lakh-prospect field map for the sales team — the engine behind 10K+ verified agents onboarded from zero.