Stream: Virtual Room 8
Time: 11:15 - 12:00
In today's digital ecosystem, where data volumes continue to surge and latency-sensitive applications are becoming the norm, traditional batch processing is no longer adequate for competitive operations. Organisations leveraging real-time streaming analytics consistently outperform those still relying on legacy batch systems a gap that continues to widen as data velocity accelerates across industries. This 45-minute session delivers an educator-focused exploration of real-time data processing pipelines, examining the critical architectural distinctions between micro-batching and true streaming approaches. Attendees will gain a clear understanding of how these architectural decisions directly drive measurable business outcomes across key sectors. Financial services firms implementing stream processing have reduced fraud detection timeframes from hours to milliseconds, generating significant annual savings. E-commerce platforms harnessing real-time recommendation engines have seen meaningful improvements in conversion rates and average order value through contextual personalisation. Manufacturing operations deploying streaming analytics on IoT sensor networks have achieved substantial reductions in unplanned downtime alongside notable gains in equipment efficiency. The session then addresses the technical complexities practitioners must navigate, including event-time versus processing-time paradigms and strategies for managing out-of-order events in high-velocity data streams. Delegates will walk away with practical frameworks applicable to their own environments, regardless of sector or organisation size. The focus throughout remains on transferable principles rather than vendor-specific tooling, ensuring relevance for architects, data engineers, and business leaders alike. The presentation concludes with an objective architectural comparison of leading frameworks including cloud-native services such as AWS Kinesis and Azure Event Hubs, alongside open-source solutions like Apache Kafka and Apache Flink equipping attendees with the knowledge to evaluate streaming solutions against their specific operational requirements and build truly intelligence-driven enterprises.
Sai Kaushik Ponnekanti is a seasoned Software Engineer with over 10 years of experience in designing and building large-scale distributed systems. Currently at Meta, he spearheads end-to-end monitoring systems for ML Clean Room projects and leads implementation of compliance checks across data and AI assets. Previously at Google, Sai led blockchain data ingestion for the Cloud Web3 team and significantly improved platform performance for AppSheet, achieving a 27% increase in responsiveness. During his tenure at Walmart Labs, he converted an expensive sessionized mapping to a server-side pipeline, resulting in a 34% increase in site loading time for Walmart.com, and successfully handled 1.2 million requests per second during peak traffic periods. As Lead Software Engineer at Drawbridge Inc, Sai led development of AdTech products using Spark and helped make the company GDPR compliant in just three weeks. He has consistently improved system performance throughout his career, often reducing pipeline runtimes by 50-75% through optimization techniques. Sai holds a Master's in Computer Science from the University of Southern California and a Bachelor's in Computer Science from BITS Pilani, India. His expertise spans Hadoop, MapReduce, Kafka, Spark Streaming, Hive, Pig, Giraph, Oozie, and Apache Beam, with a proven track record of handling petabytes of data across various industries.
Click here to give some Feedback so we can make it even better next year!