↓Skip to main content

Apache Kafka 102

Apache Kafka 102 #

Agenda: Kafka workshop - 2 Brief recap of what was covered in the previous workshop:

  • Basics of Apache Kafka: the difference between Kafka and database/message queue/ service bus etc.
  • Components of Kafka cluster
  • Topics, partitions
  • Producers, consumers Messages schema
  • Serialization, different serialization formats
  • Schema registry
  • Example producer and consumer with schema Kafka Connect
  • Overview of Kafka connect => source and sink connectors, architecture.
  • Examples of source and sink connectors, introduction to connector hub.
  • Examples of Microsoft-specific source and sink connectors. Kafka streams
  • Introduction to Kafka streams
  • Example of a Kafka streams application followed by demo KSQL
  • KSQL intro and architecture
  • Difference between Kafka streams and KSQL
  • Example application + demo Kafka security
  • Different authentication options
  • Authorization
  • Quotas Questions / Networking (Lunch will be provided)

Prerequisites: #

Intellij Idea community edition Gradle Java 11 WSL ( https://learn.microsoft.com/en-us/windows/wsl/install) Apache Kafka binaries Confluent cloud account (Free credits available) Speakers: => Lakshmi Narasimhan - Principal Engineer at Platformatory who always loves data engineering at its core. Currently working on different projects around ETL, and event streaming mainly around Apache Kafka - https://www.linkedin.com/in/lakshminp/

=> Ashwin Venkatesan - Senior Consultant at Platformatory working mainly on Apache Kafka and Kubernetes. Ashwin has done multiple projects building scalable data pipelines incorporating Apache Kafka in production scenarios. He also dabbles in Data science and Machine learning in his free time. His hobbies include traveling, cooking, and following Manchester United : )

https://www.meetup.com/chennai-microsoft-azure-user-group/events/292572447/

↑