Have you worked on streaming data?

Yes, I have built real‑time data pipelines using Apache Kafka as the ingestion layer and Spark Structured Streaming for processing. Data flows from Kafka topics into Spark, where we perform transformations such as windowed aggregations, joins with static reference data, and enrichment. The results are written to a downstream sink like Delta Lake or a NoSQL store. We also use Kafka Streams for lightweight processing when latency must be under a few milliseconds. For monitoring, we expose metrics to Prometheus and set up alerts for lag and throughput. This architecture allows us to handle millions of events per second with low latency and fault tolerance.