Why foreachBatch Exists in Spark Streaming

Normally, Apache Spark Structured Streaming expects you to continuously write to a sink using methods like .writeStream.format("delta").start().

MERGE operations (upserts) are not supported directly in streaming sinks. That is where foreachBatch comes in: it lets you write arbitrary batch logic, including MERGE INTO, joins, custom transformations, etc.