Feb 1, 2024
Delta is not necessarily overdimensionen for trivial data solution, as Spark might be. Delta is just files and a folder. I'm seeing an increasing trends for using Spark, whether in the form of Databricks or Microsoft Fabric. The end results of nowadays Spark processing, is nearly always written to Delta. No matter size of data. There is a need for being able to access and read this data from Delta. And some times write to Delta.
I could have used Polars, but the intention is to show, how to access Delta with Python.