Reading CSV Files¶
Reading csv files in Apache Spark is simple. In this example we’ll be using the diamonds dataset available as a Databricks Dataset. All we have to do is specify the path as well as any options that we would like.
Sometimes you may find yourself with a variety of csv files in one folder. In Apache Spark, you can read that entire directory of csv files by specifying the directory as the file location as opposed to individual files!