Apache Spark >> String

Table of Contents

In this article, we will talk about how to work with Strings in Apache Spark.
The String type is probably the type we deal with most frequently.
There are many functions for manipulating Strings.
※Based on Apache Spark 3.2.1

Apache Spark >> Column and Row

Table of Contents

In this article, we will talk about some usages of the Column and Row in Apache Spark.
※Based on Apache Spark 3.2.1

Apache Spark >> Schema

Table of Contents

In this article, we will talk about the schema of Spark DataFrame.
A Spark schema is a definition of column names and types.
In Spark, the schema can also be inferred when reading the data.

Apache Spark >> Create a Local Apache Spark Development Environment on Windows With Just Two Commands

Table of Contents

In this article, we will build a local development environment for PySpark and Jupyter Notebook using docker on windows. Just two commands are needed to create a local environment for running Apache Spark application.

Apache Spark >> SparkSession, Create DataFrame

Table of Contents

Apache Spark, written in Scala, is a general-purpose distributed computing and data processing engine. Apache Spark supports Scala, Java, Python, SQL and R language.

In this series, we will use Python as main language.