How to concatenate columns in Spark dataframe?
Concatenate functions in Spark SQL The module pyspark.sql.functions provides two concatenate functions as below concat – It concatenates multiple input
Continue readingCategory: Spark
How to convert Pandas dataframe to Spark dataframe?
Convert Pandas to Spark dataframe In some cases, we use Pandas library in Pyspark to perform the data analysis or
Continue readingHow to read BigQuery table using PySpark?
Spark BigQuery Connector The Google Cloud team has created the set of connectors to access the data in GCP. To
Continue readingHow to send Spark dataframe values as HTML table to Email?
Spark dataframe Spark dataframe is a distributed collection of rows with the same schema. The schema defines the structure of
Continue readingHow to write a Spark dataframe to Hive table in Pyspark?
Apache spark to write a Hive Table Apache Spark provides an option to read from Hive table as well as
Continue readingHow to access a Hive table using Pyspark?
Pyspark Apache Spark is an in-memory data processing framework written in Scala language. It process the data 100 times faster
Continue reading