Snippets Collections
split_col = pyspark.sql.functions.split(df['my_str_col'], '-')
df = df.withColumn('NAME1', split_col.getItem(0))
df = df.withColumn('NAME2', split_col.getItem(1))
from pyspark.sql import SparkSession
spark = SparkSession.builder.appName('abc').getOrCreate()
star

Wed Feb 24 2021 17:36:03 GMT+0000 (UTC) https://stackoverflow.com/questions/39235704/split-spark-dataframe-string-column-into-multiple-columns

#pyspark #spark #python #etl
star

Fri Sep 04 2020 05:19:26 GMT+0000 (UTC) https://stackoverflow.com/questions/39780792/how-to-build-a-sparksession-in-spark-2-0-using-pyspark

#python #pyspark #spark #spark-session

Save snippets that work with our extensions

Available in the Chrome Web Store Get Firefox Add-on Get VS Code extension