Returns a DataFrameWriterAsyncActor object that can be used to execute DataFrameWriter actions asynchronously.

Example:

val asyncJob = df.write.mode(SaveMode.Overwrite).async.saveAsTable(tableName)
// At this point, the thread is not blocked. You can perform additional work before
// calling asyncJob.getResult() to retrieve the results of the action.
// NOTE: getResult() is a blocking call.
asyncJob.getResult()

Saves the contents of the DataFrame to a CSV file on a stage.

Example 1: Write a DataFrame to a CSV file.

val result = df.write.csv("@myStage/prefix")

Example 2: Write a DataFrame to a CSV file without compression.

val result = df.write.option("compression", "none").csv("@myStage/prefix")

Saves the contents of the DataFrame to a JSON file on a stage.

NOTE: You can call this method only on a DataFrame that contains a column of the type Variant, Array, or Map. If the DataFrame does not contain a column of one of these types, you must call the to_variant, array_construct, or object_construct to return a DataFrame that contains a column of one of these types.

Example 1: Write a DataFrame with one variant to a JSON file.

val result = session.sql("select to_variant('a')").write.json("@myStage/prefix")

Example 2: Transform a DataFrame with some columns with array_construct() and write to a JSON file without compression.

val df = Seq((1, 1.1, "a"), (2, 2.2, "b")).toDF("a", "b", "c")
val df2 = df.select(array_construct(df.schema.names.map(df(_)): _*))
val result = df2.write.option("compression", "none").json("@myStage/prefix")

Example 3: Transform a DataFrame with some columns with object_construct() and write to a JSON file without compression.

val df = Seq((1, 1.1, "a"), (2, 2.2, "b")).toDF("a", "b", "c")
val df2 = df.select(object_construct(df.schema.names.map(x => Seq(lit(x), df(x))).flatten: _*))
val result = df2.write.option("compression", "none").json("@myStage/prefix")

Returns a new DataFrameWriter with the specified save mode configuration.

Returns a new DataFrameWriter with the specified save mode configuration.

Sets the specified option in the DataFrameWriter.

Sets the specified option for saving data to a table

Use this method to configure options:

• columnOrder: save data into a table with table's column name order if saveMode is Append and target table exists.

Sets the specified option for saving data to a file on a stage

Use this method to configure options:

• format-specific options
• copy options
• PARTITION BY or HEADER

Note that you cannot use the option and options methods to set the following options:

• The TYPE format type option.
• The OVERWRITE copy option. To set this option, use the mode method instead.
  • To set OVERWRITE to TRUE, use SaveMode.Overwrite.
  • To set OVERWRITE to FALSE, use SaveMode.ErrorIfExists.

Example 1: Write a DataFrame to a CSV file.

val result = df.write.csv("@myStage/prefix")

Example 2: Write a DataFrame to a CSV file without compression.

val result = df.write.option("compression", "none").csv("@myStage/prefix")

Sets multiple specified options in the DataFrameWriter.

Sets the specified options for saving Data to a Table

Use this method to configure options:

• columnOrder: save data into a table with table's column name order if saveMode is Append and target table exists.

Sets the specified options for saving data to a file on a stage

Use this method to configure options:

• format-specific options
• copy options
• PARTITION BY or HEADER

Note that you cannot use the option and options methods to set the following options:

• The TYPE format type option.
• The OVERWRITE copy option. To set this option, use the mode method instead.
  • To set OVERWRITE to TRUE, use SaveMode.Overwrite.
  • To set OVERWRITE to FALSE, use SaveMode.ErrorIfExists.

Example 1: Write a DataFrame to a CSV file.

val result = df.write.csv("@myStage/prefix")

Example 2: Write a DataFrame to a CSV file without compression.

val result = df.write.option("compression", "none").csv("@myStage/prefix")

Saves the contents of the DataFrame to a Parquet file on a stage.

Example 1: Write a DataFrame to a parquet file.

val result = df.write.parquet("@myStage/prefix")

Example 2: Write a DataFrame to a Parquet file without compression.

val result = df.write.option("compression", "LZO").parquet("@myStage/prefix")

Writes the data to the specified table in a Snowflake database.

For example:

  val list = new java.util.ArrayList[String](3)
  list.add(db)
  list.add(sc)
  list.add(tableName)
df.write.saveAsTable(list)

Writes the data to the specified table in a Snowflake database.

For example:

df.write.saveAsTable(Seq("db_name", "schema_name", "table_name"))

Writes the data to the specified table in a Snowflake database. tableName can be a fully-qualified object identifier.

For example:

df.write.saveAsTable("db1.public_schema.table1")