com.snowflake.snowpark_java

Class DataFrameWriter

java.lang.Object

com.snowflake.snowpark_java.DataFrameWriter

public class DataFrameWriter
extends Object

Provides methods for writing data from a DataFrame to supported output destinations. You can write data to the following locations:

• A Snowflake table
• A file on a stage

Saving Data to a Table

To use this object to write into a table:

• Access an instance of a DataFrameWriter by calling the DataFrame.write method.
• Specify the save mode to use (overwrite or append) by calling the mode() method. This method returns a DataFrameWriter that is configured to save data using the specified mode. The default SaveMode is SaveMode.Append.
• (Optional) If you need to set some options for the save operation (e.g. columnOrder), call the options or option method.
• Call a saveAsTable method to save the data to the specified destination.

Saving Data to a File on a Stage

To use this object to write into a file:

• Access an instance of a DataFrameWriter by calling the DataFrame.write method.
• Specify the save mode to use (Overwrite or ErrorIfExists) by calling the DataFrame.mode method. This method returns a DataFrameWriter that is configured to save data using the specified mode. The default SaveMode is SaveMode.ErrorIfExists for this case.
• (Optional) If you need to set some options for the save operation (e.g. file format options), call the options or option method.
• Call the method named after a file format to save the data in the specified format:
  • To save the data in CSV format, call the csv method.
  • To save the data in JSON format, call the json method.
  • To save the data in PARQUET format, call the parquet method.

Since:

1.1.0

Method Summary

Modifier and Type	Method and Description
DataFrameWriterAsyncActor	async() Returns a DataFrameWriterAsyncActor object that can be used to execute DataFrameWriter actions asynchronously.
WriteFileResult	csv(String path) Saves the contents of the DataFrame to a CSV file on a stage.
WriteFileResult	json(String path) Saves the contents of the DataFrame to a JSON file on a stage.
DataFrameWriter	mode(SaveMode saveMode) Returns a new DataFrameWriter with the specified save mode configuration.
DataFrameWriter	mode(String saveMode) Returns a new DataFrameWriter with the specified save mode configuration.
DataFrameWriter	option(String key, Object value) Sets the specified option in the DataFrameWriter.
DataFrameWriter	options(Map<String,Object> configs) Sets multiple specified options in the DataFrameWriter.
WriteFileResult	parquet(String path) Saves the contents of the DataFrame to a Parquet file on a stage.
void	saveAsTable(String tableName) Writes the data to the specified table in a Snowflake database.
void	saveAsTable(String[] multipartIdentifier) Writes the data to the specified table in a Snowflake database.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

option

public DataFrameWriter option(String key,
                              Object value)

Sets the specified option in the DataFrameWriter.

Sets the specified option for saving data to a table

Use this method to configure options:

• columnOrder: save data into a table with table's column name order if saveMode is Append and the target table exists.

Sets the specified option for saving data to a file on a stage

Use this method to configure options:

• format-specific options
• copy options
• PARTITION BY or HEADER

Note that you cannot use the options or option methods to set the following options:

• The TYPE format type option.
• The OVERWRITE copy option. To set this option, use the mode method instead.
  • To set OVERWRITE to TRUE, use SaveMode.Overwrite.
  • To set OVERWRITE to FALSE, use SaveMode.ErrorIfExists.

For example:

 df.write().option("compression", "none").csv("@myStage/prefix");
 

Parameters:

key - Name of the option.

value - Value of the option.

Returns:

A DataFrameWriter

Since:

1.4.0

options

public DataFrameWriter options(Map<String,Object> configs)

Sets multiple specified options in the DataFrameWriter.

Sets the specified option for saving data to a table

Use this method to configure options:

• columnOrder: save data into a table with table's column name order if saveMode is Append and the target table exists.

Sets the specified option for saving data to a file on a stage

Use this method to configure options:

• format-specific options
• copy options
• PARTITION BY or HEADER

Note that you cannot use the options or option methods to set the following options:

• The TYPE format type option.
• The OVERWRITE copy option. To set this option, use the mode method instead.
  • To set OVERWRITE to TRUE, use SaveMode.Overwrite.
  • To set OVERWRITE to FALSE, use SaveMode.ErrorIfExists.

For example:

 Map<String, Object> configs = new HashMap<>();
 configs.put("compression", "none");
 df.write().options(configs).csv("@myStage/prefix");
 

Parameters:

configs - Map of the names of options (e.g. compression, etc.) and their corresponding values.

Returns:

A DataFrameWriter

Since:

1.5.0

csv

public WriteFileResult csv(String path)

Saves the contents of the DataFrame to a CSV file on a stage.

For example:

 df.write().option("compression", "none").csv("@myStage/prefix");
 

Parameters:

path - The path (including the stage name) to the CSV file.

Returns:

A WriteFileResult

Since:

1.5.0

json

public WriteFileResult json(String path)

Saves the contents of the DataFrame to a JSON file on a stage.

NOTE: You can call this method only on a DataFrame that contains a column of the type Variant, Array, or Map. If the DataFrame does not contain a column of one of these types, you must call the to_variant, array_construct, or object_construct to return a DataFrame that contains a column of one of these types.

For example:

 df.write().option("compression", "none").json("@myStage/prefix");
 

Parameters:

path - The path (including the stage name) to the JSON file.

Returns:

A WriteFileResult

Since:

1.5.0

parquet

public WriteFileResult parquet(String path)

Saves the contents of the DataFrame to a Parquet file on a stage.

For example:

 df.write().option("compression", "lzo").parquet("@myStage/prefix");
 

Parameters:

path - The path (including the stage name) to the Parquet file.

Returns:

A WriteFileResult

Since:

1.5.0

saveAsTable

public void saveAsTable(String tableName)

Writes the data to the specified table in a Snowflake database. tableName can be a fully-qualified object identifier.

For example:

 df.write().saveAsTable("db1.public_schema.table1");
 

Parameters:

tableName - Name of the table where the data should be saved.

Since:

1.1.0

saveAsTable

public void saveAsTable(String[] multipartIdentifier)

Writes the data to the specified table in a Snowflake database.

For example:

 df.write().saveAsTable(new String[]{"db_name", "schema_name", "table_name"})
 

Parameters:

multipartIdentifier - A sequence of strings that specify the database name, schema name, and table name.

Since:

1.1.0

mode

public DataFrameWriter mode(String saveMode)

Returns a new DataFrameWriter with the specified save mode configuration.

Parameters:

saveMode - One of the following strings: `"APPEND"`, `"OVERWRITE"`, `"ERRORIFEXISTS"`, or `"IGNORE"`

Returns:

This DataFrameWriter

Since:

1.1.0

mode

public DataFrameWriter mode(SaveMode saveMode)

Returns a new DataFrameWriter with the specified save mode configuration.

Parameters:

saveMode - One of the following save modes: SaveMode.Append, SaveMode.Overwrite, SaveMode.ErrorIfExists, SaveMode.Ignore

Returns:

This DataFrameWriter

Since:

1.1.0

async

public DataFrameWriterAsyncActor async()

Returns a DataFrameWriterAsyncActor object that can be used to execute DataFrameWriter actions asynchronously.

Returns:

A DataFrameWriterAsyncActor object

Since:

1.2.0