Snowpark Scala API Reference 1.21.0 - com.snowflake.snowpark.DataFrameWriter
Keywords: Snowpark Scala API Reference 1.21.0 com.snowflake.snowpark.DataFrameWriter

Snowpark Scala API Reference1.21.0 < Back
 

# Packages

-  package root
  Definition Classes
    root
-  package com
  Definition Classes
    root
-  package snowflake
  Definition Classes
    com
-  package snowpark
  Definition Classes
    snowflake
-  package types

  This package contains all Snowpark logical types.

  This package contains all Snowpark logical types.

  Definition Classes
    snowpark
  Since

    0.1.0

-  package udaf
  Definition Classes
    snowpark
-  package udtf
  Definition Classes
    snowpark
- AggregateFunction
- AsyncJob
- CaseExpr
- Column
- CopyableDataFrame
- CopyableDataFrameAsyncActor
- DataFrame
- DataFrameAsyncActor
- DataFrameNaFunctions
- DataFrameReader
- DataFrameStatFunctions
- DataFrameWriter
- DataFrameWriterAsyncActor
- DeleteResult
- FileOperation
- GetResult
- GroupingSets
- HasCachedResult
- MatchedClauseBuilder
- MergeBuilder
- MergeBuilderAsyncActor
- MergeResult
- MergeTypedAsyncJob
- NotMatchedClauseBuilder
- PublicPreview
- PutResult
- RelationalGroupedDataFrame
- Row
- SProcRegistration
- SaveMode
- Session
- SnowparkClientException
- StoredProcedure
- TableFunction
- TypedAsyncJob
- UDAFRegistration
- UDFRegistration
- UDTFRegistration
- Updatable
- UpdatableAsyncActor
- UpdateResult
- UserDefinedFunction
- Window
- WindowSpec
- WriteFileResult
- functions
- tableFunctions

c

com.snowflake.snowpark

# DataFrameWriter 

###

#### class DataFrameWriter extends AnyRef

Provides methods for writing data from a DataFrame to supported output destinations.

You can write data to the following locations:

- A Snowflake table
- A file on a stage

### Saving Data to a Table

To use this object to write into a table:

1. Access an instance of a DataFrameWriter by calling the DataFrame.write method.
2. Specify the save mode to use (overwrite or append) by calling the mode method. This method returns a DataFrameWriter that is configured to save data using the specified mode. The default SaveMode is SaveMode.Append.
3. (Optional) If you need to set some options for the save operation (e.g. columnOrder), call the options or option method.
4. Call a `saveAs*` method to save the data to the specified destination.

For example:

    df.write.mode("overwrite").saveAsTable("T")

### Saving Data to a File on a Stage

To save data to a file on a stage:

1. Access an instance of a DataFrameWriter by calling the DataFrame.write method.
2. Specify the save mode to use (Overwrite or ErrorIfExists) by calling the mode method. This method returns a DataFrameWriter that is configured to save data using the specified mode. The default SaveMode is SaveMode.ErrorIfExists for this case.
3. (Optional) If you need to set some options for the save operation (e.g. file format options), call the options or option method.
4. Call the method named after a file format to save the data in the specified format:
   - To save the data in CSV format, call the csv method.
   - To save the data in JSON format, call the json method.
   - To save the data in PARQUET format, call the parquet method.

For example:

Example 1: Write a DataFrame to a CSV file.

    val result = df.write.csv("@myStage/prefix")

Example 2: Write a DataFrame to a CSV file without compression.

    val result = df.write.option("compression", "none").csv("@myStage/prefix")

Since

  0.1.0

Linear Supertypes
AnyRef, Any
 
Ordering

1. Alphabetic
2. By Inheritance

Inherited

1. DataFrameWriter
2. AnyRef
3. Any

1. Hide All
2. Show All

Visibility

1. Public
2. All

### Instance Constructors

1.  new DataFrameWriter(dataFrame: DataFrame)

   dataFrame

     Input DataFrame

### Value Members

1.  final def !=(arg0: Any): Boolean
   Definition Classes
     AnyRef → Any
2.  final def ##(): Int
   Definition Classes
     AnyRef → Any
3.  final def ==(arg0: Any): Boolean
   Definition Classes
     AnyRef → Any
4.  def action[T](funcName: String)(func: ⇒ T): T
   Attributes
     protected
   Annotations
     @inline()
5.  final def asInstanceOf[T0]: T0
   Definition Classes
     Any
6.  def async: DataFrameWriterAsyncActor

   Returns a DataFrameWriterAsyncActor object that can be used to execute DataFrameWriter actions asynchronously.

   Returns a DataFrameWriterAsyncActor object that can be used to execute DataFrameWriter actions asynchronously.

   Example:

       val asyncJob = df.write.mode(SaveMode.Overwrite).async.saveAsTable(tableName)
       // At this point, the thread is not blocked. You can perform additional work before
       // calling asyncJob.getResult() to retrieve the results of the action.
       // NOTE: getResult() is a blocking call.
       asyncJob.getResult()

   returns

     A DataFrameWriterAsyncActor object

   Since

     0.11.0

7.  def clone(): AnyRef
   Attributes
     protected[lang]
   Definition Classes
     AnyRef
   Annotations
     @throws( ... ) @native() @HotSpotIntrinsicCandidate()
8.  def csv(path: String): WriteFileResult

   Saves the contents of the DataFrame to a CSV file on a stage.

   Saves the contents of the DataFrame to a CSV file on a stage.

   Example 1: Write a DataFrame to a CSV file.

       val result = df.write.csv("@myStage/prefix")

   Example 2: Write a DataFrame to a CSV file without compression.

       val result = df.write.option("compression", "none").csv("@myStage/prefix")

   path

     The path (including the stage name) to the CSV file.

   returns

     A WriteFileResult

   Since

     1.5.0

9.  final def eq(arg0: AnyRef): Boolean
   Definition Classes
     AnyRef
10.  def equals(arg0: Any): Boolean
    Definition Classes
      AnyRef → Any
11.  final def getClass(): Class[_]
    Definition Classes
      AnyRef → Any
    Annotations
      @native() @HotSpotIntrinsicCandidate()
12.  def hashCode(): Int
    Definition Classes
      AnyRef → Any
    Annotations
      @native() @HotSpotIntrinsicCandidate()
13.  final def isInstanceOf[T0]: Boolean
    Definition Classes
      Any
14.  def json(path: String): WriteFileResult

    Saves the contents of the DataFrame to a JSON file on a stage.

    Saves the contents of the DataFrame to a JSON file on a stage.

    NOTE: You can call this method only on a DataFrame that contains a column of the type Variant, Array, or Map. If the DataFrame does not contain a column of one of these types, you must call the `to_variant`, `array_construct`, or `object_construct` to return a DataFrame that contains a column of one of these types.

    Example 1: Write a DataFrame with one variant to a JSON file.

        val result = session.sql("select to_variant('a')").write.json("@myStage/prefix")

    Example 2: Transform a DataFrame with some columns with array_construct() and write to a JSON file without compression.

        val df = Seq((1, 1.1, "a"), (2, 2.2, "b")).toDF("a", "b", "c")
        val df2 = df.select(array_construct(df.schema.names.map(df(_)): _*))
        val result = df2.write.option("compression", "none").json("@myStage/prefix")

    Example 3: Transform a DataFrame with some columns with object_construct() and write to a JSON file without compression.

        val df = Seq((1, 1.1, "a"), (2, 2.2, "b")).toDF("a", "b", "c")
        val df2 = df.select(object_construct(df.schema.names.map(x => Seq(lit(x), df(x))).flatten: _*))
        val result = df2.write.option("compression", "none").json("@myStage/prefix")

    path

      The path (including the stage name) to the JSON file.

    returns

      A WriteFileResult

    Since

      1.5.0

15.  def mode(saveMode: SaveMode): DataFrameWriter

    Returns a new DataFrameWriter with the specified save mode configuration.

    Returns a new DataFrameWriter with the specified save mode configuration.

    saveMode

      One of the following save modes: SaveMode.Append, SaveMode.Overwrite, SaveMode.ErrorIfExists, SaveMode.Ignore

    Since

      0.1.0

16.  def mode(saveMode: String): DataFrameWriter

    Returns a new DataFrameWriter with the specified save mode configuration.

    Returns a new DataFrameWriter with the specified save mode configuration.

    saveMode

      One of the following strings: `"APPEND"`, `"OVERWRITE"`, `"ERRORIFEXISTS"`, or `"IGNORE"`

    Since

      0.1.0

17.  final def ne(arg0: AnyRef): Boolean
    Definition Classes
      AnyRef
18.  final def notify(): Unit
    Definition Classes
      AnyRef
    Annotations
      @native() @HotSpotIntrinsicCandidate()
19.  final def notifyAll(): Unit
    Definition Classes
      AnyRef
    Annotations
      @native() @HotSpotIntrinsicCandidate()
20.  def option(key: String, value: Any): DataFrameWriter

    Sets the specified option in the DataFrameWriter.

    Sets the specified option in the DataFrameWriter.

    ### Sets the specified option for saving data to a table

    Use this method to configure options:

    - columnOrder: save data into a table with table's column name order if saveMode is Append and target table exists.

    ### Sets the specified option for saving data to a file on a stage

    Use this method to configure options:

    - format-specific options
    - copy options
    - PARTITION BY or HEADER

    Note that you cannot use the `option` and `options` methods to set the following options:

    - The `TYPE` format type option.
    - The `OVERWRITE` copy option. To set this option, use the mode method instead.
      - To set `OVERWRITE` to `TRUE`, use `SaveMode.Overwrite`.
      - To set `OVERWRITE` to `FALSE`, use `SaveMode.ErrorIfExists`.

    Example 1: Write a DataFrame to a CSV file.

        val result = df.write.csv("@myStage/prefix")

    Example 2: Write a DataFrame to a CSV file without compression.

        val result = df.write.option("compression", "none").csv("@myStage/prefix")

    key

      Name of the option.

    value

      Value of the option.

    returns

      A DataFrameWriter

    Since

      1.4.0

21.  def options(configs: Map[String, Any]): DataFrameWriter

    Sets multiple specified options in the DataFrameWriter.

    Sets multiple specified options in the DataFrameWriter.

    ### Sets the specified options for saving Data to a Table

    Use this method to configure options:

    - columnOrder: save data into a table with table's column name order if saveMode is Append and target table exists.

    ### Sets the specified options for saving data to a file on a stage

    Use this method to configure options:

    - format-specific options
    - copy options
    - PARTITION BY or HEADER

    Note that you cannot use the `option` and `options` methods to set the following options:

    - The `TYPE` format type option.
    - The `OVERWRITE` copy option. To set this option, use the mode method instead.
      - To set `OVERWRITE` to `TRUE`, use `SaveMode.Overwrite`.
      - To set `OVERWRITE` to `FALSE`, use `SaveMode.ErrorIfExists`.

    Example 1: Write a DataFrame to a CSV file.

        val result = df.write.csv("@myStage/prefix")

    Example 2: Write a DataFrame to a CSV file without compression.

        val result = df.write.option("compression", "none").csv("@myStage/prefix")

    configs

      Map of the names of options (e.g. `compression` , etc.) and their corresponding values.

    returns

      A DataFrameWriter

    Since

      1.5.0

22.  def parquet(path: String): WriteFileResult

    Saves the contents of the DataFrame to a Parquet file on a stage.

    Saves the contents of the DataFrame to a Parquet file on a stage.

    Example 1: Write a DataFrame to a parquet file.

        val result = df.write.parquet("@myStage/prefix")

    Example 2: Write a DataFrame to a Parquet file without compression.

        val result = df.write.option("compression", "LZO").parquet("@myStage/prefix")

    path

      The path (including the stage name) to the Parquet file.

    returns

      A WriteFileResult

    Since

      1.5.0

23.  def saveAsTable(multipartIdentifier: List[String]): Unit

    Writes the data to the specified table in a Snowflake database.

    Writes the data to the specified table in a Snowflake database.

    For example:

          val list = new java.util.ArrayList[String](3)
          list.add(db)
          list.add(sc)
          list.add(tableName)
        df.write.saveAsTable(list)

    multipartIdentifier

      A list of strings that specify the database name, schema name, and table name.

    Since

      0.5.0

24.  def saveAsTable(multipartIdentifier: Seq[String]): Unit

    Writes the data to the specified table in a Snowflake database.

    Writes the data to the specified table in a Snowflake database.

    For example:

        df.write.saveAsTable(Seq("db_name", "schema_name", "table_name"))

    multipartIdentifier

      A sequence of strings that specify the database name, schema name, and table name (e.g. `Seq("database_name", "schema_name", "table_name")` ).

    Since

      0.5.0

25.  def saveAsTable(tableName: String): Unit

    Writes the data to the specified table in a Snowflake database.

    Writes the data to the specified table in a Snowflake database. `tableName` can be a fully-qualified object identifier.

    For example:

        df.write.saveAsTable("db1.public_schema.table1")

    tableName

      Name of the table where the data should be saved.

    Since

      0.1.0

26.  final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
      AnyRef
27.  def toString(): String
    Definition Classes
      AnyRef → Any
28.  final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
      AnyRef
    Annotations
      @throws( ... )
29.  final def wait(arg0: Long): Unit
    Definition Classes
      AnyRef
    Annotations
      @throws( ... ) @native()
30.  final def wait(): Unit
    Definition Classes
      AnyRef
    Annotations
      @throws( ... )

### Deprecated Value Members

1.  def finalize(): Unit
   Attributes
     protected[lang]
   Definition Classes
     AnyRef
   Annotations
     @throws( classOf[java.lang.Throwable] ) @Deprecated
   Deprecated

### Inherited from AnyRef

### Inherited from Any

### Ungrouped