DataFrame
Namespace: Polars.FSharp
An eager DataFrame holding data in memory.
DataFrames are 2D tabular data structures with named columns of potentially different types.
Properties & Indexers
| Property | Type | Description |
|---|---|---|
ColumnNames | string[] | - |
Columns | string[] | - |
DataTypes | DataType list | - |
Height | int64 | - |
IsEmpty | bool | True if the DataFrame contains no rows. |
IsVoid | bool | - |
Schema | PolarsSchema | Get the schema |
Shape | (int64 * int64) | Returns the shape of the DataFrame as (Height, Width). |
Width | int64 | - |
Constructors & Static Methods
ConcatDiagonal
Diagonal concatenation.
static member ConcatDiagonal(dfs: seq<DataFrame>) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
dfs | seq<DataFrame> | - |
ConcatHorizontal
Horizontal concatenation (Index alignment).
static member ConcatHorizontal(
dfs: seq<DataFrame>,
?checkDuplicates: bool,
?strict: bool,
?unitLengthAsScalar: bool
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
dfs | seq<DataFrame> | - |
checkDuplicates | bool (optional) | - |
strict | bool (optional) | For Horizontal: if true, error on height mismatch. |
unitLengthAsScalar | bool (optional) | For Horizontal: if true, broadcast length-1 DataFrames to match height. |
ConcatVertical
Vertical concatenation (Column alignment).
static member ConcatVertical(dfs: seq<DataFrame>) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
dfs | seq<DataFrame> | - |
FromArrow
Create a DataFrame directly from an Apache Arrow RecordBatch.
static member FromArrow(batch: RecordBatch) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
batch | RecordBatch | - |
FromArrowStream
static member FromArrowStream(stream: IArrowArrayStream) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
stream | IArrowArrayStream | - |
JsonNormalize
Normalize a single JSON object (represented as a Map) into a DataFrame.
Normalize a sequence of JSON objects into a DataFrame.
static member JsonNormalize(
data: Map<string, obj>,
?separator: string,
?maxLevel: int,
?schema: PolarsSchema,
?strict: bool,
?inferSchemaLength: uint32,
?encoder: (obj -> string)
) : DataFrame
static member JsonNormalize(
data: seq<Map<string, obj>>,
?separator: string,
?maxLevel: int,
?schema: PolarsSchema,
?strict: bool,
?inferSchemaLength: uint32,
?encoder: (obj -> string)
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
data | Map<string, obj> / seq<Map<string, obj>> | A single JSON object. |
separator | string (optional) | Separator for nested fields. |
maxLevel | int (optional) | Max nesting level. |
schema | PolarsSchema (optional) | Optional schema for the output. |
strict | bool (optional) | Strict mode for type inference. |
inferSchemaLength | uint32 (optional) | Rows used for schema inference. |
encoder | (obj -> string) (optional) | Custom value-to-string encoder. |
JsonNormalizeFromString
Normalize a JSON string representing a single object or an array of objects.
static member JsonNormalizeFromString(
jsonString: string,
?separator: string,
?maxLevel: int,
?schema: PolarsSchema,
?strict: bool,
?inferSchemaLength: uint32,
?encoder: (obj -> string)
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
jsonString | string | A valid JSON string. |
separator | string (optional) | Separator for nested fields. |
maxLevel | int (optional) | Max nesting level. |
schema | PolarsSchema (optional) | Optional schema for the output. |
strict | bool (optional) | Strict mode for type inference. |
inferSchemaLength | uint32 (optional) | Rows used for schema inference. |
encoder | (obj -> string) (optional) | Custom value-to-string encoder. |
ReadAdbc
Generate DataFrame from ADBC query results
Executes a SQL query directly against an ADBC connection and reads the result into a zero-copy Polars DataFrame. Pure syntactic sugar: automatically manages the creation and disposal of the underlying AdbcStatement.
static member ReadAdbc(statement: AdbcStatement) : DataFrame
static member ReadAdbc(connection: AdbcConnection, sqlQuery: string) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
statement | AdbcStatement | - |
connection | AdbcConnection | The active ADBC connection (e.g., DuckDB, SQLite). |
sqlQuery | string | The SQL query string to execute. |
ReadAvro
Read an Avro file into a DataFrame.
Read an Avro memory buffer into a DataFrame.
Read an Avro memory Stream into a DataFrame.
static member ReadAvro(
path: string,
?nRows: uint64,
?columns: seq<string>,
?projection: seq<int>
) : DataFrame
static member ReadAvro(
buffer: byte[],
?nRows: uint64,
?columns: seq<string>,
?projection: seq<int>
) : DataFrame
static member ReadAvro(
stream: Stream,
?nRows: uint64,
?columns: seq<string>,
?projection: seq<int>
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | The path to the Avro file. |
nRows | uint64 (optional) | Stop reading when nRows are read. |
columns | seq<string> (optional) | Columns to select/project by name. |
projection | seq<int> (optional) | Columns to select/project by index. |
buffer | byte[] | The byte array containing Avro data. |
stream | Stream | The stream containing Avro data. |
Returns: A new DataFrame.
ReadCsv
Read a DataFrame from a CSV file.
Note: This method internally uses LazyFrame.ScanCsv and collects the result. For larger-than-memory datasets or better query optimization, consider using LazyFrame.ScanCsv directly.
Read a DataFrame from a CSV memory buffer.
Read a DataFrame from a CSV memory stream.
static member ReadCsv(
path: string,
?columns: seq<string>,
?hasHeader: bool,
?separator: char,
?quoteChar: char,
?eolChar: char,
?ignoreErrors: bool,
?tryParseDates: bool,
?lowMemory: bool,
?skipRows: uint64,
?nRows: uint64,
?inferSchemaLength: uint64,
?inferSchemaFiles: uint64,
?schema: PolarsSchema,
?dtypeOverride: PolarsSchema,
?encoding: CsvEncoding,
?nullValues: seq<string>,
?missingIsNull: bool,
?commentPrefix: string,
?decimalComma: bool,
?truncateRaggedLines: bool,
?rowIndexName: string,
?rowIndexOffset: uint64,
?cloudOptions: CloudOptions
) : DataFrame
static member ReadCsv(
buffer: byte[],
?columns: seq<string>,
?hasHeader: bool,
?separator: char,
?quoteChar: char,
?eolChar: char,
?ignoreErrors: bool,
?tryParseDates: bool,
?lowMemory: bool,
?skipRows: uint64,
?nRows: uint64,
?inferSchemaLength: uint64,
?inferSchemaFiles: uint64,
?schema: PolarsSchema,
?dtypeOverride: PolarsSchema,
?encoding: CsvEncoding,
?nullValues: seq<string>,
?missingIsNull: bool,
?commentPrefix: string,
?decimalComma: bool,
?truncateRaggedLines: bool,
?rowIndexName: string,
?rowIndexOffset: uint64
) : DataFrame
static member ReadCsv(
stream: Stream,
?columns: seq<string>,
?hasHeader: bool,
?separator: char,
?quoteChar: char,
?eolChar: char,
?ignoreErrors: bool,
?tryParseDates: bool,
?lowMemory: bool,
?skipRows: uint64,
?nRows: uint64,
?inferSchemaLength: uint64,
?inferSchemaFiles: uint64,
?schema: PolarsSchema,
?encoding: CsvEncoding,
?nullValues: seq<string>,
?missingIsNull: bool,
?commentPrefix: string,
?decimalComma: bool,
?truncateRaggedLines: bool,
?rowIndexName: string,
?rowIndexOffset: uint64
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
columns | seq<string> (optional) | - |
hasHeader | bool (optional) | - |
separator | char (optional) | - |
quoteChar | char (optional) | - |
eolChar | char (optional) | - |
ignoreErrors | bool (optional) | - |
tryParseDates | bool (optional) | - |
lowMemory | bool (optional) | - |
skipRows | uint64 (optional) | - |
nRows | uint64 (optional) | - |
inferSchemaLength | uint64 (optional) | - |
inferSchemaFiles | uint64 (optional) | - |
schema | PolarsSchema (optional) | - |
dtypeOverride | PolarsSchema (optional) | - |
encoding | CsvEncoding (optional) | - |
nullValues | seq<string> (optional) | - |
missingIsNull | bool (optional) | - |
commentPrefix | string (optional) | - |
decimalComma | bool (optional) | - |
truncateRaggedLines | bool (optional) | - |
rowIndexName | string (optional) | - |
rowIndexOffset | uint64 (optional) | - |
cloudOptions | CloudOptions (optional) | - |
buffer | byte[] | - |
stream | Stream | - |
ReadCsvAsync
Read a CSV file asynchronously into a DataFrame.
static member ReadCsvAsync(
path: string,
?columns: seq<string>,
?hasHeader: bool,
?separator: char,
?quoteChar: char,
?eolChar: char,
?ignoreErrors: bool,
?tryParseDates: bool,
?lowMemory: bool,
?skipRows: uint64,
?nRows: uint64,
?inferSchemaLength: uint64,
?inferSchemaFiles: uint64,
?schema: PolarsSchema,
?encoding: CsvEncoding,
?nullValues: seq<string>,
?missingIsNull: bool,
?commentPrefix: string,
?decimalComma: bool,
?truncateRaggedLines: bool,
?rowIndexName: string,
?rowIndexOffset: uint64,
?cloudOptions: CloudOptions
) : Task<DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
path | string | - |
columns | seq<string> (optional) | - |
hasHeader | bool (optional) | - |
separator | char (optional) | - |
quoteChar | char (optional) | - |
eolChar | char (optional) | - |
ignoreErrors | bool (optional) | - |
tryParseDates | bool (optional) | - |
lowMemory | bool (optional) | - |
skipRows | uint64 (optional) | - |
nRows | uint64 (optional) | - |
inferSchemaLength | uint64 (optional) | - |
inferSchemaFiles | uint64 (optional) | - |
schema | PolarsSchema (optional) | - |
encoding | CsvEncoding (optional) | - |
nullValues | seq<string> (optional) | - |
missingIsNull | bool (optional) | - |
commentPrefix | string (optional) | - |
decimalComma | bool (optional) | - |
truncateRaggedLines | bool (optional) | - |
rowIndexName | string (optional) | - |
rowIndexOffset | uint64 (optional) | - |
cloudOptions | CloudOptions (optional) | - |
ReadDb
[Eager] Create a DataFrame from an IDataReader (e.g. SqlDataReader).
Uses high-performance streaming ingestion via Apache Arrow.
static member ReadDb(reader: IDataReader, ?batchSize: int) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
reader | IDataReader | The open IDataReader instance. |
batchSize | int (optional) | Number of rows per Arrow batch (default 50,000). |
ReadDelta
Create a DataFrame by reading a Delta Lake table.
static member ReadDelta(
path: string,
?version: uint64,
?datetime: string,
?nRows: uint64,
?parallelStrategy: ParallelStrategy,
?lowMemory: bool,
?useStatistics: bool,
?glob: bool,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?schema: PolarsSchema,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool,
?cloudOptions: CloudOptions
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | Path to the Delta Lake table (folder containing _delta_log). |
version | uint64 (optional) | The version of the table to read (e.g., 0L, 1L). Mutually exclusive with datetime. |
datetime | string (optional) | The timestamp to read (ISO-8601 string, e.g., "2026-02-09T12:00:00Z"). Mutually exclusive with version. |
nRows | uint64 (optional) | - |
parallelStrategy | ParallelStrategy (optional) | - |
lowMemory | bool (optional) | - |
useStatistics | bool (optional) | - |
glob | bool (optional) | - |
rechunk | bool (optional) | - |
cache | bool (optional) | - |
rowIndexName | string (optional) | - |
rowIndexOffset | uint32 (optional) | - |
includePathColumn | string (optional) | - |
schema | PolarsSchema (optional) | - |
hivePartitioning | bool (optional) | - |
hivePartitionSchema | PolarsSchema (optional) | - |
tryParseHiveDates | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
Returns: A new LazyFrame.
ReadExcel
Read an Excel file (.xlsx) into a DataFrame using the high-performance Rust 'calamine' engine.
static member ReadExcel(
path: string,
?sheetName: string,
?sheetIndex: uint64,
?schema: PolarsSchema,
?hasHeader: bool,
?inferSchemaLen: uint64,
?dropEmptyRows: bool,
?raiseIfEmpty: bool
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | Path to the .xlsx file. |
sheetName | string (optional) | Name of the sheet to read. If specified, takes precedence over sheetIndex. |
sheetIndex | uint64 (optional) | Index of the sheet to read (0-based). Default is 0. |
schema | PolarsSchema (optional) | Optional schema overrides to enforce specific column types. |
hasHeader | bool (optional) | Indicates if the first row contains header names. Default is true. |
inferSchemaLen | uint64 (optional) | Number of rows to use for schema inference. Default is 100. |
dropEmptyRows | bool (optional) | If true, drop rows where all cells are empty/null. Default is true. |
raiseIfEmpty | bool (optional) | If true, raises an error if the sheet is empty. Default is true. |
ReadIpc
Read an Arrow IPC (Feather v2) file into a DataFrame.
Note: This method internally uses LazyFrame.ScanIpc and collects the result.
Read IPC from in-memory bytes.
Read IPC from a Stream.
static member ReadIpc(
path: string,
?columns: seq<string>,
?nRows: uint64,
?rechunk: bool,
?cache: bool,
?glob: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?schema: PolarsSchema,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool,
?cloudOptions: CloudOptions
) : DataFrame
static member ReadIpc(
buffer: byte[],
?columns: seq<string>,
?nRows: uint64,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?schema: PolarsSchema,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool
) : DataFrame
static member ReadIpc(
stream: Stream,
?columns: seq<string>,
?nRows: uint64,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?schema: PolarsSchema,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
columns | seq<string> (optional) | - |
nRows | uint64 (optional) | - |
rechunk | bool (optional) | - |
cache | bool (optional) | - |
glob | bool (optional) | - |
rowIndexName | string (optional) | - |
rowIndexOffset | uint32 (optional) | - |
includePathColumn | string (optional) | - |
schema | PolarsSchema (optional) | - |
hivePartitioning | bool (optional) | - |
hivePartitionSchema | PolarsSchema (optional) | - |
tryParseHiveDates | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
buffer | byte[] | - |
stream | Stream | - |
ReadIpcStream
Read into a DataFrame from Arrow IPC record batch stream.
static member ReadIpcStream(
path: string,
?columns: seq<string>,
?projection: seq<uint32>,
?nRows: uint64,
?rowIndexName: string,
?rowIndexOffset: uint32,
?rechunk: bool
) : DataFrame
static member ReadIpcStream(
buffer: ReadOnlySpan<byte>,
?columns: seq<string>,
?projection: seq<uint32>,
?nRows: uint64,
?rowIndexName: string,
?rowIndexOffset: uint32,
?rechunk: bool
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | Path to the IPC stream file or a byte buffer |
columns | seq<string> (optional) | Columns to select. |
projection | seq<uint32> (optional) | Column indices to select. |
nRows | uint64 (optional) | Stop reading from IPC stream after reading n rows |
rowIndexName | string (optional) | Insert a row index column with the given name into the DataFrame as the first column. If set to None (default), no row index column is created. |
rowIndexOffset | uint32 (optional) | Start the row index at this offset.Only used if row_index_name is set. |
rechunk | bool (optional) | Make sure that all data is contiguous. |
buffer | ReadOnlySpan<byte> | - |
ReadIpcStreamSchema
Get the schema of an IPC file without reading data.
static member ReadIpcStreamSchema(path: string) : PolarsSchema
static member ReadIpcStreamSchema(buffer: ReadOnlySpan<byte>) : PolarsSchemaParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
buffer | ReadOnlySpan<byte> | - |
ReadJson
Read a JSON file into a DataFrame.
Read JSON from in-memory bytes.
Read JSON from a Stream.
static member ReadJson(
path: string,
?columns: seq<string>,
?schema: PolarsSchema,
?inferSchemaLen: uint64,
?batchSize: uint64,
?ignoreErrors: bool,
?jsonFormat: JsonFormat
) : DataFrame
static member ReadJson(
buffer: byte[],
?columns: seq<string>,
?schema: PolarsSchema,
?inferSchemaLen: uint64,
?batchSize: uint64,
?ignoreErrors: bool,
?jsonFormat: JsonFormat
) : DataFrame
static member ReadJson(
stream: Stream,
?columns: seq<string>,
?schema: PolarsSchema,
?inferSchemaLen: uint64,
?batchSize: uint64,
?ignoreErrors: bool,
?jsonFormat: JsonFormat
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
columns | seq<string> (optional) | - |
schema | PolarsSchema (optional) | - |
inferSchemaLen | uint64 (optional) | - |
batchSize | uint64 (optional) | - |
ignoreErrors | bool (optional) | - |
jsonFormat | JsonFormat (optional) | - |
buffer | byte[] | - |
stream | Stream | - |
ReadParquet
Read a parquet file into a DataFrame (Eager).
Note: This method internally uses LazyFrame.ScanParquet and collects the result.
Read Parquet from an in-memory byte array.
Read Parquet from a Stream.
static member ReadParquet(
path: string,
?columns: seq<string>,
?nRows: uint64,
?parallelStrategy: ParallelStrategy,
?lowMemory: bool,
?useStatistics: bool,
?glob: bool,
?allowMissingColumns: bool,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?schema: PolarsSchema,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool,
?cloudOptions: CloudOptions
) : DataFrame
static member ReadParquet(
buffer: byte[],
?columns: seq<string>,
?nRows: uint64,
?parallelStrategy: ParallelStrategy,
?lowMemory: bool,
?useStatistics: bool,
?allowMissingColumns: bool,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?schema: PolarsSchema,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool
) : DataFrame
static member ReadParquet(
stream: Stream,
?columns: seq<string>,
?nRows: uint64,
?parallelStrategy: ParallelStrategy,
?lowMemory: bool,
?useStatistics: bool,
?allowMissingColumns: bool,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?schema: PolarsSchema,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
columns | seq<string> (optional) | - |
nRows | uint64 (optional) | - |
parallelStrategy | ParallelStrategy (optional) | - |
lowMemory | bool (optional) | - |
useStatistics | bool (optional) | - |
glob | bool (optional) | - |
allowMissingColumns | bool (optional) | - |
rechunk | bool (optional) | - |
cache | bool (optional) | - |
rowIndexName | string (optional) | - |
rowIndexOffset | uint32 (optional) | - |
includePathColumn | string (optional) | - |
schema | PolarsSchema (optional) | - |
hivePartitioning | bool (optional) | - |
hivePartitionSchema | PolarsSchema (optional) | - |
tryParseHiveDates | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
buffer | byte[] | - |
stream | Stream | - |
ReadParquetAsync
Asynchronously read a Parquet file.
static member ReadParquetAsync(
path: string,
?columns: seq<string>,
?nRows: uint64,
?parallelStrategy: ParallelStrategy,
?lowMemory: bool,
?useStatistics: bool,
?glob: bool,
?allowMissingColumns: bool,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?schema: PolarsSchema,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool,
?cloudOptions: CloudOptions
) : Task<DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
path | string | - |
columns | seq<string> (optional) | - |
nRows | uint64 (optional) | - |
parallelStrategy | ParallelStrategy (optional) | - |
lowMemory | bool (optional) | - |
useStatistics | bool (optional) | - |
glob | bool (optional) | - |
allowMissingColumns | bool (optional) | - |
rechunk | bool (optional) | - |
cache | bool (optional) | - |
rowIndexName | string (optional) | - |
rowIndexOffset | uint32 (optional) | - |
includePathColumn | string (optional) | - |
schema | PolarsSchema (optional) | - |
hivePartitioning | bool (optional) | - |
hivePartitionSchema | PolarsSchema (optional) | - |
tryParseHiveDates | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
create
Create a DataFrame from a list of Series.
Create a DataFrame from an array of Series.
static member create(series: seq<Series>) : DataFrame
static member create([<ParamArray>] series: Series[]) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
series | seq<Series> / Series[] | - |
ofColumns
Creates a DataFrame from an F# anonymous record (or any SoA container) where fields are collections. Example: DataFrame.ofColumns {| Time = [| dt1; dt2 |]; Val = [| 1.0; 2.0 |] |}
Creates a DataFrame from a sequence of column pairs (Name * Array). Example: DataFrame.ofColumns [ "A", box [| 1; 2 |]; "B", box [| "x"; "y" |] ]
static member ofColumns(columns: 'T) : DataFrame
static member ofColumns(columns: seq<(string * Array)>) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | 'T / seq<(string * Array)> | - |
ofMaps
Build a DataFrame from a sequence of Maps or dictionaries.
static member ofMaps(data: seq<'a>, ?strict: bool, ?inferSchemaLength: uint32) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
data | seq<'a> | - |
strict | bool (optional) | - |
inferSchemaLength | uint32 (optional) | - |
ofRecords
Creates a DataFrame from an array or sequence of F# records. Uses Zero-Copy fast-path if all columns are supported primitives, dates, or options. Falls back to Arrow serialization for complex nested structs and lists.
static member ofRecords(data: seq<'T>) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
data | seq<'T> | - |
ofSeq
Create a DataFrame from a sequence of objects using Arrow streaming.
static member ofSeq(data: seq<'T>, ?batchSize: int) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
data | seq<'T> | - |
batchSize | int (optional) | - |
Instance Methods
AsDataReader
Export DataFrame As DbDataReader (Zero-Copy Enabled)
member AsDataReader(?bufferSize: int, ?typeOverrides: Dictionary<string, Type>) : DbDataReaderParameters
| Name | Type | Description |
|---|---|---|
bufferSize | int (optional) | - |
typeOverrides | Dictionary<string, Type> (optional) | - |
AsQueryable
Convert DataFrame into an IQueryable LINQ query provider. Operations will be translated to Polars Expressions and compiled into execution plans.
Directly creates an IQueryable with type inferred from an existing sample sequence
member AsQueryable : unit -> IQueryable<'T>
member AsQueryable(sample: seq<'T>) : IQueryable<'T>Parameters
| Name | Type | Description |
|---|---|---|
sample | seq<'T> | - |
AsTensor
Converts selected DataFrame columns into a managed 2D Row-Major Tensor [Rows, Columns]. Automatically handles memory fragmentation and performs the Column-Major to Row-Major transposition.
Converts all columns in the DataFrame into a managed 2D Row-Major Tensor.
member AsTensor([<ParamArray>] columnNames: string[]) : Tensor<'T>
member AsTensor : unit -> Tensor<'T>Parameters
| Name | Type | Description |
|---|---|---|
columnNames | string[] | - |
BottomK
Get the bottom k rows based on the given columns.
member BottomK(k: int, by: seq<'a>, ?reverse: seq<bool>) : DataFrame
member BottomK(k: int, by: seq<'a>, reverse: bool) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
k | int | - |
by | seq<'a> | - |
reverse | seq<bool> (optional) | - |
Cell
Get a value from the DataFrame using a generic type argument. Eliminates the need for unbox, but throws if type mismatches.
member Cell(colName: string, rowIndex: int) : 'T
member Cell(rowIndex: int, colName: string) : 'TParameters
| Name | Type | Description |
|---|---|---|
colName | string | - |
rowIndex | int | - |
CellList
Helper to get a cell value as an F# List directly.
member CellList(colName: string, row: int) : 'T listParameters
| Name | Type | Description |
|---|---|---|
colName | string | - |
row | int | - |
Clear
Create an empty (n=0) or n-row null-filled (n>0) copy of the DataFrame. Returns a n-row null-filled DataFrame with an identical schema.
member Clear(?n: int64) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
n | int64 (optional) | Number of (null-filled) rows to return in the cleared frame. |
Returns: A new DataFrame.
Clone
member Clone : unit -> DataFrameColumn
member Column(name: string) : Series
member Column(index: int64) : SeriesParameters
| Name | Type | Description |
|---|---|---|
name | string | - |
index | int64 | - |
Count
Return the number of non-null elements for each column.
member Count : unit -> DataFrameDescribe
Generate a summary statistics DataFrame (count, mean, std, min, 25%, 50%, 75%, max). Similar to pandas/polars describe().
member Describe : unit -> DataFrameDrop
Drop one or more columns from the DataFrame. Returns a new DataFrame.
Drop columns using Polars Selectors or Expressions.
member Drop([<ParamArray>] columns: string[]) : DataFrame
member Drop(exprs: seq<Expr>) : DataFrame
member Drop([<ParamArray>] exprs: Expr[]) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | string[] | - |
exprs | seq<Expr> / Expr[] | - |
DropInPlace
Drop a column in-place and return it as a Series. Note: This mutates the original DataFrame.
member DropInPlace(name: string) : SeriesParameters
| Name | Type | Description |
|---|---|---|
name | string | - |
DropNans
Drop rows containing one or more NaN values.
Drop rows with NaN in specific columns.
Drop rows with NaN by specific Expressions.
member DropNans(?subset: Selector) : DataFrame
member DropNans([<ParamArray>] subset: string[]) : DataFrame
member DropNans([<ParamArray>] exprs: Expr[]) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
subset | Selector (optional) / string[] | - |
exprs | Expr[] | - |
DropNulls
Drop rows containing one or more Null values.
Drop rows with Nulls in specific columns.
Drop rows with Nulls by specific Expressions.
member DropNulls(?subset: Selector) : DataFrame
member DropNulls([<ParamArray>] subset: string[]) : DataFrame
member DropNulls([<ParamArray>] exprs: Expr[]) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
subset | Selector (optional) / string[] | - |
exprs | Expr[] | - |
EstimatedSize
Return an estimation of the total (heap) allocated size of the DataFrame. Estimated size is given in the specified unit (bytes by default).
member EstimatedSize(unit: SizeUnit) : floatParameters
| Name | Type | Description |
|---|---|---|
unit | SizeUnit | Scale the returned size to the given unit (uses 1024 base). |
Returns: The estimated size as a double.
Explode
Explode list columns to rows using a Selector.
Explode list columns to rows using column names.
Explode a single column by name.
member Explode(selector: Selector, ?emptyAsNull: bool, ?keepNulls: bool) : DataFrame
member Explode(columns: seq<string>, ?emptyAsNull: bool, ?keepNulls: bool) : DataFrame
member Explode(column: string, ?emptyAsNull: bool, ?keepNulls: bool) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
selector | Selector | - |
emptyAsNull | bool (optional) | - |
keepNulls | bool (optional) | - |
columns | seq<string> | - |
column | string | - |
ExportBatches
Export the DataFrame as a stream of Arrow RecordBatches (Zero-Copy). Calls 'onBatch' for each chunk in the DataFrame. Useful for custom eager sinks (e.g. WriteDatabase).
member ExportBatches(onBatch: Action<RecordBatch>) : unitParameters
| Name | Type | Description |
|---|---|---|
onBatch | Action<RecordBatch> | - |
Extend
Extend another DataFrame to this one. Checks that the schema matches. NChunks wont't change.
member Extend(other: DataFrame) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | - |
FillNan
Fill NaN values in all floating point columns with a specified value.
member FillNan(fillValue: Expr) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
fillValue | Expr | - |
FillNull
Fill null values in all columns with a specified value.
member FillNull(fillValue: Expr) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
fillValue | Expr | - |
Filter
Filter rows based on a boolean expression (predicate).
Filter rows based on a boolean Series.
member Filter(expr: Expr) : DataFrame
member Filter(series: Series) : DataFrame
member Filter(mask: seq<bool>) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
expr | Expr | - |
series | Series | - |
mask | seq<bool> | - |
Fold
Fold over all columns (left to right) using the first column as the initial accumulator. Throws if the DataFrame is empty.
Fold over all columns with a custom starting accumulator.
member Fold(folder: (Series -> (Series -> Series))) : Series
member Fold(state: 'State, folder: ('State -> (Series -> 'State))) : 'StateParameters
| Name | Type | Description |
|---|---|---|
folder | (Series -> (Series -> Series)) / ('State -> (Series -> 'State)) | - |
state | 'State | - |
Gather
Selects rows from this DataFrame at the given indices.
member Gather(index: seq<int64>, ?nullOnOutOfBounds: bool) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
index | seq<int64> | The indices of the rows to select. |
nullOnOutOfBounds | bool (optional) | If true when an index is out-of-bounds a null row will be generated instead of raising an error. |
GatherEvery
Take every nth row in the Frame and return as a new Frame.
member GatherEvery(n: uint64, ?offset: uint64) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
n | uint64 | Gather every n-th row. |
offset | uint64 (optional) | Starting Index |
GetColumns
Returns all columns as a Series array.
Get all columns as an array of Series. Order is guaranteed to match the physical column order.
member GetColumns : unit -> Series[]GetSlice
Slice the DataFrame using F# slicing syntax (e.g., df.[start..finish]). Note: F# slicing bounds are inclusive at the end.
member GetSlice(start: int option, finish: int option) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
start | int option | - |
finish | int option | - |
Glimpse
Print a dense preview of the DataFrame to the standard output.
member Glimpse(?maxItemsPerColumn: int, ?maxColnameLength: int) : unitParameters
| Name | Type | Description |
|---|---|---|
maxItemsPerColumn | int (optional) | - |
maxColnameLength | int (optional) | - |
GlimpseFrame
Return a dense preview of the DataFrame as a new DataFrame. Schema: "column" (String), "dtype" (String), "values" (List[String])
member GlimpseFrame(?maxItemsPerColumn: int, ?maxColnameLength: int) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
maxItemsPerColumn | int (optional) | - |
maxColnameLength | int (optional) | - |
GlimpseString
Return a dense preview of the DataFrame as a formatted string.
member GlimpseString(?maxItemsPerColumn: int, ?maxColnameLength: int) : stringParameters
| Name | Type | Description |
|---|---|---|
maxItemsPerColumn | int (optional) | - |
maxColnameLength | int (optional) | - |
GroupBy
member GroupBy(keys: seq<Expr>, ?maintainOrder: bool) : GroupBy
member GroupBy(keys: seq<'c>, ?maintainOrder: bool) : GroupByParameters
| Name | Type | Description |
|---|---|---|
keys | seq<Expr> / seq<'c> | - |
maintainOrder | bool (optional) | - |
GroupByDynamic
Start a dynamic group-by operation. Returns a Builder.
member GroupByDynamic(
indexCol: string,
every: Dur,
?period: Dur,
?offset: Dur,
?by: seq<'b>,
?label: Label,
?includeBoundaries: bool,
?closedWindow: ClosedInterval,
?startBy: StartBy
) : GroupByParameters
| Name | Type | Description |
|---|---|---|
indexCol | string | - |
every | Dur | - |
period | Dur (optional) | - |
offset | Dur (optional) | - |
by | seq<'b> (optional) | - |
label | Label (optional) | - |
includeBoundaries | bool (optional) | - |
closedWindow | ClosedInterval (optional) | - |
startBy | StartBy (optional) | - |
GroupByRolling
Start a rolling group-by operation. Returns a Builder.
member GroupByRolling(
indexCol: string,
period: Dur,
?offset: Dur,
?by: seq<'a>,
?closedWindow: ClosedInterval
) : GroupByParameters
| Name | Type | Description |
|---|---|---|
indexCol | string | - |
period | Dur | - |
offset | Dur (optional) | - |
by | seq<'a> (optional) | - |
closedWindow | ClosedInterval (optional) | - |
HStack
Horizontally stack columns to the DataFrame. Returns a new DataFrame with the new columns appended.
member HStack(columns: seq<Series>) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | seq<Series> | - |
HashRows
Hash and combine the rows in this DataFrame.
member HashRows(?seed: uint64) : SeriesParameters
| Name | Type | Description |
|---|---|---|
seed | uint64 (optional) | - |
Head
Get the first n rows.
member Head(?rows: int) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
rows | int (optional) | - |
InsertColumn
Insert a column into the DataFrame at a specified index. Accepts a specific index and a Series to be inserted.
member InsertColumn(index: int, column: Series) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
index | int | The index at which to insert the column. Negative indices count from the end. |
column | Series | The Series to insert as a column. |
Interpolate
Interpolate intermediate values. The interpolation method is linear. Nulls at the beginning and end of the series remain null.
member Interpolate : unit -> DataFrameIsDuplicated
Get a mask of all duplicated rows in this DataFrame.
member IsDuplicated : unit -> SeriesIsFinite
member IsFinite(col: string) : SeriesParameters
| Name | Type | Description |
|---|---|---|
col | string | - |
IsInfinite
member IsInfinite(col: string) : SeriesParameters
| Name | Type | Description |
|---|---|---|
col | string | - |
IsNan
member IsNan(col: string) : SeriesParameters
| Name | Type | Description |
|---|---|---|
col | string | - |
IsNotNan
member IsNotNan(col: string) : SeriesParameters
| Name | Type | Description |
|---|---|---|
col | string | - |
IsNullAt
Check if the value at the specified column and row is null.
member IsNullAt(col: string, row: int, ?uncheck: bool) : boolParameters
| Name | Type | Description |
|---|---|---|
col | string | - |
row | int | - |
uncheck | bool (optional) | - |
IsSorted
Check whether the DataFrame is sorted by the given columns.
Single column overload for checking sort status.
member IsSorted(by: seq<string>, ?descending: seq<bool>, ?nullsLast: seq<bool>) : bool
member IsSorted(by: string, ?descending: bool, ?nullsLast: bool) : boolParameters
| Name | Type | Description |
|---|---|---|
by | seq<string> | Column name(s) to check. |
descending | seq<bool> (optional) / bool (optional) | Sort in descending order. When sorting by multiple columns, can be specified per column by passing a sequence of booleans. |
nullsLast | seq<bool> (optional) / bool (optional) | Place null values last. When sorting by multiple columns, can be specified per column by passing a sequence of booleans. |
IsUnique
Get a mask of all unique rows in this DataFrame.
member IsUnique : unit -> SeriesItem
[Indexer] Access cell value by Row Index and Column Index. Syntax: df.[rowIndex, colIndex]
[Indexer] Access cell value by Row Index and Column Name. Syntax: df.[rowIndex, "colName"]
member Item(columnName: string) : Series
member Item(columnIndex: int) : Series
member Item(rowIndex: int, columnName: string) : obj
member Item(rowIndex: int, columnIndex: int) : objParameters
| Name | Type | Description |
|---|---|---|
columnName | string | - |
columnIndex | int | - |
rowIndex | int | - |
Iter2
Applies an action pairwise to strongly-typed rows from two equal-height DataFrames.
member Iter2(other: DataFrame, action: ('T1 -> ('T2 -> unit))) : unitParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | - |
action | ('T1 -> ('T2 -> unit)) | - |
IterRows
Applies the given action to each row in the DataFrame materialized as a strongly-typed F# Record or DTO. Utilizes pre-compiled expression tree hydration and hoisted Series handles for extreme throughput.
member IterRows(action: ('T -> unit)) : unitParameters
| Name | Type | Description |
|---|---|---|
action | ('T -> unit) | The side-effecting action executed for each row. |
IterSlices
Returns an iterator over slices of the DataFrame.
member IterSlices(?nRows: int) : seq<DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
nRows | int (optional) | The number of rows per slice. Defaults to 10,000. |
Iteri2
Applies an action pairwise to strongly-typed rows from two DataFrames along with their 64-bit row index.
member Iteri2(other: DataFrame, action: (int64 -> ('T1 -> ('T2 -> unit)))) : unitParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | - |
action | (int64 -> ('T1 -> ('T2 -> unit))) | - |
IteriRows
Applies the given action to each row in the DataFrame along with its zero-based 64-bit row index.
member IteriRows(action: (int64 -> ('T -> unit))) : unitParameters
| Name | Type | Description |
|---|---|---|
action | (int64 -> ('T -> unit)) | The action executed for each row (int64 index -> 'T row -> unit). |
Join
Join with another DataFrame.
Join with another DataFrame with single on Expr.
member Join(
other: DataFrame,
leftOn: seq<Expr>,
rightOn: seq<Expr>,
how: JoinType,
?suffix: string,
?validation: JoinValidation,
?coalesce: JoinCoalesce,
?maintainOrder: JoinMaintainOrder,
?joinSide: JoinSide,
?nullsEqual: bool,
?sliceOffset: int64,
?sliceLen: uint64
) : DataFrame
member Join(
other: DataFrame,
on: seq<Expr>,
how: JoinType,
?suffix: string,
?validation: JoinValidation,
?coalesce: JoinCoalesce,
?maintainOrder: JoinMaintainOrder,
?joinSide: JoinSide,
?nullsEqual: bool,
?sliceOffset: int64,
?sliceLen: uint64
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | The right DataFrame to join with. |
leftOn | seq<Expr> | Column names in the left DataFrame to join on. |
rightOn | seq<Expr> | Column names in the right DataFrame to join on. |
how | JoinType | Type of join (Inner, Left, Outer, Cross, etc.). Default is Inner. |
suffix | string (optional) | Suffix to append to columns with same name in right DataFrame. Default "_right". |
validation | JoinValidation (optional) | Check if join keys are unique. |
coalesce | JoinCoalesce (optional) | How to coalesce the join keys. |
maintainOrder | JoinMaintainOrder (optional) | How to maintain the order of the join. |
joinSide | JoinSide (optional) | Specifies the strategy for the hash join build side. |
nullsEqual | bool (optional) | Consider nulls as equal. |
sliceOffset | int64 (optional) | Slice the result starting at this offset. |
sliceLen | uint64 (optional) | Length of the slice. |
on | seq<Expr> | - |
JoinAsOf
Perform an As-of join (also known as a time-series join).
This is similar to a left join except that we match on nearest key rather than equal keys. The join keys must be sorted.
member JoinAsOf(
other: DataFrame,
leftOn: Expr,
rightOn: Expr,
tolerance: Tolerance,
?byLeft: seq<Expr>,
?byRight: seq<Expr>,
?strategy: AsofStrategy,
?allowEq: bool,
?checkSorted: bool,
?suffix: string,
?validation: JoinValidation,
?coalesce: JoinCoalesce,
?maintainOrder: JoinMaintainOrder,
?joinSide: JoinSide,
?nullsEqual: bool,
?sliceOffset: int64,
?sliceLen: uint64
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | The right LazyFrame to join with. |
leftOn | Expr | Join key of the left LazyFrame. Must be sorted. |
rightOn | Expr | Join key of the right LazyFrame. Must be sorted. |
tolerance | Tolerance | Tolerance as a time duration string (e.g., "2h", "10s", "1d"), int or double or TimeSpan. Matches that are further away than this duration are discarded. |
byLeft | seq<Expr> (optional) | - |
byRight | seq<Expr> (optional) | - |
strategy | AsofStrategy (optional) | The strategy to determine which value is "nearest" (Backward, Forward, or Nearest). Defaults to AsofStrategy.Backward. |
allowEq | bool (optional) | If true, allow exact matches to be included in the result. If false, a match must be strictly unequal (e.g. less than for Backward strategy) to the key. |
checkSorted | bool (optional) | Check if the join keys are sorted. If false, the user must ensure keys are sorted; otherwise results are undefined (but execution is faster). |
suffix | string (optional) | Suffix to append to columns with name conflicts. Defaults to "_right". |
validation | JoinValidation (optional) | Check if join keys are unique (mostly relevant for the 'by' columns). |
coalesce | JoinCoalesce (optional) | How to coalesce the join keys. |
maintainOrder | JoinMaintainOrder (optional) | How to maintain the order of the join. |
joinSide | JoinSide (optional) | pecifies the strategy for the hash join build side. |
nullsEqual | bool (optional) | Consider nulls as equal. |
sliceOffset | int64 (optional) | Slice the result starting at this offset (optimization). |
sliceLen | uint64 (optional) | Length of the slice to keep. |
JoinWhere
Join with another LazyFrame using multiple boolean expressions.
member JoinWhere(
other: DataFrame,
predicates: seq<Expr>,
?how: JoinType,
?suffix: string,
?validation: JoinValidation,
?coalesce: JoinCoalesce,
?maintainOrder: JoinMaintainOrder,
?nullsEqual: bool
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | - |
predicates | seq<Expr> | - |
how | JoinType (optional) | - |
suffix | string (optional) | - |
validation | JoinValidation (optional) | - |
coalesce | JoinCoalesce (optional) | - |
maintainOrder | JoinMaintainOrder (optional) | - |
nullsEqual | bool (optional) | - |
Lazy
Convert dataframe to lazyframe
member Lazy : unit -> LazyFrameMap
Transforms each column in the DataFrame using a column-wise mapping function (Series -> Series). Preserves the original column name if the mapped Series does not provide a new one.
member Map(mapping: (Series -> Series)) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
mapping | (Series -> Series) | The transformation function applied to each column Series. |
Returns: A new DataFrame composed of the transformed Series.
MapRows
Transforms DataFrame rows into a new DataFrame using a strongly-typed F# Record mapping function ('TIn -> 'TOut). Combines compiled zero-allocation row enumerator with zero-reflection columnar transposition.
member MapRows(mapping: ('TIn -> 'TOut)) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
mapping | ('TIn -> 'TOut) | The pure transformation function applied to each row. |
Returns: A new DataFrame materialized from the mapped output records.
Max
Aggregate the columns in the Frame to their maximum value.
member Max : unit -> DataFrameMean
Aggregate the columns in the Frame to their mean value.
member Mean : unit -> DataFrameMedian
Aggregate the columns in the Frame to their median value.
member Median : unit -> DataFrameMelt
Alias for Unpivot.
Melt (Unpivot) overload for simple string lists. Auto-converts string seqs to Column Selectors.
member Melt(
index: Selector,
on: Selector,
?variableName: string,
?valueName: string
) : DataFrame
member Melt(
index: seq<string>,
on: seq<string>,
?variableName: string,
?valueName: string
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector / seq<string> | - |
on | Selector / seq<string> | - |
variableName | string (optional) | - |
valueName | string (optional) | - |
Merge
Initiates a Merge (upsert) plan using this DataFrame as the target and the given source. Internally both frames are converted to LazyFrame, so the resulting MergePlan can still be executed eagerly by piping into Merge.executeEager.
member Merge(source: DataFrame, on: IColumnExpr) : MergePlanParameters
| Name | Type | Description |
|---|---|---|
source | DataFrame | The source DataFrame containing updates/inserts. |
on | IColumnExpr | A column expression or selector that resolves to the merge key columns. |
Returns: A MergePlan backed by LazyFrames, ready for further configuration and execution.
MergeCatalogRecords
Starts building a Merge (Upsert) operation using this DataFrame as the source.
member MergeCatalogRecords(
catalog: UnityCatalog,
catalogName: string,
schemaName: string,
tableName: string,
mergeKeys: string[],
?canEvolve: bool,
?cloudOptions: CloudOptions
) : DeltaMergeBuilderParameters
| Name | Type | Description |
|---|---|---|
catalog | UnityCatalog | - |
catalogName | string | - |
schemaName | string | - |
tableName | string | - |
mergeKeys | string[] | - |
canEvolve | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
MergeDeltaOrdered
Starts a fluent builder to merge a DataFrame into a Delta Lake table with strict, order-preserving SQL MERGE semantics.
Unlike traditional merge methods, this builder guarantees that chained actions (Update, Delete, Insert) are evaluated exactly in the order they are defined. If no actions are specified before execution, it intelligently defaults to a standard Upsert (WhenMatchedUpdate + WhenNotMatchedInsert).
member MergeDeltaOrdered(
path: string,
mergeKeys: seq<string>,
?canEvolve: bool,
?cloudOptions: CloudOptions
) : DeltaMergeBuilderParameters
| Name | Type | Description |
|---|---|---|
path | string | The URI to the target Delta Lake table (local or cloud). |
mergeKeys | seq<string> | The column names to join on (must exist in both the Source DataFrame and Target Delta table). |
canEvolve | bool (optional) | If set to true, allows schema evolution (e.g., adding new columns from the Source to the Target). Default is false. |
cloudOptions | CloudOptions (optional) | Cloud storage credentials and configuration (e.g., AWS S3, Azure Blob). |
Returns: A DeltaMergeBuilder instance used to chain match conditions, culminating in a call to .Execute().
MergeSorted
Take two sorted DataFrames and merge them by the sorted key.
member MergeSorted(other: DataFrame, key: string, ?maintainOrder: bool) : DataFrame
member MergeSorted(other: DataFrame, key: string[], ?maintainOrder: bool) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | Other DataFrame that must be merged |
key | string / string[] | Key that is sorted. |
maintainOrder | bool (optional) | If True, the output is guaranteed to have left-biased ordering for equal keys: rows from the left frame appear before rows from the right frame when their keys are equal. |
Min
Aggregate the columns in the Frame to their minimum value.
member Min : unit -> DataFrameNChunks
Get the number of chunks used by the first Column of this DataFrame.
member NChunks : unit -> int64NChunksAll
Get an array containing the number of chunks for all columns in this DataFrame.
member NChunksAll : unit -> int64[]NUnique
Return the number of unique rows, or the number of unique row-subsets.
member NUnique(?subset: seq<string>) : int64
member NUnique([<ParamArray>] subset: Expr[]) : int64Parameters
| Name | Type | Description |
|---|---|---|
subset | seq<string> (optional) / Expr[] | - |
NullCount
Aggregate the columns in the Frame as the sum of their null value count.
Get the number of null values in a specific column.
member NullCount : unit -> DataFrame
member NullCount(colName: string) : int64Parameters
| Name | Type | Description |
|---|---|---|
colName | string | - |
PartitionBy
Group by the given columns and return the groups as separate dataframes.
Group by a single column and return the groups as separate dataframes.
member PartitionBy(byCols: seq<string>, ?maintainOrder: bool, ?includeKey: bool) : DataFrame[]
member PartitionBy(byCol: string, ?maintainOrder: bool, ?includeKey: bool) : DataFrame[]Parameters
| Name | Type | Description |
|---|---|---|
byCols | seq<string> | - |
maintainOrder | bool (optional) | - |
includeKey | bool (optional) | - |
byCol | string | - |
PartitionMap
Partitions the DataFrame by a single column and associates each partition with its strongly-typed key in an immutable F# Map.
Partitions the DataFrame by two columns and associates each partition with a strongly-typed tuple key in an immutable F# Map.
Partitions the DataFrame by the specified columns and hydrates each partition's key into an F# Record or composite key.
Partitions the DataFrame using an F# Record key, automatically inferring partition column names from 'Key record fields.
Partitions the DataFrame by arbitrary multiple columns and associates each partition with an untyped PartitionKey in an immutable F# Map.
member PartitionMap(byCol: string, ?maintainOrder: bool) : Map<'Key, DataFrame>
member PartitionMap(col1: string, col2: string, ?maintainOrder: bool) : Map<('K1 * 'K2), DataFrame>
member PartitionMap(byCols: seq<string>, ?maintainOrder: bool) : Map<'Key, DataFrame>
member PartitionMap(?maintainOrder: bool) : Map<'Key, DataFrame>
member PartitionMap(byCols: seq<string>, ?maintainOrder: bool) : Map<PartitionKey, DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
byCol | string | - |
maintainOrder | bool (optional) | Ensure consistent group ordering. Default is true. |
col1 | string | - |
col2 | string | - |
byCols | seq<string> | Column names to partition by. |
Pipe
Offers a structured way to apply a sequence of user-defined functions (UDFs).
member Pipe(func: (DataFrame -> 'T)) : 'TParameters
| Name | Type | Description |
|---|---|---|
func | (DataFrame -> 'T) | A function that receives the current expression and returns a value of type T. Typically this function wraps several Polars API calls that operate on the given expression. |
Returns: The result of applying func to this DataFrame.
Pivot
Pivot the DataFrame from long to wide format.
Pivot the DataFrame using column names.
Pivot the DataFrame using column names and a custom aggregation expression.
member Pivot(
index: Selector,
columns: Selector,
values: Selector,
?aggregateExpr: Expr,
?aggregateFunction: PivotAgg,
?sortColumns: bool,
?maintainOrder: bool,
?separator: string,
?columnNaming: PivotColumnNaming
) : DataFrame
member Pivot(
index: seq<string>,
columns: seq<string>,
values: seq<string>,
?aggFn: PivotAgg,
?sortColumns: bool,
?maintainOrder: bool,
?separator: string,
?columnNaming: PivotColumnNaming
) : DataFrame
member Pivot(
index: seq<string>,
columns: seq<string>,
values: seq<string>,
aggExpr: Expr,
?sortColumns: bool,
?maintainOrder: bool,
?separator: string,
?columnNaming: PivotColumnNaming
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector / seq<string> | Selector for the index column(s) (the rows). |
columns | Selector / seq<string> | Selector for the column(s) to pivot (the new column headers). |
values | Selector / seq<string> | Selector for the value column(s) to populate the cells. |
aggregateExpr | Expr (optional) | Optional expression to aggregate the values. If null, uses aggregateFunction. |
aggregateFunction | PivotAgg (optional) | Aggregation function to use if aggregateExpr is null. Default is First. |
sortColumns | bool (optional) | Sort the pivoted columns. |
maintainOrder | bool (optional) | Maintain the order of the data. |
separator | string (optional) | Separator used to combine column names when multiple value columns are selected. |
columnNaming | PivotColumnNaming (optional) | Controls how column names are constructed. Default is Auto. |
aggFn | PivotAgg (optional) | - |
aggExpr | Expr | - |
PrintSchema
Print schema in a readable format.
member PrintSchema : unit -> unitQuantile
Aggregate the columns in the Frame to their quantile value.
member Quantile(quantile: float, ?method: QuantileMethod) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
quantile | float | Quantile between 0.0 and 1.0. |
method | QuantileMethod (optional) | ['nearest’, ‘higher’, ‘lower’, ‘midpoint’, ‘linear’] Interpolation method. |
Rechunk
Rechunk the data in this DataFrame to a contiguous allocation. This will make sure all subsequent operations have optimal and predictable performance.
member Rechunk : unit -> DataFrameRemove
member Remove(predicate: Expr) : LazyFrame
member Remove(predicate: Series) : LazyFrame
member Remove(mask: seq<bool>) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
predicate | Expr / Series | - |
mask | seq<bool> | - |
Rename
Rename a column. Returns a new DataFrame.
Rename a list of columns.
Rename columns using a sequence of (oldName, newName) tuples. Example: df.Rename [ "colA", "col1"; "colB", "col2" ]
Rename columns using a dictionary mapping old names to new names. Example: df.Rename (dict [ "colA", "col1"; "colB", "col2" ])
member Rename(oldName: string, newName: string) : DataFrame
member Rename(oldNames: seq<string>, newNames: seq<string>) : DataFrame
member Rename(mapping: seq<(string * string)>) : DataFrame
member Rename(mapping: IReadOnlyDictionary<string, string>) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
oldName | string | - |
newName | string | - |
oldNames | seq<string> | - |
newNames | seq<string> | - |
mapping | seq<(string * string)> / IReadOnlyDictionary<string, string> | - |
ReplaceColumn
Replace a column by its index in-place. Supports negative indexing (e.g., -1 replaces the last column).
Replace a column by its name in-place.
member ReplaceColumn(index: int, newColumn: Series, ?keepName: bool) : DataFrame
member ReplaceColumn(columnName: string, newColumn: Series, ?keepName: bool) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
index | int | The index of the column to replace. Negative values count from the end. |
newColumn | Series | The new Series to insert. |
keepName | bool (optional) | If true, keeps the original column name. If false, uses the new Series's name. Default is false. |
columnName | string | The name of the column to replace. |
Returns: The current DataFrame instance to support method chaining.
Reverse
Reverse the DataFrame.
member Reverse : unit -> DataFrameRow
Get data for a specific row as an object array. Similar to DataTable.Rows[i].ItemArray.
member Row(index: int) : obj[]Parameters
| Name | Type | Description |
|---|---|---|
index | int | - |
Rows
Returns a zero-allocation stack enumerator for strongly-typed row hydration. Supports F# Records and parameterless DTOs. Usage: for row in df.Rows{MyRecord}() do ...
member Rows : unit -> RowEnumerator<'T>RowsAsync
Streams in-memory DataFrame rows asynchronously as strongly-typed F# Records/Tuples/DTOs. Serves as the asynchronous counterpart to DataFrame.Rows<'T>().
member RowsAsync(?chunkSize: int, ?cancellationToken: CancellationToken) : IAsyncEnumerable<'T>Parameters
| Name | Type | Description |
|---|---|---|
chunkSize | int (optional) | - |
cancellationToken | CancellationToken (optional) | - |
Sample
Sample n rows from the DataFrame.
Sample a fraction of rows from the DataFrame.
member Sample(
n: int,
?withReplacement: bool,
?shuffle: bool,
?seed: uint64
) : DataFrame
member Sample(
frac: float,
?withReplacement: bool,
?shuffle: bool,
?seed: uint64
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
n | int | - |
withReplacement | bool (optional) | - |
shuffle | bool (optional) | - |
seed | uint64 (optional) | - |
frac | float | - |
Select
Select columns using expressions.
Select a single column using an expression. Usage: df.Select(pl.col("A"))
Select columns using generic column expressions (Expr or Selectors).
member Select(exprs: seq<Expr>) : DataFrame
member Select(selector: Selector) : DataFrame
member Select(columns: seq<'b>) : DataFrame
member Select(expr: Expr) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
exprs | seq<Expr> | - |
selector | Selector | - |
columns | seq<'b> | - |
expr | Expr | - |
SetSorted
member SetSorted(columns: Selector, ?descending: bool, ?nullsLast: bool) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | Selector | - |
descending | bool (optional) | - |
nullsLast | bool (optional) | - |
Show
Print the DataFrame to Console (Stdout).
member Show : unit -> unitShrinkToFit
Shrink DataFrame memory usage.
member ShrinkToFit : unit -> DataFrameReturns: A new DataFrame
ShrinkToFitInplace
Shrink DataFrame memory usage.This won't return a new DataFrame
member ShrinkToFitInplace : unit -> unitSlice
Slice the DataFrame along the rows. Negative offsets count from the end of the DataFrame. If length is omitted, it slices to the end of the DataFrame.
member Slice(offset: int64, ?length: uint64) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
offset | int64 | - |
length | uint64 (optional) | - |
Sort
Sort the DataFrame.
Sort by a single column name.
Sort with simple broadcasting options.
Sort by a single expression.
member Sort(
columns: seq<IColumnExpr>,
descending: seq<bool>,
nullsLast: seq<bool>,
?maintainOrder: bool
) : DataFrame
member Sort(
columns: seq<Expr>,
descending: seq<bool>,
nullsLast: seq<bool>,
?maintainOrder: bool
) : DataFrame
member Sort(
columns: seq<IColumnExpr>,
?descending: bool,
?nullsLast: bool,
?maintainOrder: bool
) : DataFrame
member Sort(
columns: seq<Expr>,
?descending: bool,
?nullsLast: bool,
?maintainOrder: bool
) : DataFrame
member Sort(expr: Expr, ?descending: bool, ?nullsLast: bool) : DataFrame
member Sort(
colName: string,
?descending: bool,
?nullsLast: bool,
?maintainOrder: bool
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | seq<IColumnExpr> / seq<Expr> | the column which needs to be sorted (Expr/Selector)。 |
descending | seq<bool> / bool (optional) | sort direction (true=descending).Length must be 1 (broadcasting) or same with columns. |
nullsLast | seq<bool> / bool (optional) | null value position (true=last).Length must be 1 (broadcasting) or same with columns |
maintainOrder | bool (optional) | Stable Sort option |
expr | Expr | - |
colName | string | - |
Std
Aggregate the columns in the Frame to their standard deviation value.
member Std(?ddof: byte) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
ddof | byte (optional) | “Delta Degrees of Freedom”: the divisor used in the calculation is N - ddof, where N represents the number of elements. By default ddof is 1. |
Sum
Aggregate the columns in the Frame to their sum value.
member Sum : unit -> DataFrameTail
Get the last n rows.
member Tail(?n: int) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
n | int (optional) | - |
Take
Take rows by physical integer indices. Note: Negative indices are not supported. All values must be >= 0.
member Take(indices: Series) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
indices | Series | A Series containing integer indices (e.g., UInt32, Int32). |
ToArrow
member ToArrow : unit -> RecordBatchToArrowStream
Export DataFrame to Arrow C Data Interface Stream. Supports zero-copy and lazy chunked reading.
Export DataFrame to Arrow C Data Interface Stream with Column Pruning.
Export DataFrame to Arrow C Data Interface Stream with Column Pruning (Array friendly).
member ToArrowStream(?seed: uint64) : IArrowArrayStream
member ToArrowStream(columnIndices: ReadOnlySpan<int>, ?seed: uint64) : IArrowArrayStream
member ToArrowStream(columnIndices: int[], ?seed: uint64) : IArrowArrayStreamParameters
| Name | Type | Description |
|---|---|---|
seed | uint64 (optional) | Optional seed for Native Global Shuffle. |
columnIndices | ReadOnlySpan<int> / int[] | Column indices to prune the export (Projection Pushdown). |
Returns: Standard IArrowArrayStream
ToAsyncEnumerable
Streams in-memory DataFrame rows asynchronously as strongly-typed F# Records/Tuples/DTOs. Uses cooperative task slicing to prevent thread pool starvation during large scans.
member ToAsyncEnumerable(?chunkSize: int, ?cancellationToken: CancellationToken) : IAsyncEnumerable<'T>Parameters
| Name | Type | Description |
|---|---|---|
chunkSize | int (optional) | - |
cancellationToken | CancellationToken (optional) | - |
ToDummies
Convert categorical columns to dummy/one-hot encoded variables.
member ToDummies(
?columns: seq<string>,
?separator: string,
?dropFirst: bool,
?dropNulls: bool
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | seq<string> (optional) | Optional. Specific column names to convert. If omitted, all string and categorical columns are converted. |
separator | string (optional) | The separator between the original column name and the dummy category. Default is "_". |
dropFirst | bool (optional) | If true, drops the first dummy column to avoid multicollinearity. Default is false. |
dropNulls | bool (optional) | If true, drops dummy columns representing null values. Default is false. |
Returns: A new DataFrame with dummy variables.
ToHtml
Generates a rich HTML table representation of the DataFrame for interactive environments (e.g., LINQPad, Jupyter).
member ToHtml : unit -> stringReturns: An HTML string representing the preview of the DataFrame.
ToMap
Convert the DataFrame to a dictionary of column name to Series.
member ToMap : unit -> Map<string, Series>ToRecordList
Materializes all DataFrame rows into an F# list of records.
member ToRecordList : unit -> 'T listToRecords
Converts the DataFrame rows to a sequence of F# Record instances. Throws ArgumentException if 'T is not an F# Record type.
member ToRecords : unit -> seq<'T>ToStruct
Convert a DataFrame to a Series of type Struct.
member ToStruct(?name: string) : SeriesParameters
| Name | Type | Description |
|---|---|---|
name | string (optional) | Name for the struct Series. |
TopK
Get the top k rows based on the given columns. This is often faster than a full sort followed by a head.
member TopK(k: int, by: seq<'a>, ?reverse: seq<bool>) : DataFrame
member TopK(k: int, by: seq<'a>, reverse: bool) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
k | int | Number of rows to return. |
by | seq<'a> | Columns to sort by. |
reverse | seq<bool> (optional) | Sort direction per column. Default is false (no reverse). |
Transpose
Transpose a DataFrame over the diagonal.
Transpose a DataFrame over the diagonal, using an existing column for column names.
Transpose a DataFrame over the diagonal, using custom names for transposed columns.
member Transpose(?includeHeader: bool, ?headerName: string) : DataFrame
member Transpose(columnName: string, ?includeHeader: bool, ?headerName: string) : DataFrame
member Transpose(customNames: seq<string>, ?includeHeader: bool, ?headerName: string) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
includeHeader | bool (optional) | If true, the column names will be added as the first column. |
headerName | string (optional) | Name of the header column if includeHeader is true (default "column"). |
columnName | string | Existing column name to use for naming transposed columns. |
customNames | seq<string> | Sequence of names for the value columns in the transposed data. |
Unique
Return the number of unique rows, or the number of unique row-subsets.
Return unique rows from the DataFrame with a condition.
member Unique(
?subset: seq<string>,
?keep: UniqueKeepStrategy,
?maintainOrder: bool,
?offset: int64,
?len: int64
) : DataFrame
member Unique(
subset: seq<Expr>,
?keep: UniqueKeepStrategy,
?maintainOrder: bool,
?offset: int64,
?len: int64
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
subset | seq<string> (optional) / seq<Expr> | - |
keep | UniqueKeepStrategy (optional) | - |
maintainOrder | bool (optional) | - |
offset | int64 (optional) | - |
len | int64 (optional) | - |
UnnestColumn
Decompose a struct column into multiple columns.
member UnnestColumn(column: string, ?separator: string) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
column | string | - |
separator | string (optional) | - |
UnnestColumns
Decompose multiple struct columns.
member UnnestColumns(columns: seq<string>, ?separator: string) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | seq<string> | - |
separator | string (optional) | - |
Unpivot
Unpivot (Melt) the DataFrame from wide to long format using Selectors. This is the primary implementation backed by native binding.
Unpivot (Melt) overload for simple string lists. Auto-converts string lists to Column Selectors.
member Unpivot(
index: Selector,
on: Selector,
?variableName: string,
?valueName: string
) : DataFrame
member Unpivot(
index: seq<string>,
on: seq<string>,
?variableName: string,
?valueName: string
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector / seq<string> | Selector for ID variables (columns to keep) |
on | Selector / seq<string> | Selector for Value variables (columns to melt) |
variableName | string (optional) | Name for the variable column (default: "variable") |
valueName | string (optional) | Name for the value column (default: "value") |
Unstack
Unstack a long table to a wide form without doing an aggregation. This can be much faster than a pivot, because it can skip the grouping phase.
member Unstack(
step: int64,
?columns: Selector,
?how: UnstackDirection,
?fillValues: seq<Expr>
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
step | int64 | Number of rows in the unstacked frame. |
columns | Selector (optional) | Column name(s) or selector(s) to include in the operation. If set to None (default), use all columns. |
how | UnstackDirection (optional) | Direction of the unstack. |
fillValues | seq<Expr> (optional) | Fill values that don’t fit the new size with this value. |
UnstackInternal
member UnstackInternal(
step: int64,
how: UnstackDirection,
resolvedColumns: string[],
?fillValues: seq<Expr>
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
step | int64 | - |
how | UnstackDirection | - |
resolvedColumns | string[] | - |
fillValues | seq<Expr> (optional) | - |
Update
member Update(
other: DataFrame,
?on: IColumnExpr,
?how: JoinType,
?leftOn: IColumnExpr,
?rightOn: IColumnExpr,
?includeNulls: bool,
?maintainOrder: JoinMaintainOrder
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | - |
on | IColumnExpr (optional) | - |
how | JoinType (optional) | - |
leftOn | IColumnExpr (optional) | - |
rightOn | IColumnExpr (optional) | - |
includeNulls | bool (optional) | - |
maintainOrder | JoinMaintainOrder (optional) | - |
Upsample
Upsample a DataFrame at a regular frequency.
member Upsample(
timeColumn: Selector,
every: Dur,
?groupBy: Selector,
?maintainOrder: bool
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
timeColumn | Selector | The column used for the time/datetime index. Must resolve to exactly one column. |
every | Dur | The interval to upsample to (e.g., Duration.String "1d", or Duration.TimeSpan ts). |
groupBy | Selector (optional) | Optional. Group by this selector before upsampling. |
maintainOrder | bool (optional) | If true, maintains the original order of the groups. Default is false. |
Returns: A new DataFrame with missing time steps filled with nulls.
VStack
Vertically stack another DataFrame to this one. Checks that the schema matches.
member VStack(other: DataFrame) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | - |
Var
Aggregate the columns in the Frame to their variance value.
member Var(?ddof: byte) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
ddof | byte (optional) | “Delta Degrees of Freedom”: the divisor used in the calculation is N - ddof, where N represents the number of elements. By default ddof is 1. |
WithColumns
Add or replace columns using expressions.
Add or replace columns using generic column expressions (Expr or Selectors).
member WithColumns(exprs: seq<Expr>) : DataFrame
member WithColumns(columns: seq<'a>) : DataFrame
member WithColumns(expr: Expr) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
exprs | seq<Expr> | - |
columns | seq<'a> | - |
expr | Expr | - |
WithRowIndex
Add a column at index 0 that counts the rows.
This can be useful to generate a unique index or to maintain order in operations that would otherwise discard it.
member WithRowIndex(?name: string, ?offset: int) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
name | string (optional) | The name of the new row index column. Defaults to "index". |
offset | int (optional) | The starting value of the row index. Defaults to 0. |
Returns: A new DataFrame with the row index column added.
WriteAvro
Write the DataFrame to an Apache Avro file.
member WriteAvro(path: string, ?compression: AvroCompression, ?name: string) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | The file path to write to. |
compression | AvroCompression (optional) | The compression algorithm to use. |
name | string (optional) | The name of the Avro record. |
WriteAvroMemory
Write the DataFrame to an Apache Avro memory buffer.
member WriteAvroMemory(?compression: AvroCompression, ?name: string) : byte[]Parameters
| Name | Type | Description |
|---|---|---|
compression | AvroCompression (optional) | The compression algorithm to use. |
name | string (optional) | The name of the Avro record. |
Returns: A byte array containing the Avro data.
WriteCatalogTable
Writes the DataFrame into a Unity Catalog table by converting it to a LazyFrame.
member WriteCatalogTable(
catalog: UnityCatalog,
catalogName: string,
schemaName: string,
tableName: string,
?partitionBy: Selector,
?mode: DeltaSaveMode,
?canEvolve: bool,
?includeKeys: bool,
?keysPreGrouped: bool,
?maxRowsPerFile: int,
?approxBytesPerFile: int64,
?compression: ParquetCompression,
?compressionLevel: int,
?statistics: bool,
?rowGroupSize: uint32,
?dataPageSize: uint32,
?compatLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
catalog | UnityCatalog | - |
catalogName | string | - |
schemaName | string | - |
tableName | string | - |
partitionBy | Selector (optional) | - |
mode | DeltaSaveMode (optional) | - |
canEvolve | bool (optional) | - |
includeKeys | bool (optional) | - |
keysPreGrouped | bool (optional) | - |
maxRowsPerFile | int (optional) | - |
approxBytesPerFile | int64 (optional) | - |
compression | ParquetCompression (optional) | - |
compressionLevel | int (optional) | - |
statistics | bool (optional) | - |
rowGroupSize | uint32 (optional) | - |
dataPageSize | uint32 (optional) | - |
compatLevel | int (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
WriteCsv
Write DataFrame to a comma-separated values (CSV) file.
This uses the Lazy execution engine internally to support streaming and cloud storage.
member WriteCsv(
path: string,
?includeHeader: bool,
?includeBom: bool,
?separator: char,
?quoteChar: char,
?quoteStyle: QuoteStyle,
?nullValue: string,
?lineTerminator: string,
?floatScientific: bool,
?floatPrecision: int,
?decimalComma: bool,
?dateFormat: string,
?timeFormat: string,
?datetimeFormat: string,
?checkExtension: bool,
?compression: ExternalCompression,
?compressionLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?batchSize: int,
?cloudOptions: CloudOptions
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
includeHeader | bool (optional) | - |
includeBom | bool (optional) | - |
separator | char (optional) | - |
quoteChar | char (optional) | - |
quoteStyle | QuoteStyle (optional) | - |
nullValue | string (optional) | - |
lineTerminator | string (optional) | - |
floatScientific | bool (optional) | - |
floatPrecision | int (optional) | - |
decimalComma | bool (optional) | - |
dateFormat | string (optional) | - |
timeFormat | string (optional) | - |
datetimeFormat | string (optional) | - |
checkExtension | bool (optional) | - |
compression | ExternalCompression (optional) | - |
compressionLevel | int (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
batchSize | int (optional) | - |
cloudOptions | CloudOptions (optional) | - |
WriteCsvMemory
Write DataFrame to a CSV format in memory.
member WriteCsvMemory(
?includeBom: bool,
?includeHeader: bool,
?batchSize: int,
?checkExtension: bool,
?compressionCode: ExternalCompression,
?compressionLevel: int,
?dateFormat: string,
?timeFormat: string,
?datetimeFormat: string,
?floatScientific: int,
?floatPrecision: int,
?decimalComma: bool,
?separator: byte,
?quoteChar: byte,
?nullValue: string,
?lineTerminator: string,
?quoteStyle: QuoteStyle,
?maintainOrder: bool
) : byte[]Parameters
| Name | Type | Description |
|---|---|---|
includeBom | bool (optional) | - |
includeHeader | bool (optional) | - |
batchSize | int (optional) | - |
checkExtension | bool (optional) | - |
compressionCode | ExternalCompression (optional) | - |
compressionLevel | int (optional) | - |
dateFormat | string (optional) | - |
timeFormat | string (optional) | - |
datetimeFormat | string (optional) | - |
floatScientific | int (optional) | - |
floatPrecision | int (optional) | - |
decimalComma | bool (optional) | - |
separator | byte (optional) | - |
quoteChar | byte (optional) | - |
nullValue | string (optional) | - |
lineTerminator | string (optional) | - |
quoteStyle | QuoteStyle (optional) | - |
maintainOrder | bool (optional) | - |
Returns: A byte array containing the serialized CSV data.
WriteCsvPartitioned
Write DataFrame to a partitioned comma-separated values (CSV) file.
This uses the Lazy execution engine internally to support streaming and cloud storage.
member WriteCsvPartitioned(
path: string,
partitionBy: Selector,
?includeKeys: bool,
?keysPreGrouped: bool,
?maxRowsPerFile: int,
?approxBytesPerFile: int64,
?includeHeader: bool,
?includeBom: bool,
?separator: char,
?quoteChar: char,
?quoteStyle: QuoteStyle,
?nullValue: string,
?lineTerminator: string,
?floatScientific: bool,
?floatPrecision: int,
?decimalComma: bool,
?dateFormat: string,
?timeFormat: string,
?datetimeFormat: string,
?checkExtension: bool,
?compression: ExternalCompression,
?compressionLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?batchSize: int,
?cloudOptions: CloudOptions
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
partitionBy | Selector | - |
includeKeys | bool (optional) | - |
keysPreGrouped | bool (optional) | - |
maxRowsPerFile | int (optional) | - |
approxBytesPerFile | int64 (optional) | - |
includeHeader | bool (optional) | - |
includeBom | bool (optional) | - |
separator | char (optional) | - |
quoteChar | char (optional) | - |
quoteStyle | QuoteStyle (optional) | - |
nullValue | string (optional) | - |
lineTerminator | string (optional) | - |
floatScientific | bool (optional) | - |
floatPrecision | int (optional) | - |
decimalComma | bool (optional) | - |
dateFormat | string (optional) | - |
timeFormat | string (optional) | - |
datetimeFormat | string (optional) | - |
checkExtension | bool (optional) | - |
compression | ExternalCompression (optional) | - |
compressionLevel | int (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
batchSize | int (optional) | - |
cloudOptions | CloudOptions (optional) | - |
WriteDelta
Write the DataFrame to a Delta Lake table with partition discovery.
This operation performs a "blind write" of partitioned Parquet files (Hive-style) and then commits a transaction to the Delta Log, registering the new files.
member WriteDelta(
path: string,
?partitionBy: Selector,
?mode: DeltaSaveMode,
?canEvolve: bool,
?includeKeys: bool,
?keysPreGrouped: bool,
?maxRowsPerFile: int,
?approxBytesPerFile: int64,
?compression: ParquetCompression,
?compressionLevel: int,
?statistics: bool,
?rowGroupSize: uint32,
?dataPageSize: uint32,
?compatLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | Path to the root of the Delta Table. Can be local (e.g. "./data/table") or remote (e.g. "s3://bucket/table"). |
partitionBy | Selector (optional) | The selector(s) to partition the data by. Directories will be created in the format "col=value". |
mode | DeltaSaveMode (optional) | Save mode (Append, Overwrite, ErrorIfExists, Ignore). Default is Append. |
canEvolve | bool (optional) | - |
includeKeys | bool (optional) | Whether to include the partition keys in the Parquet files themselves. Default is true (recommended for Delta Lake compatibility). |
keysPreGrouped | bool (optional) | Assert that the keys are already pre-grouped. This can speed up the operation if true. |
maxRowsPerFile | int (optional) | Maximum number of rows per file. 0 means no limit. |
approxBytesPerFile | int64 (optional) | Approximate size in bytes per file. 0 means no limit. |
compression | ParquetCompression (optional) | Compression codec to use (Snappy, Zstd, etc.). |
compressionLevel | int (optional) | Compression level (depends on the codec). |
statistics | bool (optional) | Write statistics to the Parquet file. Delta Lake uses these stats for data skipping, so 'true' is highly recommended. |
rowGroupSize | uint32 (optional) | Target row group size (in rows). |
dataPageSize | uint32 (optional) | Target data page size (in bytes). |
compatLevel | int (optional) | IPC format compatibility. |
maintainOrder | bool (optional) | Maintain the order of the data. |
syncOnClose | SyncOnClose (optional) | Whether to sync the file to disk on close. |
mkdir | bool (optional) | Create parent directories if they don't exist. |
cloudOptions | CloudOptions (optional) | Options for cloud storage authentication and configuration. |
WriteExcel
Write the DataFrame to an Excel file (.xlsx) using the native high-performance engine.
UInt64, Int128, UInt128 are automatically written as Text to preserve precision.
member WriteExcel(
path: string,
?sheetName: string,
?dateFormat: string,
?datetimeFormat: string
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | Destination file path. |
sheetName | string (optional) | Optional sheet name (default: "Sheet1"). |
dateFormat | string (optional) | Optional Excel format string for Date columns (e.g. "yyyy-mm-dd"). |
datetimeFormat | string (optional) | Optional Excel format string for Datetime columns (e.g. "yyyy-mm-dd hh:mm:ss"). |
WriteIpc
Write DataFrame to an IPC (Arrow/Feather) file.
This uses the Lazy execution engine internally to support streaming and cloud storage.
member WriteIpc(
path: string,
?compression: IpcCompression,
?compatLevel: int,
?recordBatchSize: int,
?recordBatchStatistics: bool,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
compression | IpcCompression (optional) | - |
compatLevel | int (optional) | - |
recordBatchSize | int (optional) | - |
recordBatchStatistics | bool (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
WriteIpcMemory
Write DataFrame to an IPC (Arrow/Feather) format in memory.
member WriteIpcMemory(
?compression: IpcCompression,
?compatLevel: int,
?recordBatchSize: int,
?recordBatchStatistics: bool,
?maintainOrder: bool
) : byte[]Parameters
| Name | Type | Description |
|---|---|---|
compression | IpcCompression (optional) | - |
compatLevel | int (optional) | - |
recordBatchSize | int (optional) | - |
recordBatchStatistics | bool (optional) | - |
maintainOrder | bool (optional) | - |
Returns: A byte array containing the serialized IPC data.
WriteIpcPartitioned
Write DataFrame to a partitioned IPC (Arrow/Feather) file.
This uses the Lazy execution engine internally to support streaming and cloud storage.
member WriteIpcPartitioned(
path: string,
partitionBy: Selector,
?includeKeys: bool,
?keysPreGrouped: bool,
?maxRowsPerFile: int,
?approxBytesPerFile: int64,
?compression: IpcCompression,
?compatLevel: int,
?recordBatchSize: int,
?recordBatchStatistics: bool,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
partitionBy | Selector | - |
includeKeys | bool (optional) | - |
keysPreGrouped | bool (optional) | - |
maxRowsPerFile | int (optional) | - |
approxBytesPerFile | int64 (optional) | - |
compression | IpcCompression (optional) | - |
compatLevel | int (optional) | - |
recordBatchSize | int (optional) | - |
recordBatchStatistics | bool (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
WriteIpcStream
Write to Arrow IPC record batch stream.
member WriteIpcStream(path: string, compression: IpcCompression option, compatLevel: int option) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | Path to the IPC stream file or a byte buffer |
compression | IpcCompression option | Compression method. Defaults to None. |
compatLevel | int option | Use a specific compatibility level when exporting Polars’ internal data structures. |
WriteIpcStreamMemory
Write to Arrow IPC record batch stream.
member WriteIpcStreamMemory(compression: IpcCompression option, compatLevel: int option) : byte[]Parameters
| Name | Type | Description |
|---|---|---|
compression | IpcCompression option | Compression method. Defaults to None. |
compatLevel | int option | Use a specific compatibility level when exporting Polars’ internal data structures. |
WriteJson
Write DataFrame to a JSON file.
member WriteJson(path: string, ?format: JsonFormat) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
format | JsonFormat (optional) | - |
WriteNdJson
Write DataFrame to a NDJSON (JsonLines) file.
member WriteNdJson(path: string) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
WriteParquet
Write DataFrame to a Parquet file.
This uses the Lazy execution engine internally to support streaming, statistics, and cloud storage.
member WriteParquet(
path: string,
?compression: ParquetCompression,
?compressionLevel: int,
?statistics: bool,
?rowGroupSize: int,
?dataPageSize: int,
?compatLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
compression | ParquetCompression (optional) | - |
compressionLevel | int (optional) | - |
statistics | bool (optional) | - |
rowGroupSize | int (optional) | - |
dataPageSize | int (optional) | - |
compatLevel | int (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
WriteParquetMemory
Write DataFrame to a Parquet format in memory.
member WriteParquetMemory(
?compression: ParquetCompression,
?compressionLevel: int,
?statistics: bool,
?rowGroupSize: int,
?dataPageSize: int,
?compatLevel: int,
?maintainOrder: bool
) : byte[]Parameters
| Name | Type | Description |
|---|---|---|
compression | ParquetCompression (optional) | - |
compressionLevel | int (optional) | - |
statistics | bool (optional) | - |
rowGroupSize | int (optional) | - |
dataPageSize | int (optional) | - |
compatLevel | int (optional) | - |
maintainOrder | bool (optional) | - |
Returns: A byte array containing the serialized Parquet data.
WriteParquetPartitioned
Write DataFrame to a partitioned Parquet file.
This uses the Lazy execution engine internally to support streaming and cloud storage.
member WriteParquetPartitioned(
path: string,
partitionBy: Selector,
?includeKeys: bool,
?keysPreGrouped: bool,
?maxRowsPerFile: int,
?approxBytesPerFile: int64,
?compression: ParquetCompression,
?compressionLevel: int,
?statistics: bool,
?rowGroupSize: int,
?dataPageSize: int,
?compatLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
partitionBy | Selector | - |
includeKeys | bool (optional) | - |
keysPreGrouped | bool (optional) | - |
maxRowsPerFile | int (optional) | - |
approxBytesPerFile | int64 (optional) | - |
compression | ParquetCompression (optional) | - |
compressionLevel | int (optional) | - |
statistics | bool (optional) | - |
rowGroupSize | int (optional) | - |
dataPageSize | int (optional) | - |
compatLevel | int (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
WriteTo
Stream the DataFrame directly to a database or other IDataReader consumer.
Uses a producer-consumer pattern. This method blocks until the consumer finishes reading. Ideal for SqlBulkCopy.WriteToServer or NpgsqlBinaryImporter.
member WriteTo(writerAction: Action<IDataReader>, ?bufferSize: int, ?typeOverrides: IDictionary<string, Type>) : unitParameters
| Name | Type | Description |
|---|---|---|
writerAction | Action<IDataReader> | Callback that receives an IDataReader. |
bufferSize | int (optional) | Max number of batches to buffer in memory (default: 5). |
typeOverrides | IDictionary<string, Type> (optional) | Dictionary to force specific C# types for columns (optional). |
WriteToAdbc
Zero-copy bulk ingest of the current DataFrame into an ADBC database (e.g., DuckDB, SQLite).
Zero-copy bulk ingest of the current DataFrame into an ADBC database table. Pure syntactic sugar: automatically manages the creation, configuration, and disposal of the underlying AdbcStatement.
member WriteToAdbc(statement: AdbcStatement) : UpdateResult
member WriteToAdbc(connection: AdbcConnection, tableName: string, ?ingestMode: AdbcIngestMode) : UpdateResultParameters
| Name | Type | Description |
|---|---|---|
statement | AdbcStatement | An AdbcStatement configured with ingest options (e.g., target table). |
connection | AdbcConnection | The active ADBC connection (e.g., DuckDB, SQLite). |
tableName | string | The name of the target table to ingest data into. |
ingestMode | AdbcIngestMode (optional) | The ingestion mode. Defaults to Create. |
Returns: The UpdateResult containing the number of rows affected.
Pipeline Module Functions (DataFrame.*)
DataFrame.agg
Aggregation clause for GroupBy.
DataFrame.agg : aggs: seq<Expr> -> builder: GroupBy -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
aggs | seq<Expr> | - |
builder | GroupBy | - |
DataFrame.asLazy
Convert the DataFrame to a LazyFrame.
DataFrame.asLazy : df: DataFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
DataFrame.collectSchema
Get the schema of the DataFrame.
DataFrame.collectSchema : df: DataFrame -> PolarsSchemaParameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
DataFrame.columnNames
Get the column names of the DataFrame.
DataFrame.columnNames : df: DataFrame -> string[]Parameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
DataFrame.concatDiagonal
Diagonally concat DataFrames
DataFrame.concatDiagonal : dfs: seq<DataFrame> -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
dfs | seq<DataFrame> | - |
DataFrame.concatHorizontal
Horizontally concat DataFrames.
DataFrame.concatHorizontal : dfs: seq<DataFrame> -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
dfs | seq<DataFrame> | - |
DataFrame.concatHorizontalNoCheck
Horizontally concat DataFrames (Allow duplicates).
DataFrame.concatHorizontalNoCheck : dfs: seq<DataFrame> -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
dfs | seq<DataFrame> | - |
DataFrame.concatVertical
Vertically concat DataFrames (Standard concat).
DataFrame.concatVertical : dfs: seq<DataFrame> -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
dfs | seq<DataFrame> | - |
DataFrame.drop
Drop columns from the DataFrame.
DataFrame.drop : columns: 'a -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | 'a | - |
df | DataFrame | - |
DataFrame.dropNans
Drop NaN values from the DataFrame.
DataFrame.dropNans : df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
DataFrame.dropNansOn
Drop NaN values from the DataFrame with a condition.
DataFrame.dropNansOn : subset: 'a -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
subset | 'a | - |
df | DataFrame | - |
DataFrame.dropNulls
Drop null values from the DataFrame.
DataFrame.dropNulls : df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
DataFrame.dropNullsOn
Drop null values from the DataFrame with specific subset.
DataFrame.dropNullsOn : subset: 'a -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
subset | 'a | - |
df | DataFrame | - |
DataFrame.explode
Explode list-like columns into multiple rows.
DataFrame.explode : columns: 'a -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | 'a | - |
df | DataFrame | - |
DataFrame.fillNan
Fill NaN values in all columns with a specified value.
DataFrame.fillNan : fillValue: Expr -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
fillValue | Expr | - |
df | DataFrame | - |
DataFrame.fillNull
Fill null values in all columns with a specified value.
DataFrame.fillNull : fillValue: Expr -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
fillValue | Expr | - |
df | DataFrame | - |
DataFrame.filter
Filter rows based on a boolean expression.
DataFrame.filter : expr: Expr -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
expr | Expr | - |
df | DataFrame | - |
DataFrame.filterAll
Filter frame rows by combining multiple predicates with logical AND.
DataFrame.filterAll : predicates: seq<Expr> -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
predicates | seq<Expr> | - |
df | DataFrame | - |
DataFrame.fold
Applies a folding function across all columns of the DataFrame from left to right, using the first column as the seed.
DataFrame.fold : folder: (Series -> (Series -> Series)) -> df: DataFrame -> SeriesParameters
| Name | Type | Description |
|---|---|---|
folder | (Series -> (Series -> Series)) | - |
df | DataFrame | - |
DataFrame.foldState
Applies a folding function across all columns of the DataFrame from left to right, threading an accumulator state.
DataFrame.foldState : folder: ('State -> (Series -> 'State)) -> state: 'State -> df: DataFrame -> 'StateParameters
| Name | Type | Description |
|---|---|---|
folder | ('State -> (Series -> 'State)) | - |
state | 'State | - |
df | DataFrame | - |
DataFrame.groupBy
Group by keys and apply aggregations.
DataFrame.groupBy : keys: seq<Expr> -> df: DataFrame -> GroupByParameters
| Name | Type | Description |
|---|---|---|
keys | seq<Expr> | - |
df | DataFrame | - |
DataFrame.having
Having clause for GroupBy.
DataFrame.having : predicate: Expr -> builder: GroupBy -> GroupByParameters
| Name | Type | Description |
|---|---|---|
predicate | Expr | - |
builder | GroupBy | - |
DataFrame.head
Get the first n rows of the DataFrame.
DataFrame.head : n: int -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
n | int | - |
df | DataFrame | - |
DataFrame.hstack
Horizontally stack columns to the DataFrame.
DataFrame.hstack : columns: seq<Series> -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | seq<Series> | - |
df | DataFrame | - |
DataFrame.iter2
Pipeline combinator for DataFrame.Iter2: ('T1 -> 'T2 -> unit) -> DataFrame -> DataFrame -> unit.
DataFrame.iter2 : action: ('T1 -> ('T2 -> unit)) -> df1: DataFrame -> df2: DataFrame -> unitParameters
| Name | Type | Description |
|---|---|---|
action | ('T1 -> ('T2 -> unit)) | - |
df1 | DataFrame | - |
df2 | DataFrame | - |
DataFrame.iterRows
Pipeline combinator for DataFrame.IterRows: ('T -> unit) -> DataFrame -> unit.
DataFrame.iterRows : action: ('T -> unit) -> df: DataFrame -> unitParameters
| Name | Type | Description |
|---|---|---|
action | ('T -> unit) | - |
df | DataFrame | - |
DataFrame.iterate
Repeatedly applies an update function to a DataFrame for n steps, returning the final materialized state.
DataFrame.iterate : steps: int -> stepFn: (int -> (DataFrame -> DataFrame)) -> initial: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
steps | int | - |
stepFn | (int -> (DataFrame -> DataFrame)) | - |
initial | DataFrame | - |
DataFrame.iteri2
Pipeline combinator for DataFrame.Iteri2: (int64 -> 'T1 -> 'T2 -> unit) -> DataFrame -> DataFrame -> unit.
DataFrame.iteri2 : action: (int64 -> ('T1 -> ('T2 -> unit))) -> df1: DataFrame -> df2: DataFrame -> unitParameters
| Name | Type | Description |
|---|---|---|
action | (int64 -> ('T1 -> ('T2 -> unit))) | - |
df1 | DataFrame | - |
df2 | DataFrame | - |
DataFrame.iteriRows
Pipeline combinator for DataFrame.IteriRows: (int64 -> 'T -> unit) -> DataFrame -> unit.
DataFrame.iteriRows : action: (int64 -> ('T -> unit)) -> df: DataFrame -> unitParameters
| Name | Type | Description |
|---|---|---|
action | (int64 -> ('T -> unit)) | - |
df | DataFrame | - |
DataFrame.join
Perform a join between two DataFrames.
DataFrame.join : other: DataFrame -> leftOn: seq<Expr> -> rightOn: seq<Expr> -> how: JoinType -> left: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | - |
leftOn | seq<Expr> | - |
rightOn | seq<Expr> | - |
how | JoinType | - |
left | DataFrame | - |
DataFrame.joinOn
Perform a join between two DataFrames on a common column.
DataFrame.joinOn : other: DataFrame -> on: seq<Expr> -> how: JoinType -> left: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | - |
on | seq<Expr> | - |
how | JoinType | - |
left | DataFrame | - |
DataFrame.map
Transforms each column in the DataFrame using a column-wise mapping function (Series -> Series).
DataFrame.map : mapping: (Series -> Series) -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
mapping | (Series -> Series) | The function applied to each column Series. |
df | DataFrame | - |
Returns: A new DataFrame composed of the transformed Series.
DataFrame.mapRows
Transforms DataFrame rows into a new DataFrame using a strongly-typed F# Record mapping function ('TIn -> 'TOut).
DataFrame.mapRows : mapping: ('TIn -> 'TOut) -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
mapping | ('TIn -> 'TOut) | The pure transformation function applied to each row. |
df | DataFrame | - |
Returns: A new DataFrame materialized from the mapped output records.
DataFrame.melt
Alias for DataFrame.unpivot.
DataFrame.melt : index: Selector -> on: Selector -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector | - |
on | Selector | - |
df | DataFrame | - |
DataFrame.meltNamed
Alias for DataFrame.unpivotNamed.
DataFrame.meltNamed : index: Selector -> on: Selector -> variableName: string -> valueName: string -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector | - |
on | Selector | - |
variableName | string | - |
valueName | string | - |
df | DataFrame | - |
DataFrame.orderByAscending
Order By the DataFrame in ascending order.
DataFrame.orderByAscending : columns: seq<IColumnExpr> -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | seq<IColumnExpr> | - |
df | DataFrame | - |
DataFrame.orderByDescending
Order By the DataFrame in descending order.
DataFrame.orderByDescending : columns: seq<IColumnExpr> -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | seq<IColumnExpr> | - |
df | DataFrame | - |
DataFrame.partitionBy
Group by the given columns and return the groups as separate dataframes.
DataFrame.partitionBy : columns: seq<string> -> df: DataFrame -> DataFrame[]Parameters
| Name | Type | Description |
|---|---|---|
columns | seq<string> | - |
df | DataFrame | - |
DataFrame.partitionMap
Partitions the DataFrame using an F# Record key with default ordering preserved (maintainOrder = true). Automatically infers partition column names from 'Key record fields.
DataFrame.partitionMap : df: DataFrame -> Map<'Key, DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | The source DataFrame. |
Returns: An immutable F# Map of partition keys to partition DataFrames.
DataFrame.partitionMapBy
Partitions the DataFrame by explicitly specified column names into an F# Record key.
DataFrame.partitionMapBy : byCols: seq<string> -> df: DataFrame -> Map<'Key, DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
byCols | seq<string> | Column names to partition by. |
df | DataFrame | The source DataFrame. |
DataFrame.partitionMapByWith
Partitions the DataFrame by explicitly specified column names with full control over ordering.
DataFrame.partitionMapByWith : byCols: seq<string> -> maintainOrder: bool -> df: DataFrame -> Map<'Key, DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
byCols | seq<string> | - |
maintainOrder | bool | - |
df | DataFrame | - |
DataFrame.partitionMapWith
Partitions the DataFrame using an F# Record key with explicit maintainOrder configuration.
DataFrame.partitionMapWith : maintainOrder: bool -> df: DataFrame -> Map<'Key, DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
maintainOrder | bool | Ensure consistent group ordering. Setting to false improves partitioning throughput. |
df | DataFrame | The source DataFrame. |
DataFrame.rename
Rename columns based on a mapping dictionary or key-value pairs.
DataFrame.rename : mapping: seq<(string * string)> -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
mapping | seq<(string * string)> | - |
df | DataFrame | - |
DataFrame.reverse
Reverse the order of the DataFrame.
DataFrame.reverse : df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
DataFrame.select
Select columns from the DataFrame.
DataFrame.select : exprs: seq<'a> -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
exprs | seq<'a> | - |
df | DataFrame | - |
DataFrame.show
Print the DataFrame to Console (Table format).
DataFrame.show : df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
DataFrame.slice
Slice rows starting at offset with length.
DataFrame.slice : offset: int64 -> length: uint64 -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
offset | int64 | - |
length | uint64 | - |
df | DataFrame | - |
DataFrame.sortAscending
Sort (Order By) the DataFrame in ascending order.
DataFrame.sortAscending : columns: seq<IColumnExpr> -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | seq<IColumnExpr> | - |
df | DataFrame | - |
DataFrame.sortDescending
Sort (Order By) the DataFrame in descending order.
DataFrame.sortDescending : columns: seq<IColumnExpr> -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | seq<IColumnExpr> | - |
df | DataFrame | - |
DataFrame.tail
Get the last n rows of the DataFrame.
DataFrame.tail : n: int -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
n | int | - |
df | DataFrame | - |
DataFrame.toArrow
Convert the DataFrame to an Arrow RecordBatch.
DataFrame.toArrow : df: DataFrame -> RecordBatchParameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
DataFrame.toAsyncEnumerable
Streams in-memory DataFrame rows asynchronously.
DataFrame.toAsyncEnumerable : chunkSize: int -> df: DataFrame -> IAsyncEnumerable<'T>Parameters
| Name | Type | Description |
|---|---|---|
chunkSize | int | - |
df | DataFrame | - |
DataFrame.toMap
Convert the DataFrame to a map of column name to Series.
DataFrame.toMap : df: DataFrame -> Map<string, Series>Parameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
DataFrame.toRecords
Convert the DataFrame to a sequence of records.
DataFrame.toRecords : df: DataFrame -> seq<'T>Parameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
DataFrame.toRows
Returns a row enumerator for the DataFrame, allowing iteration over rows of type 'T.
DataFrame.toRows : df: DataFrame -> RowEnumerator<'T>Parameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | The DataFrame to enumerate. |
Returns: A row enumerator for the DataFrame.
DataFrame.toRowsAsync
Streams in-memory DataFrame rows asynchronously.
DataFrame.toRowsAsync : chunkSize: int -> df: DataFrame -> IAsyncEnumerable<'T>Parameters
| Name | Type | Description |
|---|---|---|
chunkSize | int | - |
df | DataFrame | - |
DataFrame.toStruct
Convert the DataFrame to a sequence of structs.
DataFrame.toStruct : name: string -> df: DataFrame -> SeriesParameters
| Name | Type | Description |
|---|---|---|
name | string | - |
df | DataFrame | - |
DataFrame.unique
Return unique rows from the DataFrame.
DataFrame.unique : df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
DataFrame.uniqueOn
Drop duplicate rows based on specific subset expression(s), keeping the first occurrence.
DataFrame.uniqueOn : subset: seq<Expr> -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
subset | seq<Expr> | - |
df | DataFrame | - |
DataFrame.uniqueWith
Drop duplicate rows based on subset with a specified keep strategy (First, Last, None, Any).
DataFrame.uniqueWith : subset: seq<Expr> -> keep: UniqueKeepStrategy -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
subset | seq<Expr> | - |
keep | UniqueKeepStrategy | - |
df | DataFrame | - |
DataFrame.unnestColumns
Decompose multiple struct columns.
DataFrame.unnestColumns : columns: seq<string> -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
columns | seq<string> | - |
df | DataFrame | - |
DataFrame.unpivot
Unpivot (Melt) the DataFrame from wide to long format using Selectors. Default column names ("variable", "value") will be used.
DataFrame.unpivot : index: Selector -> on: Selector -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector | - |
on | Selector | - |
df | DataFrame | - |
DataFrame.unpivotNamed
Unpivot (Melt) the DataFrame using Selectors with custom variable and value column names.
DataFrame.unpivotNamed : index: Selector -> on: Selector -> variableName: string -> valueName: string -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector | - |
on | Selector | - |
variableName | string | - |
valueName | string | - |
df | DataFrame | - |
DataFrame.vstack
Vertically stack another DataFrame to this one.
DataFrame.vstack : other: DataFrame -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
other | DataFrame | - |
df | DataFrame | - |
DataFrame.withColumn
Add or replace a single column in the DataFrame.
DataFrame.withColumn : expr: Expr -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
expr | Expr | - |
df | DataFrame | - |
DataFrame.withColumns
Add or replace multiple columns in the DataFrame.
DataFrame.withColumns : exprs: seq<Expr> -> df: DataFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
exprs | seq<Expr> | - |
df | DataFrame | - |