LazyFrame
Namespace: Polars.FSharp
A LazyFrame represents a logical plan of operations that will be optimized and executed only when collected.
Operations on LazyFrame are not executed immediately. Instead, they build a query plan. Use Collect() to execute the plan and get a DataFrame.
Properties & Indexers
| Property | Type | Description |
|---|---|---|
ColumnNames | string list | - |
Columns | string list | - |
DataTypes | DataType list | - |
Schema | PolarsSchema | Get the schema of the LazyFrame without executing it. Uses Zero-Copy native introspection. |
Constructors & Static Methods
Concat
Concatenate multiple LazyFrames.
static member Concat(
lfs: seq<LazyFrame>,
?how: ConcatType,
?rechunk: bool,
?useParallel: bool
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
lfs | seq<LazyFrame> | - |
how | ConcatType (optional) | - |
rechunk | bool (optional) | - |
useParallel | bool (optional) | - |
ConcatDiagonal
Concatenate multiple LazyFrames diagonally.
static member ConcatDiagonal(lfs: seq<LazyFrame>) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
lfs | seq<LazyFrame> | - |
ConcatHorizontal
Concatenate multiple LazyFrames horizontally.
static member ConcatHorizontal(lfs: seq<LazyFrame>) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
lfs | seq<LazyFrame> | - |
ConcatVertical
Concatenate multiple LazyFrames vertically.
static member ConcatVertical(lfs: seq<LazyFrame>) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
lfs | seq<LazyFrame> | - |
ScanCsv
Lazily scans a CSV file into a LazyFrame.
This allows for query optimization (predicate pushdown, projection pushdown) and streaming processing of datasets larger than memory.
Lazily scans a CSV from an in-memory byte array.
Lazily scans a CSV from a Stream.
This reads the stream fully into memory to construct the Lazy execution plan.
Helper: Scan CSV with default settings
static member ScanCsv(
path: string,
?schema: PolarsSchema,
?dtypeOverride: PolarsSchema,
?hasHeader: bool,
?separator: char,
?quoteChar: char,
?eolChar: char,
?ignoreErrors: bool,
?tryParseDates: bool,
?lowMemory: bool,
?cache: bool,
?glob: bool,
?rechunk: bool,
?raiseIfEmpty: bool,
?skipRows: uint64,
?skipRowsAfterHeader: uint64,
?skipLines: uint64,
?nRows: uint64,
?inferSchemaLength: uint64,
?inferSchemaFiles: uint64,
?nThreads: uint64,
?chunkSize: uint64,
?rowIndexName: string,
?rowIndexOffset: uint64,
?includeFilePaths: string,
?encoding: CsvEncoding,
?nullValues: seq<string>,
?missingIsNull: bool,
?commentPrefix: string,
?decimalComma: bool,
?truncateRaggedLines: bool,
?cloudOptions: CloudOptions
) : LazyFrame
static member ScanCsv(
buffer: byte[],
?schema: PolarsSchema,
?dtypeOverride: PolarsSchema,
?hasHeader: bool,
?separator: char,
?quoteChar: char,
?eolChar: char,
?ignoreErrors: bool,
?tryParseDates: bool,
?lowMemory: bool,
?cache: bool,
?glob: bool,
?rechunk: bool,
?raiseIfEmpty: bool,
?skipRows: uint64,
?skipRowsAfterHeader: uint64,
?skipLines: uint64,
?nRows: uint64,
?inferSchemaLength: uint64,
?inferSchemaFiles: uint64,
?nThreads: uint64,
?chunkSize: uint64,
?rowIndexName: string,
?rowIndexOffset: uint64,
?includeFilePaths: string,
?encoding: CsvEncoding,
?nullValues: seq<string>,
?missingIsNull: bool,
?commentPrefix: string,
?decimalComma: bool,
?truncateRaggedLines: bool
) : LazyFrame
static member ScanCsv(
stream: Stream,
?schema: PolarsSchema,
?hasHeader: bool,
?separator: char,
?quoteChar: char,
?eolChar: char,
?ignoreErrors: bool,
?tryParseDates: bool,
?lowMemory: bool,
?cache: bool,
?glob: bool,
?rechunk: bool,
?raiseIfEmpty: bool,
?skipRows: uint64,
?skipRowsAfterHeader: uint64,
?skipLines: uint64,
?nRows: uint64,
?inferSchemaLength: uint64,
?inferSchemaFiles: uint64,
?nThreads: uint64,
?chunkSize: uint64,
?rowIndexName: string,
?rowIndexOffset: uint64,
?includeFilePaths: string,
?encoding: CsvEncoding,
?nullValues: seq<string>,
?missingIsNull: bool,
?commentPrefix: string,
?decimalComma: bool,
?truncateRaggedLines: bool
) : LazyFrame
static member ScanCsv(path: string) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
schema | PolarsSchema (optional) | - |
dtypeOverride | PolarsSchema (optional) | - |
hasHeader | bool (optional) | - |
separator | char (optional) | - |
quoteChar | char (optional) | - |
eolChar | char (optional) | - |
ignoreErrors | bool (optional) | - |
tryParseDates | bool (optional) | - |
lowMemory | bool (optional) | - |
cache | bool (optional) | - |
glob | bool (optional) | - |
rechunk | bool (optional) | - |
raiseIfEmpty | bool (optional) | - |
skipRows | uint64 (optional) | - |
skipRowsAfterHeader | uint64 (optional) | - |
skipLines | uint64 (optional) | - |
nRows | uint64 (optional) | - |
inferSchemaLength | uint64 (optional) | - |
inferSchemaFiles | uint64 (optional) | - |
nThreads | uint64 (optional) | - |
chunkSize | uint64 (optional) | - |
rowIndexName | string (optional) | - |
rowIndexOffset | uint64 (optional) | - |
includeFilePaths | string (optional) | - |
encoding | CsvEncoding (optional) | - |
nullValues | seq<string> (optional) | - |
missingIsNull | bool (optional) | - |
commentPrefix | string (optional) | - |
decimalComma | bool (optional) | - |
truncateRaggedLines | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
buffer | byte[] | - |
stream | Stream | - |
ScanDb
Scan a database query lazily. Requires a factory function to create new IDataReaders for potential multi-pass scans.
[Lazy][Streaming] Scan a database DataReader directly.
static member ScanDb(readerFactory: (unit -> IDataReader), ?batchSize: int, ?useBuffered: bool) : LazyFrame
static member ScanDb(reader: IDataReader, ?batchSize: int) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
readerFactory | (unit -> IDataReader) | - |
batchSize | int (optional) | - |
useBuffered | bool (optional) | - |
reader | IDataReader | - |
ScanDelta
Create a LazyFrame by scanning a Delta Lake table.
static member ScanDelta(
path: string,
?version: uint64,
?datetime: string,
?nRows: uint64,
?parallelStrategy: ParallelStrategy,
?lowMemory: bool,
?useStatistics: bool,
?glob: bool,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?schema: PolarsSchema,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool,
?cloudOptions: CloudOptions
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | Path to the Delta Lake table (folder containing _delta_log). |
version | uint64 (optional) | The version of the table to read (e.g., 0L, 1L). Mutually exclusive with datetime. |
datetime | string (optional) | The timestamp to read (ISO-8601 string, e.g., "2026-02-09T12:00:00Z"). Mutually exclusive with version. |
nRows | uint64 (optional) | - |
parallelStrategy | ParallelStrategy (optional) | - |
lowMemory | bool (optional) | - |
useStatistics | bool (optional) | - |
glob | bool (optional) | - |
rechunk | bool (optional) | - |
cache | bool (optional) | - |
rowIndexName | string (optional) | - |
rowIndexOffset | uint32 (optional) | - |
includePathColumn | string (optional) | - |
schema | PolarsSchema (optional) | - |
hivePartitioning | bool (optional) | - |
hivePartitionSchema | PolarsSchema (optional) | - |
tryParseHiveDates | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
Returns: A new LazyFrame.
ScanIpc
Lazily read an Arrow IPC (Feather v2) file, multiple files via glob patterns, or cloud storage.
Lazily read Arrow IPC (Feather v2) from in-memory bytes.
Lazily read Arrow IPC (Feather v2) from a Stream.
static member ScanIpc(
path: string,
?schema: PolarsSchema,
?nRows: uint64,
?rechunk: bool,
?cache: bool,
?glob: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool,
?cloudOptions: CloudOptions
) : LazyFrame
static member ScanIpc(
buffer: byte[],
?schema: PolarsSchema,
?nRows: uint64,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool
) : LazyFrame
static member ScanIpc(
stream: Stream,
?schema: PolarsSchema,
?nRows: uint64,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
schema | PolarsSchema (optional) | - |
nRows | uint64 (optional) | - |
rechunk | bool (optional) | - |
cache | bool (optional) | - |
glob | bool (optional) | - |
rowIndexName | string (optional) | - |
rowIndexOffset | uint32 (optional) | - |
includePathColumn | string (optional) | - |
hivePartitioning | bool (optional) | - |
hivePartitionSchema | PolarsSchema (optional) | - |
tryParseHiveDates | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
buffer | byte[] | - |
stream | Stream | - |
NOTE
This reads the stream fully into memory to construct the Lazy execution plan.
ScanNdjson
Lazily read a NDJSON file.
Lazily read NDJSON from in-memory bytes.
Lazily read NDJSON from a Stream.
static member ScanNdjson(
path: string,
?schema: PolarsSchema,
?inferSchemaLen: uint64,
?batchSize: uint64,
?nRows: uint64,
?lowMemory: bool,
?rechunk: bool,
?ignoreErrors: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?cloudOptions: CloudOptions
) : LazyFrame
static member ScanNdjson(
buffer: byte[],
?schema: PolarsSchema,
?inferSchemaLen: uint64,
?batchSize: uint64,
?nRows: uint64,
?lowMemory: bool,
?rechunk: bool,
?ignoreErrors: bool,
?rowIndexName: string,
?rowIndexOffset: uint32
) : LazyFrame
static member ScanNdjson(
stream: Stream,
?schema: PolarsSchema,
?inferSchemaLen: uint64,
?batchSize: uint64,
?nRows: uint64,
?lowMemory: bool,
?rechunk: bool,
?ignoreErrors: bool,
?rowIndexName: string,
?rowIndexOffset: uint32
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
schema | PolarsSchema (optional) | - |
inferSchemaLen | uint64 (optional) | - |
batchSize | uint64 (optional) | - |
nRows | uint64 (optional) | - |
lowMemory | bool (optional) | - |
rechunk | bool (optional) | - |
ignoreErrors | bool (optional) | - |
rowIndexName | string (optional) | - |
rowIndexOffset | uint32 (optional) | - |
includePathColumn | string (optional) | - |
cloudOptions | CloudOptions (optional) | - |
buffer | byte[] | - |
stream | Stream | - |
ScanParquet
Scan a parquet file into a LazyFrame.
Lazily read from a parquet file or a common cloud store (S3, GCS, Azure, etc.).
[Memory] Lazily read parquet from a byte array (in-memory buffer).
static member ScanParquet(
path: string,
?nRows: uint64,
?parallelStrategy: ParallelStrategy,
?lowMemory: bool,
?useStatistics: bool,
?glob: bool,
?allowMissingColumns: bool,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?schema: PolarsSchema,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool,
?cloudOptions: CloudOptions
) : LazyFrame
static member ScanParquet(
buffer: byte[],
?nRows: uint64,
?parallelStrategy: ParallelStrategy,
?lowMemory: bool,
?useStatistics: bool,
?glob: bool,
?allowMissingColumns: bool,
?rechunk: bool,
?cache: bool,
?rowIndexName: string,
?rowIndexOffset: uint32,
?includePathColumn: string,
?schema: PolarsSchema,
?hivePartitioning: bool,
?hivePartitionSchema: PolarsSchema,
?tryParseHiveDates: bool
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
path | string | Path to file or cloud location. |
nRows | uint64 (optional) | Stop reading after n rows. |
parallelStrategy | ParallelStrategy (optional) | - |
lowMemory | bool (optional) | Reduce memory pressure at the expense of performance. |
useStatistics | bool (optional) | Use parquet statistics to prune row groups. |
glob | bool (optional) | Expand path using globbing rules. |
allowMissingColumns | bool (optional) | If true, do not fail if columns are missing. |
rechunk | bool (optional) | Rechunk the memory to contiguous chunks when reading. (default: false) |
cache | bool (optional) | Cache the result after reading. (default: true) |
rowIndexName | string (optional) | If provided, add a row index column with this name. |
rowIndexOffset | uint32 (optional) | Start index for the row index column. |
includePathColumn | string (optional) | If provided, add a column with the path of the file. |
schema | PolarsSchema (optional) | Overwrite the schema of the dataset. |
hivePartitioning | bool (optional) | - |
hivePartitionSchema | PolarsSchema (optional) | The schema of the hive partitions. |
tryParseHiveDates | bool (optional) | Attempt to parse hive values as Date/Datetime. |
cloudOptions | CloudOptions (optional) | Options for cloud storage (AWS S3, Azure Blob, GCS, etc.). |
buffer | byte[] | The byte array containing parquet data. |
ScanSeq
Lazily scan a sequence of objects using Apache Arrow Stream Interface. This supports predicate pushdown and streaming execution. Data is pulled from the sequence only when needed.
static member ScanSeq(data: seq<'T>, ?batchSize: int, ?useBuffered: bool) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
data | seq<'T> | The data source sequence. |
batchSize | int (optional) | Rows per Arrow batch (default: 100,000). |
useBuffered | bool (optional) | Choose whether disk buffer file needed (for big data) |
Instance Methods
AsQueryable
Convert LazyFrame into an IQueryable LINQ query provider. Operations will be translated to Polars Expressions and compiled into execution plans.
Directly creates an IQueryable with type inferred from an existing sample sequence
member AsQueryable : unit -> IQueryable<'T>
member AsQueryable(sample: seq<'T>) : IQueryable<'T>Parameters
| Name | Type | Description |
|---|---|---|
sample | seq<'T> | - |
BottomK
Get the bottom k rows based on the given columns.
member BottomK(k: int, by: seq<'a>, ?reverse: seq<bool>) : LazyFrame
member BottomK(k: int, by: seq<'a>, reverse: bool) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
k | int | - |
by | seq<'a> | - |
reverse | seq<bool> (optional) | - |
Cast
Cast LazyFrame column(s) to the specified dtype(s) using a dictionary mapping.
Cast all columns in the LazyFrame to the specified dtype.
Cast columns matching an Expr/Selector to a specific DataType.
Cast multiple expressions/selectors to their target DataTypes. Example: lf.Cast([ (pl.cs.Numeric(), DataType.Float32); (col("Id"), DataType.Int32) ])
member Cast(dtypes: seq<(string * DataType)>, ?strict: bool) : LazyFrame
member Cast(dtype: DataType, ?strict: bool) : LazyFrame
member Cast(expr: Expr, dtype: DataType, ?strict: bool) : LazyFrame
member Cast(dtypes: seq<(Expr * DataType)>, ?strict: bool) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
dtypes | seq<(string * DataType)> / seq<(Expr * DataType)> | - |
strict | bool (optional) | - |
dtype | DataType | - |
expr | Expr | - |
Clear
Create an empty (n=0) or n-row null-filled (n>0) copy of the LazyFrame. Returns a n-row null-filled LazyFrame with an identical schema.
member Clear(?n: int64) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
n | int64 (optional) | Number of (null-filled) rows to return in the cleared frame. |
Returns: A new LazyFrame.
Clone
member Clone : unit -> LazyFrameCollect
Execute the plan and return a DataFrame.
member Collect(?engine: Engine) : DataFrameParameters
| Name | Type | Description |
|---|---|---|
engine | Engine (optional) | - |
CollectAsync
member CollectAsync(?engine: Engine, ?cancellationToken: CancellationToken) : Task<DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
engine | Engine (optional) | - |
cancellationToken | CancellationToken (optional) | - |
CollectBatchesAsync
Streams materialized DataFrame batches asynchronously using the native Polars streaming batch engine. Native memory consumption is strictly bounded by O(batch) size.
member CollectBatchesAsync(
?chunkSize: int64,
?engine: Engine,
?maintainOrder: bool,
?cancellationToken: CancellationToken
) : IAsyncEnumerable<DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
chunkSize | int64 (optional) | Maximum row count per native batch (0 uses Polars default batch size). |
engine | Engine (optional) | Execution engine selection. |
maintainOrder | bool (optional) | Whether to enforce deterministic row ordering across batches. |
cancellationToken | CancellationToken (optional) | Optional cooperative cancellation token. |
Returns: An asynchronous stream of DataFrame batches.
CollectSchema
member CollectSchema : unit -> PolarsSchemaCount
Return the number of non-null elements for each column.
member Count : unit -> LazyFrameDescribe
Generate a summary statistics DataFrame (count, mean, std, min, 25%, 50%, 75%, max). Similar to pandas/polars describe(). Notice: This will collect LazyFrame once, but LazyFrame won't be consumed.
member Describe : unit -> DataFrameDrop
Drop selected columns by selector.
Drop selected columns by column names.
Drop columns by specific Expressions.
member Drop(selector: Selector) : LazyFrame
member Drop([<ParamArray>] columns: string[]) : LazyFrame
member Drop(columns: 'a) : LazyFrame
member Drop(exprs: seq<Expr>) : LazyFrame
member Drop([<ParamArray>] exprs: Expr[]) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
selector | Selector | - |
columns | string[] / 'a | - |
exprs | seq<Expr> / Expr[] | - |
DropNans
Drop rows containing one or more NaN values.
Drop rows with NaN in specific columns.
Drop rows with NaN by specific Expressions.
member DropNans(?subset: Selector) : LazyFrame
member DropNans([<ParamArray>] subset: string[]) : LazyFrame
member DropNans([<ParamArray>] exprs: Expr[]) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
subset | Selector (optional) / string[] | - |
exprs | Expr[] | - |
DropNulls
Drop rows containing one or more Null values.
Drop rows with Nulls in specific columns.
Drop rows with Nulls by specific Expressions.
member DropNulls(?subset: Selector) : LazyFrame
member DropNulls([<ParamArray>] subset: string[]) : LazyFrame
member DropNulls([<ParamArray>] exprs: Expr[]) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
subset | Selector (optional) / string[] | - |
exprs | Expr[] | - |
Explain
Print the query plan.
member Explain(?optimized: bool) : stringParameters
| Name | Type | Description |
|---|---|---|
optimized | bool (optional) | - |
Explode
member Explode(selector: Selector, ?emptyAsNull: bool, ?keepNulls: bool) : LazyFrame
member Explode(columns: seq<string>) : LazyFrame
member Explode(column: string) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
selector | Selector | - |
emptyAsNull | bool (optional) | - |
keepNulls | bool (optional) | - |
columns | seq<string> | - |
column | string | - |
FillNan
Fill NaN values in floating point columns with a specified Expression.
member FillNan(fillValue: Expr) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
fillValue | Expr | - |
FillNull
Fill null values in all columns with a specified Expression.
member FillNull(fillValue: Expr) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
fillValue | Expr | - |
Filter
Filter rows based on a boolean expression.
In a LazyFrame, this operation is added to the logical plan and is optimized before execution. Polars will attempt to push this filter down as close to the data source as possible (Predicate Pushdown).
member Filter(expr: Expr) : LazyFrame
member Filter(series: Series) : LazyFrame
member Filter(mask: seq<bool>) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
expr | Expr | - |
series | Series | - |
mask | seq<bool> | - |
First
member First : unit -> LazyFrameGather
Selects rows from this LazyFrame at the given indices.
member Gather(index: seq<int64>, ?nullOnOutOfBounds: bool) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
index | seq<int64> | The indices of the rows to select. |
nullOnOutOfBounds | bool (optional) | If true when an index is out-of-bounds a null row will be generated instead of raising an error. |
GatherEvery
Take every nth row in the Frame and return as a new Frame.
member GatherEvery(n: uint64, ?offset: uint64) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
n | uint64 | Gather every n-th row. |
offset | uint64 (optional) | Starting Index |
GroupBy
Start a GroupBy operation.
member GroupBy(keys: seq<'a>, ?maintainOrder: bool) : LazyGroupByParameters
| Name | Type | Description |
|---|---|---|
keys | seq<'a> | - |
maintainOrder | bool (optional) | - |
GroupByDynamic
Start a dynamic group-by (rolling window) operation. Returns a Builder.
member GroupByDynamic(
indexCol: string,
every: Dur,
?period: Dur,
?offset: Dur,
?by: seq<'a>,
?label: Label,
?includeBoundaries: bool,
?closedWindow: ClosedInterval,
?startBy: StartBy
) : LazyGroupByParameters
| Name | Type | Description |
|---|---|---|
indexCol | string | - |
every | Dur | - |
period | Dur (optional) | - |
offset | Dur (optional) | - |
by | seq<'a> (optional) | - |
label | Label (optional) | - |
includeBoundaries | bool (optional) | - |
closedWindow | ClosedInterval (optional) | - |
startBy | StartBy (optional) | - |
GroupByRolling
Start a rolling group-by operation. Returns a Builder.
member GroupByRolling(
indexCol: string,
period: Dur,
?offset: Dur,
?by: seq<'a>,
?closedWindow: ClosedInterval
) : LazyGroupByParameters
| Name | Type | Description |
|---|---|---|
indexCol | string | - |
period | Dur | - |
offset | Dur (optional) | - |
by | seq<'a> (optional) | - |
closedWindow | ClosedInterval (optional) | - |
Head
member Head(?n: uint32) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
n | uint32 (optional) | - |
Interpolate
Interpolate intermediate values. The interpolation method is linear. Nulls at the beginning and end of the series remain null.
member Interpolate : unit -> LazyFrameJoin
Join with another LazyFrame using column names.
member Join(
other: LazyFrame,
leftOn: seq<Expr>,
rightOn: seq<Expr>,
how: JoinType,
?suffix: string,
?validation: JoinValidation,
?coalesce: JoinCoalesce,
?maintainOrder: JoinMaintainOrder,
?joinSide: JoinSide,
?nullsEqual: bool,
?sliceOffset: int64,
?sliceLen: uint64
) : LazyFrame
member Join(
other: LazyFrame,
on: seq<Expr>,
how: JoinType,
?suffix: string,
?validation: JoinValidation,
?coalesce: JoinCoalesce,
?maintainOrder: JoinMaintainOrder,
?joinSide: JoinSide,
?nullsEqual: bool,
?sliceOffset: int64,
?sliceLen: uint64
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
other | LazyFrame | The right DataFrame to join with. |
leftOn | seq<Expr> | Column names in the left DataFrame to join on. |
rightOn | seq<Expr> | Column names in the right DataFrame to join on. |
how | JoinType | Type of join (Inner, Left, Outer, Cross, etc.). Default is Inner. |
suffix | string (optional) | Suffix to append to columns with same name in right DataFrame. Default "_right". |
validation | JoinValidation (optional) | Check if join keys are unique. |
coalesce | JoinCoalesce (optional) | How to coalesce the join keys. |
maintainOrder | JoinMaintainOrder (optional) | How to maintain the order of the join. |
joinSide | JoinSide (optional) | Specifies the strategy for the hash join build side. |
nullsEqual | bool (optional) | Consider nulls as equal. |
sliceOffset | int64 (optional) | Slice the result starting at this offset. |
sliceLen | uint64 (optional) | Length of the slice. |
on | seq<Expr> | - |
JoinAsOf
Perform an As-of join (also known as a time-series join).
This is similar to a left join except that we match on nearest key rather than equal keys. The join keys must be sorted.
member JoinAsOf(
other: LazyFrame,
leftOn: Expr,
rightOn: Expr,
tolerance: Tolerance,
?strategy: AsofStrategy,
?byLeft: seq<Expr>,
?byRight: seq<Expr>,
?allowEq: bool,
?checkSorted: bool,
?suffix: string,
?validation: JoinValidation,
?coalesce: JoinCoalesce,
?maintainOrder: JoinMaintainOrder,
?joinSide: JoinSide,
?nullsEqual: bool,
?sliceOffset: int64,
?sliceLen: uint64
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
other | LazyFrame | The right LazyFrame to join with. |
leftOn | Expr | Join key of the left LazyFrame. Must be sorted. |
rightOn | Expr | Join key of the right LazyFrame. Must be sorted. |
tolerance | Tolerance | Tolerance as a time duration string (e.g., "2h", "10s", "1d"), int or double or TimeSpan. Matches that are further away than this duration are discarded. |
strategy | AsofStrategy (optional) | The strategy to determine which value is "nearest" (Backward, Forward, or Nearest). Defaults to AsofStrategy.Backward. |
byLeft | seq<Expr> (optional) | - |
byRight | seq<Expr> (optional) | - |
allowEq | bool (optional) | If true, allow exact matches to be included in the result. If false, a match must be strictly unequal (e.g. less than for Backward strategy) to the key. |
checkSorted | bool (optional) | Check if the join keys are sorted. If false, the user must ensure keys are sorted; otherwise results are undefined (but execution is faster). |
suffix | string (optional) | Suffix to append to columns with name conflicts. Defaults to "_right". |
validation | JoinValidation (optional) | Check if join keys are unique (mostly relevant for the 'by' columns). |
coalesce | JoinCoalesce (optional) | How to coalesce the join keys. |
maintainOrder | JoinMaintainOrder (optional) | How to maintain the order of the join. |
joinSide | JoinSide (optional) | pecifies the strategy for the hash join build side. |
nullsEqual | bool (optional) | Consider nulls as equal. |
sliceOffset | int64 (optional) | Slice the result starting at this offset (optimization). |
sliceLen | uint64 (optional) | Length of the slice to keep. |
JoinWhere
Join with another LazyFrame using multiple boolean expressions.
member JoinWhere(
other: LazyFrame,
predicates: seq<Expr>,
?how: JoinType,
?suffix: string,
?validation: JoinValidation,
?coalesce: JoinCoalesce,
?maintainOrder: JoinMaintainOrder,
?nullsEqual: bool
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
other | LazyFrame | - |
predicates | seq<Expr> | - |
how | JoinType (optional) | - |
suffix | string (optional) | - |
validation | JoinValidation (optional) | - |
coalesce | JoinCoalesce (optional) | - |
maintainOrder | JoinMaintainOrder (optional) | - |
nullsEqual | bool (optional) | - |
Last
Get the last row.
member Last : unit -> LazyFrameLimit
Get the first n rows.
member Limit(?n: uint32) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
n | uint32 (optional) | Number of rows to return. |
MatchToSchema
member MatchToSchema(
schema: PolarsSchema,
?extraColumns: ExtraColumnsPolicy,
?defaultConfig: MatchSchemaConfig,
?columnOverrides: Map<string, MatchSchemaConfig>
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
schema | PolarsSchema | - |
extraColumns | ExtraColumnsPolicy (optional) | - |
defaultConfig | MatchSchemaConfig (optional) | - |
columnOverrides | Map<string, MatchSchemaConfig> (optional) | - |
Max
Aggregate the columns in the Frame to their maximum value.
member Max : unit -> LazyFrameMean
Aggregate the columns in the Frame to their mean value.
member Mean : unit -> LazyFrameMedian
Aggregate the columns in the Frame to their median value.
member Median : unit -> LazyFrameMelt
member Melt(
index: Selector,
on: Selector,
variableName: string,
valueName: string
) : LazyFrame
member Melt(
index: seq<string>,
on: seq<string>,
?variableName: string,
?valueName: string
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector / seq<string> | - |
on | Selector / seq<string> | - |
variableName | string / string (optional) | - |
valueName | string / string (optional) | - |
Merge
Initiates a Merge (upsert) plan using this LazyFrame as the target and the given source. The on selector is expanded against the target to determine the join keys.
member Merge(source: LazyFrame, on: IColumnExpr) : MergePlanParameters
| Name | Type | Description |
|---|---|---|
source | LazyFrame | The source LazyFrame containing updates/inserts. |
on | IColumnExpr | A column expression (e.g., pl.col "Id") or selector (e.g., pl.cs.endsWith "Id") that resolves to the merge key columns. |
Returns: A MergePlan that can be further configured and then executed.
MergeCatalogRecords
Starts building a Merge (Upsert) operation using this LazyFrame as the source.
member MergeCatalogRecords(
catalog: UnityCatalog,
catalogName: string,
schemaName: string,
tableName: string,
mergeKeys: string[],
?canEvolve: bool,
?cloudOptions: CloudOptions
) : DeltaMergeBuilderParameters
| Name | Type | Description |
|---|---|---|
catalog | UnityCatalog | - |
catalogName | string | - |
schemaName | string | - |
tableName | string | - |
mergeKeys | string[] | - |
canEvolve | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
MergeDeltaOrdered
Starts a fluent builder to merge a LazyFrame into a Delta Lake table with strict, order-preserving SQL MERGE semantics.
Unlike traditional merge methods, this builder guarantees that chained actions (Update, Delete, Insert) are evaluated exactly in the order they are defined. If no actions are specified before execution, it intelligently defaults to a standard Upsert (WhenMatchedUpdate + WhenNotMatchedInsert).
member MergeDeltaOrdered(
path: string,
mergeKeys: seq<string>,
?canEvolve: bool,
?cloudOptions: CloudOptions
) : DeltaMergeBuilderParameters
| Name | Type | Description |
|---|---|---|
path | string | The URI to the target Delta Lake table (local or cloud). |
mergeKeys | seq<string> | The column names to join on (must exist in both the Source DataFrame and Target Delta table). |
canEvolve | bool (optional) | If set to true, allows schema evolution (e.g., adding new columns from the Source to the Target). Default is false. |
cloudOptions | CloudOptions (optional) | Cloud storage credentials and configuration (e.g., AWS S3, Azure Blob). |
Returns: A DeltaMergeBuilder instance used to chain match conditions, culminating in a call to .Execute().
MergeSorted
Take two sorted DataFrames and merge them by the sorted key. The output of this operation will also be sorted. It is the callers responsibility that the frames are sorted in ascending order by the key, with null keys at the end, otherwise the order of the output will not make sense. The schemas of both LazyFrames must be equal.
member MergeSorted(other: LazyFrame, key: string, ?maintainOrder: bool) : LazyFrame
member MergeSorted(other: LazyFrame, key: string[], ?maintainOrder: bool) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
other | LazyFrame | Other DataFrame that must be merged |
key | string / string[] | Key that is sorted. |
maintainOrder | bool (optional) | If True, the output is guaranteed to have left-biased ordering for equal keys: rows from the left frame appear before rows from the right frame when their keys are equal. |
Min
Aggregate the columns in the Frame to their minimum value.
member Min : unit -> LazyFrameNullCount
Aggregate the columns in the Frame as the sum of their null value count.
member NullCount : unit -> LazyFrameOrderby
member Orderby(expr: Expr, desc: bool) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
expr | Expr | - |
desc | bool | - |
Pipe
Offers a structured way to apply a sequence of user-defined functions (UDFs).
member Pipe(func: (LazyFrame -> 'T)) : 'TParameters
| Name | Type | Description |
|---|---|---|
func | (LazyFrame -> 'T) | A function that receives the current expression and returns a value of type T. Typically this function wraps several Polars API calls that operate on the given expression. |
Returns: The result of applying function to this LazyFrame.
PipeWithSchema
Allows to alter the lazy frame during the plan stage with the resolved schema.
In contrast to pipe, this method does not execute function immediately but only during the plan stage. This allows to use the resolved schema of the input to dynamically alter the lazy frame. This also means that any exceptions raised by function will only be emitted during the plan stage.
member PipeWithSchema(func: (LazyFrame -> (PolarsSchema -> LazyFrame))) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
func | (LazyFrame -> (PolarsSchema -> LazyFrame)) | Callable; will receive the frame as the first parameter and the resolved schema as the second parameter. |
Pivot
Pivot the LazyFrame.
Important: Lazy pivot requires an eager onColumns DataFrame to determine the output schema (column names) during the planning phase.
Pivot the LazyFrame using column names.
Pivot the LazyFrame using column names and a custom aggregation expression.
member Pivot(
index: Selector,
columns: Selector,
values: Selector,
onColumns: DataFrame,
?aggregateExpr: Expr,
?aggregateFunction: PivotAgg,
?maintainOrder: bool,
?separator: string,
?columnNaming: PivotColumnNaming
) : LazyFrame
member Pivot(
index: seq<string>,
columns: seq<string>,
values: seq<string>,
onColumns: DataFrame,
?aggregateFunction: PivotAgg,
?maintainOrder: bool,
?separator: string,
?columnNaming: PivotColumnNaming
) : LazyFrame
member Pivot(
index: seq<string>,
columns: seq<string>,
values: seq<string>,
onColumns: DataFrame,
aggregateExpr: Expr,
?maintainOrder: bool,
?separator: string,
?columnNaming: PivotColumnNaming
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector / seq<string> | Selector for the index column(s) (the rows). |
columns | Selector / seq<string> | Selector for the column(s) to pivot (the new column headers). |
values | Selector / seq<string> | Selector for the value column(s) to populate the cells. |
onColumns | DataFrame | An Eager DataFrame containing the unique values of the columns. This is strictly used for schema inference. |
aggregateExpr | Expr (optional) | Optional expression to aggregate the values. If null, uses aggregateFunction. |
aggregateFunction | PivotAgg (optional) | Aggregation function to use if aggregateExpr is null. Default is First. |
maintainOrder | bool (optional) | Sort the result by the index column. |
separator | string (optional) | Separator used to combine column names when multiple value columns are selected. |
columnNaming | PivotColumnNaming (optional) | Controls how column names are constructed. Default is Auto. |
Returns: A new LazyFrame with the pivot operation applied.
PrintSchema
member PrintSchema : unit -> unitQuantile
Aggregate the columns in the Frame to their quantile value.
member Quantile(quantile: float, ?method: QuantileMethod) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
quantile | float | Quantile between 0.0 and 1.0. |
method | QuantileMethod (optional) | ['nearest’, ‘higher’, ‘lower’, ‘midpoint’, ‘linear’] Interpolation method. |
Remove
Remove rows that match the predicate. This is the exact opposite of Filter. Rows where the predicate evaluates to False or Null are kept.
member Remove(predicate: Expr) : LazyFrame
member Remove(predicate: Series) : LazyFrame
member Remove(mask: seq<bool>) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
predicate | Expr / Series | - |
mask | seq<bool> | - |
Rename
Rename the lazyframe columns using a sequence of (oldName, newName) tuples. Example: lf.Rename [ "colA", "col1"; "colB", "col2" ]
Rename the lazyframe columns using a dictionary mapping old column names to new column names. Example: lf.Rename (dict [ "colA", "col1"; "colB", "col2" ])
member Rename(mapping: seq<(string * string)>, ?strict: bool) : LazyFrame
member Rename(mapping: IReadOnlyDictionary<string, string>, ?strict: bool) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
mapping | seq<(string * string)> / IReadOnlyDictionary<string, string> | A sequence of key-value pairs representing (oldName, newName). |
strict | bool (optional) | If true, an error is raised if any column in mapping is not found in the schema. If false, columns that are not found are silently ignored. Default is true. |
Reverse
Reverse the LazyFrame.
member Reverse : unit -> LazyFrameRowsAsync
Streams LazyFrame rows asynchronously as strongly-typed F# Records/Tuples/DTOs. Serves as the asynchronous counterpart to DataFrame.Rows<'T>().
member RowsAsync(
?chunkSize: int64,
?engine: Engine,
?maintainOrder: bool,
?cancellationToken: CancellationToken
) : IAsyncEnumerable<'T>Parameters
| Name | Type | Description |
|---|---|---|
chunkSize | int64 (optional) | - |
engine | Engine (optional) | - |
maintainOrder | bool (optional) | - |
cancellationToken | CancellationToken (optional) | - |
Select
Select columns from the lazyframe using a sequence of column expressions.
Select columns from the lazyframe using a selector.
member Select(expr: Expr) : LazyFrame
member Select(selector: Selector) : LazyFrame
member Select(exprs: seq<Expr>) : LazyFrame
member Select(columns: seq<'a>) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
expr | Expr | - |
selector | Selector | - |
exprs | seq<Expr> | - |
columns | seq<'a> | - |
SetSorted
Mark one or multiple columns as sorted. This is an optimizer hint and will not actually execute a sorting operation.
member SetSorted(columns: Selector, ?descending: bool, ?nullsLast: bool) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
columns | Selector | The columns to mark as sorted. Accepts strings, arrays, or Selectors (e.g. Cs.Temporal()). |
descending | bool (optional) | Whether the columns are sorted in descending order. |
nullsLast | bool (optional) | Whether null values appear last. |
Returns: A new LazyFrame with the sorted hints applied to the query plan.
Show
Print the LazyFrame to Console (Stdout).
member Show : unit -> unitSinkBatches
Stream the query result in batches. This executes the query and calls 'onBatch' for each RecordBatch produced.
member SinkBatches(onBatch: Action<RecordBatch>) : unitParameters
| Name | Type | Description |
|---|---|---|
onBatch | Action<RecordBatch> | - |
SinkCatalogTable
Sinks the LazyFrame to a Unity Catalog table.
member SinkCatalogTable(
catalog: UnityCatalog,
catalogName: string,
schemaName: string,
tableName: string,
?partitionBy: Selector,
?mode: DeltaSaveMode,
?canEvolve: bool,
?includeKeys: bool,
?keysPreGrouped: bool,
?maxRowsPerFile: int,
?approxBytesPerFile: int64,
?compression: ParquetCompression,
?compressionLevel: int,
?statistics: bool,
?rowGroupSize: uint32,
?dataPageSize: uint32,
?compatLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
catalog | UnityCatalog | - |
catalogName | string | - |
schemaName | string | - |
tableName | string | - |
partitionBy | Selector (optional) | - |
mode | DeltaSaveMode (optional) | - |
canEvolve | bool (optional) | - |
includeKeys | bool (optional) | - |
keysPreGrouped | bool (optional) | - |
maxRowsPerFile | int (optional) | - |
approxBytesPerFile | int64 (optional) | - |
compression | ParquetCompression (optional) | - |
compressionLevel | int (optional) | - |
statistics | bool (optional) | - |
rowGroupSize | uint32 (optional) | - |
dataPageSize | uint32 (optional) | - |
compatLevel | int (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
SinkCsv
Execute the LazyFrame and sink the result to a CSV file.
This operation allows processing datasets larger than memory by streaming results directly to the file system.
member SinkCsv(
path: string,
?includeHeader: bool,
?includeBom: bool,
?separator: char,
?quoteChar: char,
?quoteStyle: QuoteStyle,
?nullValue: string,
?lineTerminator: string,
?floatScientific: bool,
?floatPrecision: int,
?decimalComma: bool,
?dateFormat: string,
?timeFormat: string,
?datetimeFormat: string,
?checkExtension: bool,
?compression: ExternalCompression,
?compressionLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?batchSize: int,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
includeHeader | bool (optional) | - |
includeBom | bool (optional) | - |
separator | char (optional) | - |
quoteChar | char (optional) | - |
quoteStyle | QuoteStyle (optional) | - |
nullValue | string (optional) | - |
lineTerminator | string (optional) | - |
floatScientific | bool (optional) | - |
floatPrecision | int (optional) | - |
decimalComma | bool (optional) | - |
dateFormat | string (optional) | - |
timeFormat | string (optional) | - |
datetimeFormat | string (optional) | - |
checkExtension | bool (optional) | - |
compression | ExternalCompression (optional) | - |
compressionLevel | int (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
batchSize | int (optional) | - |
cloudOptions | CloudOptions (optional) | - |
SinkCsvMemory
Sink the LazyFrame to a CSV format in memory.
This allows for streaming execution directly into a byte array without writing to disk.
member SinkCsvMemory(
?includeBom: bool,
?includeHeader: bool,
?batchSize: int,
?checkExtension: bool,
?compressionCode: ExternalCompression,
?compressionLevel: int,
?dateFormat: string,
?timeFormat: string,
?datetimeFormat: string,
?floatScientific: int,
?floatPrecision: int,
?decimalComma: bool,
?separator: byte,
?quoteChar: byte,
?nullValue: string,
?lineTerminator: string,
?quoteStyle: QuoteStyle,
?maintainOrder: bool
) : byte[]Parameters
| Name | Type | Description |
|---|---|---|
includeBom | bool (optional) | - |
includeHeader | bool (optional) | - |
batchSize | int (optional) | - |
checkExtension | bool (optional) | - |
compressionCode | ExternalCompression (optional) | - |
compressionLevel | int (optional) | - |
dateFormat | string (optional) | - |
timeFormat | string (optional) | - |
datetimeFormat | string (optional) | - |
floatScientific | int (optional) | - |
floatPrecision | int (optional) | - |
decimalComma | bool (optional) | - |
separator | byte (optional) | - |
quoteChar | byte (optional) | - |
nullValue | string (optional) | - |
lineTerminator | string (optional) | - |
quoteStyle | QuoteStyle (optional) | - |
maintainOrder | bool (optional) | - |
Returns: A byte array containing the serialized CSV data.
SinkCsvPartitioned
Execute the LazyFrame and sink the result to a CSV file, partitioned by the given selector.
member SinkCsvPartitioned(
path: string,
partitionBy: Selector,
?includeKeys: bool,
?keysPreGrouped: bool,
?maxRowsPerFile: int,
?approxBytesPerFile: int64,
?includeHeader: bool,
?includeBom: bool,
?separator: char,
?quoteChar: char,
?quoteStyle: QuoteStyle,
?nullValue: string,
?lineTerminator: string,
?floatScientific: bool,
?floatPrecision: int,
?decimalComma: bool,
?dateFormat: string,
?timeFormat: string,
?datetimeFormat: string,
?checkExtension: bool,
?compression: ExternalCompression,
?compressionLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?batchSize: int,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
partitionBy | Selector | - |
includeKeys | bool (optional) | - |
keysPreGrouped | bool (optional) | - |
maxRowsPerFile | int (optional) | - |
approxBytesPerFile | int64 (optional) | - |
includeHeader | bool (optional) | - |
includeBom | bool (optional) | - |
separator | char (optional) | - |
quoteChar | char (optional) | - |
quoteStyle | QuoteStyle (optional) | - |
nullValue | string (optional) | - |
lineTerminator | string (optional) | - |
floatScientific | bool (optional) | - |
floatPrecision | int (optional) | - |
decimalComma | bool (optional) | - |
dateFormat | string (optional) | - |
timeFormat | string (optional) | - |
datetimeFormat | string (optional) | - |
checkExtension | bool (optional) | - |
compression | ExternalCompression (optional) | - |
compressionLevel | int (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
batchSize | int (optional) | - |
cloudOptions | CloudOptions (optional) | - |
SinkDelta
Sink the LazyFrame to a Delta Lake table with partition discovery.
This operation performs a "blind write" of partitioned Parquet files (Hive-style) and then commits a transaction to the Delta Log, registering the new files.
member SinkDelta(
path: string,
?partitionBy: Selector,
?mode: DeltaSaveMode,
?canEvolve: bool,
?includeKeys: bool,
?keysPreGrouped: bool,
?maxRowsPerFile: int,
?approxBytesPerFile: int64,
?compression: ParquetCompression,
?compressionLevel: int,
?statistics: bool,
?rowGroupSize: uint32,
?dataPageSize: uint32,
?compatLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | Path to the root of the Delta Table. Can be local (e.g. "./data/table") or remote (e.g. "s3://bucket/table"). |
partitionBy | Selector (optional) | The selector(s) to partition the data by. Directories will be created in the format "col=value". |
mode | DeltaSaveMode (optional) | Save mode (Append, Overwrite, ErrorIfExists, Ignore). Default is Append. |
canEvolve | bool (optional) | - |
includeKeys | bool (optional) | Whether to include the partition keys in the Parquet files themselves. Default is true (recommended for Delta Lake compatibility). |
keysPreGrouped | bool (optional) | Assert that the keys are already pre-grouped. This can speed up the operation if true. |
maxRowsPerFile | int (optional) | Maximum number of rows per file. 0 means no limit. |
approxBytesPerFile | int64 (optional) | Approximate size in bytes per file. 0 means no limit. |
compression | ParquetCompression (optional) | Compression codec to use (Snappy, Zstd, etc.). |
compressionLevel | int (optional) | Compression level (depends on the codec). |
statistics | bool (optional) | Write statistics to the Parquet file. Delta Lake uses these stats for data skipping, so 'true' is highly recommended. |
rowGroupSize | uint32 (optional) | Target row group size (in rows). |
dataPageSize | uint32 (optional) | Target data page size (in bytes). |
compatLevel | int (optional) | IPC format compatibility. |
maintainOrder | bool (optional) | Maintain the order of the data. |
syncOnClose | SyncOnClose (optional) | Whether to sync the file to disk on close. |
mkdir | bool (optional) | Create parent directories if they don't exist. |
cloudOptions | CloudOptions (optional) | Options for cloud storage authentication and configuration. |
SinkIpc
Sink the LazyFrame to an IPC (Arrow) file.
This allows for streaming execution.
member SinkIpc(
path: string,
?compression: IpcCompression,
?compatLevel: int,
?recordBatchSize: int,
?recordBatchStatistics: bool,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | Output file path. |
compression | IpcCompression (optional) | Compression method (NoCompression, LZ4, ZSTD). Defaults to NoCompression. |
compatLevel | int (optional) | Arrow compatibility level. -1 means newest. Defaults to -1. |
recordBatchSize | int (optional) | Number of rows per record batch (0 = default). |
recordBatchStatistics | bool (optional) | Write statistics to the record batch header. Defaults to true. |
maintainOrder | bool (optional) | Whether to maintain the order of the data. Defaults to true. |
syncOnClose | SyncOnClose (optional) | File synchronization behavior on close. Defaults to None. |
mkdir | bool (optional) | Recursively create the directory if it does not exist. Defaults to false. |
cloudOptions | CloudOptions (optional) | Options for cloud storage. |
SinkIpcMemory
Sink the LazyFrame to an IPC (Arrow) format in memory.
This allows for streaming execution directly into a byte array without writing to disk.
member SinkIpcMemory(
?compression: IpcCompression,
?compatLevel: int,
?recordBatchSize: int,
?recordBatchStatistics: bool,
?maintainOrder: bool
) : byte[]Parameters
| Name | Type | Description |
|---|---|---|
compression | IpcCompression (optional) | - |
compatLevel | int (optional) | - |
recordBatchSize | int (optional) | - |
recordBatchStatistics | bool (optional) | - |
maintainOrder | bool (optional) | - |
SinkIpcPartitioned
Sink the LazyFrame to an IPC (Arrow) file, partitioned by the given selector.
member SinkIpcPartitioned(
path: string,
partitionBy: Selector,
?includeKeys: bool,
?keysPreGrouped: bool,
?maxRowsPerFile: int,
?approxBytesPerFile: int64,
?compression: IpcCompression,
?compatLevel: int,
?recordBatchSize: int,
?recordBatchStatistics: bool,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
partitionBy | Selector | The selector(s) to partition the data by. |
includeKeys | bool (optional) | Whether to include the partition keys in the output files. |
keysPreGrouped | bool (optional) | Assert that the keys are already pre-grouped. This can speed up the operation if true. Use with caution: if the data is not grouped, the output may be incorrect. |
maxRowsPerFile | int (optional) | Maximum number of rows per file. 0 means no limit. |
approxBytesPerFile | int64 (optional) | Approximate size in bytes per file. 0 means no limit. |
compression | IpcCompression (optional) | - |
compatLevel | int (optional) | - |
recordBatchSize | int (optional) | - |
recordBatchStatistics | bool (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
SinkJson
Sink the LazyFrame to a NDJSON (Newline Delimited JSON) file.
member SinkJson(
path: string,
?compression: ExternalCompression,
?compressionLevel: int,
?checkExtension: bool,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | Output file path. |
compression | ExternalCompression (optional) | Compression method (Gzip/Zstd). |
compressionLevel | int (optional) | Compression level. |
checkExtension | bool (optional) | Whether to check if the file extension matches '.json' or '.ndjson'. |
maintainOrder | bool (optional) | Maintain the order of data. |
syncOnClose | SyncOnClose (optional) | Sync to disk on close. |
mkdir | bool (optional) | Create parent directories. |
cloudOptions | CloudOptions (optional) | Cloud storage options. |
SinkJsonMemory
Sink the LazyFrame to a NDJSON (Newline Delimited JSON) format in memory.
member SinkJsonMemory(
?compression: ExternalCompression,
?compressionLevel: int,
?checkExtension: bool,
?maintainOrder: bool
) : byte[]Parameters
| Name | Type | Description |
|---|---|---|
compression | ExternalCompression (optional) | - |
compressionLevel | int (optional) | - |
checkExtension | bool (optional) | - |
maintainOrder | bool (optional) | - |
Returns: A byte array containing the serialized NDJSON data.
SinkJsonPartitioned
Sink the LazyFrame to a NDJSON (Newline Delimited JSON) file, partitioned by the given selector.
member SinkJsonPartitioned(
path: string,
partitionBy: Selector,
?includeKeys: bool,
?keysPreGrouped: bool,
?maxRowsPerFile: int,
?approxBytesPerFile: int64,
?compression: ExternalCompression,
?compressionLevel: int,
?checkExtension: bool,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
partitionBy | Selector | - |
includeKeys | bool (optional) | - |
keysPreGrouped | bool (optional) | - |
maxRowsPerFile | int (optional) | - |
approxBytesPerFile | int64 (optional) | - |
compression | ExternalCompression (optional) | - |
compressionLevel | int (optional) | - |
checkExtension | bool (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
SinkNdJson
Alias for SinkJson (Lazily evaluated JSON is always NDJSON/JsonLines).
member SinkNdJson(
path: string,
?compression: ExternalCompression,
?compressionLevel: int,
?checkExtension: bool,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
compression | ExternalCompression (optional) | - |
compressionLevel | int (optional) | - |
checkExtension | bool (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
SinkParquet
Sink the LazyFrame to a Parquet file.
This allows for streaming execution, processing the data in chunks and writing it to the file without loading the entire dataset into memory.
member SinkParquet(
path: string,
?compression: ParquetCompression,
?compressionLevel: int,
?statistics: bool,
?rowGroupSize: int,
?dataPageSize: int,
?compatLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | Path to the output file. |
compression | ParquetCompression (optional) | Compression codec to use. |
compressionLevel | int (optional) | Compression level (depends on the codec). |
statistics | bool (optional) | Write statistics to the parquet file. |
rowGroupSize | int (optional) | Target row group size (in rows). |
dataPageSize | int (optional) | Target data page size (in bytes). |
compatLevel | int (optional) | IPC format compatibility, -1: oldest, 0: default, 1: newest. |
maintainOrder | bool (optional) | Maintain the order of the data. |
syncOnClose | SyncOnClose (optional) | Whether to sync the file to disk on close. |
mkdir | bool (optional) | Create parent directories if they don't exist (Local file system only). |
cloudOptions | CloudOptions (optional) | Options for cloud storage (AWS S3, Azure Blob, GCS, etc.). |
SinkParquetMemory
Sink the LazyFrame to a Parquet format in memory.
This allows for streaming execution directly into a byte array without writing to disk.
member SinkParquetMemory(
?compression: ParquetCompression,
?compressionLevel: int,
?statistics: bool,
?rowGroupSize: int,
?dataPageSize: int,
?compatLevel: int,
?maintainOrder: bool
) : byte[]Parameters
| Name | Type | Description |
|---|---|---|
compression | ParquetCompression (optional) | - |
compressionLevel | int (optional) | - |
statistics | bool (optional) | - |
rowGroupSize | int (optional) | - |
dataPageSize | int (optional) | - |
compatLevel | int (optional) | - |
maintainOrder | bool (optional) | - |
Returns: A byte array containing the serialized Parquet data.
SinkParquetPartitioned
Sink the LazyFrame to a set of Parquet files, partitioned by the specified selector.
This writes the dataset to a directory, splitting the data into multiple files based on the partition key(s) defined in partitionBy.
member SinkParquetPartitioned(
path: string,
partitionBy: Selector,
?includeKeys: bool,
?keysPreGrouped: bool,
?maxRowsPerFile: int,
?approxBytesPerFile: int64,
?compression: ParquetCompression,
?compressionLevel: int,
?statistics: bool,
?rowGroupSize: int,
?dataPageSize: int,
?compatLevel: int,
?maintainOrder: bool,
?syncOnClose: SyncOnClose,
?mkdir: bool,
?cloudOptions: CloudOptions
) : unitParameters
| Name | Type | Description |
|---|---|---|
path | string | - |
partitionBy | Selector | - |
includeKeys | bool (optional) | - |
keysPreGrouped | bool (optional) | - |
maxRowsPerFile | int (optional) | - |
approxBytesPerFile | int64 (optional) | - |
compression | ParquetCompression (optional) | - |
compressionLevel | int (optional) | - |
statistics | bool (optional) | - |
rowGroupSize | int (optional) | - |
dataPageSize | int (optional) | - |
compatLevel | int (optional) | - |
maintainOrder | bool (optional) | - |
syncOnClose | SyncOnClose (optional) | - |
mkdir | bool (optional) | - |
cloudOptions | CloudOptions (optional) | - |
SinkTo
Stream query results directly to a database or other IDataReader consumer. Uses a producer-consumer pattern with bounded capacity for memory efficiency.
member SinkTo(writerAction: Action<IDataReader>, ?bufferSize: int, ?typeOverrides: IDictionary<string, Type>) : unitParameters
| Name | Type | Description |
|---|---|---|
writerAction | Action<IDataReader> | Callback to consume the IDataReader (e.g., using SqlBulkCopy). |
bufferSize | int (optional) | Max number of batches to buffer in memory (default: 5). |
typeOverrides | IDictionary<string, Type> (optional) | Force specific C# types for columns (e.g. map Date32 to DateTime). |
Slice
Slice the LazyFrame along the rows. If length is omitted, it slices to the end of the LazyFrame.
member Slice(offset: int64, ?length: uint32) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
offset | int64 | - |
length | uint32 (optional) | - |
Sort
member Sort(
columns: seq<IColumnExpr>,
descending: seq<bool>,
nullsLast: seq<bool>,
?maintainOrder: bool
) : LazyFrame
member Sort(
columns: seq<Expr>,
descending: seq<bool>,
nullsLast: seq<bool>,
?maintainOrder: bool
) : LazyFrame
member Sort(
columns: seq<IColumnExpr>,
?descending: bool,
?nullsLast: bool,
?maintainOrder: bool
) : LazyFrame
member Sort(
columns: seq<Expr>,
?descending: bool,
?nullsLast: bool,
?maintainOrder: bool
) : LazyFrame
member Sort(expr: Expr, ?descending: bool, ?nullsLast: bool) : LazyFrame
member Sort(colName: string, ?descending: bool, ?nullsLast: bool) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
columns | seq<IColumnExpr> / seq<Expr> | - |
descending | seq<bool> / bool (optional) | - |
nullsLast | seq<bool> / bool (optional) | - |
maintainOrder | bool (optional) | - |
expr | Expr | - |
colName | string | - |
Std
Aggregate the columns in the Frame to their standard deviation value.
member Std(?ddof: byte) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
ddof | byte (optional) | “Delta Degrees of Freedom”: the divisor used in the calculation is N - ddof, where N represents the number of elements. By default ddof is 1. |
Sum
Aggregate the columns in the Frame to their sum value.
member Sum : unit -> LazyFrameTail
member Tail(?n: uint32) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
n | uint32 (optional) | - |
ToAsyncEnumerable
Executes the LazyFrame via native streaming batch engine, yielding strongly typed F# Records/Tuples/DTOs with O(batch) memory overhead and pull-based backpressure.
member ToAsyncEnumerable(
?chunkSize: int64,
?engine: Engine,
?maintainOrder: bool,
?cancellationToken: CancellationToken
) : IAsyncEnumerable<'T>Parameters
| Name | Type | Description |
|---|---|---|
chunkSize | int64 (optional) | Maximum row count per native batch (0 uses Polars default batch size). |
engine | Engine (optional) | Execution engine selection. |
maintainOrder | bool (optional) | Whether to enforce deterministic row ordering across batches. |
cancellationToken | CancellationToken (optional) | Optional cooperative cancellation token. |
Returns: An asynchronous stream of hydrated F# row items.
ToHtml
Generates an HTML visualization of the LazyFrame execution plan and schema for LINQPad.
member ToHtml(?optimized: bool) : stringParameters
| Name | Type | Description |
|---|---|---|
optimized | bool (optional) | Whether to explain the optimized logical plan. |
Returns: An HTML representation formatted with monospace styling.
TopK
Get the top k rows based on the given columns. This is often faster than a full sort followed by a head.
member TopK(k: int, by: seq<'a>, ?reverse: seq<bool>) : LazyFrame
member TopK(k: int, by: seq<'a>, reverse: bool) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
k | int | Number of rows to return. |
by | seq<'a> | Columns to sort by. |
reverse | seq<bool> (optional) | Sort direction per column. Default is false (no reverse). |
Unique
Keep unique rows (stable) based on a subset of columns defined by a Selector.
Keep unique rows based on specific column names.
member Unique(?subset: Selector, ?keep: UniqueKeepStrategy, ?maintainOrder: bool) : LazyFrame
member Unique(columns: seq<string>, ?keep: UniqueKeepStrategy, ?maintainOrder: bool) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
subset | Selector (optional) | - |
keep | UniqueKeepStrategy (optional) | - |
maintainOrder | bool (optional) | - |
columns | seq<string> | - |
Unnest
Decompose a struct column into multiple columns.
Helper: Unnest columns by name.
Helper: Unnest a single column by name.
member Unnest(selector: Selector, ?separator: string) : LazyFrame
member Unnest(columns: seq<string>, ?separator: string) : LazyFrame
member Unnest(column: string, ?separator: string) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
selector | Selector | - |
separator | string (optional) | - |
columns | seq<string> | - |
column | string | - |
Unpivot
Unpivot (Melt) the LazyFrame using Selectors. Primary overload backed by native binding.
Unpivot (Melt) overload for simple string lists. Auto-converts to Selectors.
member Unpivot(
index: Selector,
on: Selector,
?variableName: string,
?valueName: string
) : LazyFrame
member Unpivot(
index: seq<string>,
on: seq<string>,
?variableName: string,
?valueName: string
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector / seq<string> | - |
on | Selector / seq<string> | - |
variableName | string (optional) | - |
valueName | string (optional) | - |
Update
member Update(
other: LazyFrame,
?on: IColumnExpr,
?how: JoinType,
?leftOn: IColumnExpr,
?rightOn: IColumnExpr,
?includeNulls: bool,
?maintainOrder: JoinMaintainOrder
) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
other | LazyFrame | - |
on | IColumnExpr (optional) | - |
how | JoinType (optional) | - |
leftOn | IColumnExpr (optional) | - |
rightOn | IColumnExpr (optional) | - |
includeNulls | bool (optional) | - |
maintainOrder | JoinMaintainOrder (optional) | - |
Var
Aggregate the columns in the Frame to their variance value.
member Var(?ddof: byte) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
ddof | byte (optional) | “Delta Degrees of Freedom”: the divisor used in the calculation is N - ddof, where N represents the number of elements. By default ddof is 1. |
WithColumns
Add or replace a single column in the LazyFrame.
Add or replace multiple columns in the LazyFrame.
Add or replace columns using generic column expressions (Expr or Selectors).
member WithColumns(expr: Expr) : LazyFrame
member WithColumns(exprs: seq<Expr>) : LazyFrame
member WithColumns(columns: seq<'a>) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
expr | Expr | The expression defining the new column. |
exprs | seq<Expr> | List of expressions defining the new columns. |
columns | seq<'a> | - |
WithRowIndex
Add a column at index 0 that counts the rows.
This can be useful to generate a unique index or to maintain order in operations that would otherwise discard it.
member WithRowIndex(?name: string, ?offset: int) : LazyFrameParameters
| Name | Type | Description |
|---|---|---|
name | string (optional) | The name of the new row index column. Defaults to "index". |
offset | int (optional) | The starting value of the row index. Defaults to 0. |
Returns: A new LazyFrame with the row index column added.
Pipeline Module Functions (LazyFrame.*)
LazyFrame.agg
Aggregate LazyGroupBy.
LazyFrame.agg : aggs: seq<Expr> -> builder: LazyGroupBy -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
aggs | seq<Expr> | - |
builder | LazyGroupBy | - |
LazyFrame.collect
Collect the LazyFrame using the default engine.
LazyFrame.collect : lazyFrame: LazyFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
lazyFrame | LazyFrame | - |
LazyFrame.collectAsync
Asynchronously execute the LazyFrame query plan. Useful for keeping UI responsive during heavy calculations.
LazyFrame.collectAsync : lf: LazyFrame -> FSharpAsync<DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
lf | LazyFrame | - |
LazyFrame.collectBatchesAsync
Streams materialized DataFrame batches asynchronously.
LazyFrame.collectBatchesAsync : chunkSize: int64 -> lf: LazyFrame -> IAsyncEnumerable<DataFrame>Parameters
| Name | Type | Description |
|---|---|---|
chunkSize | int64 | - |
lf | LazyFrame | - |
LazyFrame.collectSchema
Get the schema of the LazyFrame.
LazyFrame.collectSchema : lf: LazyFrame -> PolarsSchemaParameters
| Name | Type | Description |
|---|---|---|
lf | LazyFrame | - |
LazyFrame.collectWithEngine
Collect the LazyFrame using the specified engine.
LazyFrame.collectWithEngine : engine: Engine -> lazyFrame: LazyFrame -> DataFrameParameters
| Name | Type | Description |
|---|---|---|
engine | Engine | - |
lazyFrame | LazyFrame | - |
LazyFrame.concatDiagonal
Concatenate multiple LazyFrames diagonally.
LazyFrame.concatDiagonal : lfs: seq<LazyFrame> -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
lfs | seq<LazyFrame> | - |
LazyFrame.concatHorizontal
Concatenate multiple LazyFrames horizontally.
LazyFrame.concatHorizontal : lfs: seq<LazyFrame> -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
lfs | seq<LazyFrame> | - |
LazyFrame.concatVertical
Concatenate multiple LazyFrames vertically.
LazyFrame.concatVertical : lfs: seq<LazyFrame> -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
lfs | seq<LazyFrame> | - |
LazyFrame.drop
Drop columns from the LazyFrame.
LazyFrame.drop : columns: 'a -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
columns | 'a | - |
lf | LazyFrame | - |
LazyFrame.dropNans
Drop NaN values from the LazyFrame.
LazyFrame.dropNans : lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
lf | LazyFrame | - |
LazyFrame.dropNansOn
Drop NaN values from the LazyFrame with specific subset.
LazyFrame.dropNansOn : subset: 'a -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
subset | 'a | - |
lf | LazyFrame | - |
LazyFrame.dropNulls
Drop null values from the LazyFrame.
LazyFrame.dropNulls : lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
lf | LazyFrame | - |
LazyFrame.dropNullsOn
Drop null values from the LazyFrame with specific subset.
LazyFrame.dropNullsOn : subset: Expr -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
subset | Expr | - |
lf | LazyFrame | - |
LazyFrame.explain
Explain the execution plan of the LazyFrame.
LazyFrame.explain : lazyFrame: LazyFrame -> stringParameters
| Name | Type | Description |
|---|---|---|
lazyFrame | LazyFrame | - |
LazyFrame.explainUnoptimized
Explain the unoptimized LazyFrame execution plan.
LazyFrame.explainUnoptimized : lf: LazyFrame -> stringParameters
| Name | Type | Description |
|---|---|---|
lf | LazyFrame | - |
LazyFrame.fillNan
Fill NaN values in all columns with a specified value.
LazyFrame.fillNan : fillValue: Expr -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
fillValue | Expr | - |
lf | LazyFrame | - |
LazyFrame.fillNull
Fill null values in all columns with a specified value.
LazyFrame.fillNull : fillValue: Expr -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
fillValue | Expr | - |
lf | LazyFrame | - |
LazyFrame.filter
Filter rows based on a boolean expression.
LazyFrame.filter : expr: Expr -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
expr | Expr | - |
lf | LazyFrame | - |
LazyFrame.filterAll
Filter frame rows by combining multiple predicates with logical AND.
LazyFrame.filterAll : predicates: seq<Expr> -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
predicates | seq<Expr> | - |
lf | LazyFrame | - |
LazyFrame.groupBy
Group by keys.
LazyFrame.groupBy : keys: seq<Expr> -> lf: LazyFrame -> LazyGroupByParameters
| Name | Type | Description |
|---|---|---|
keys | seq<Expr> | - |
lf | LazyFrame | - |
LazyFrame.having
Having clause for LazyGroupBy.
LazyFrame.having : predicate: Expr -> builder: LazyGroupBy -> LazyGroupByParameters
| Name | Type | Description |
|---|---|---|
predicate | Expr | - |
builder | LazyGroupBy | - |
LazyFrame.head
Get the head of the LazyFrame.
LazyFrame.head : n: int -> lazyFrame: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
n | int | - |
lazyFrame | LazyFrame | - |
LazyFrame.join
Perform a join between two LazyFrames using join keys.
LazyFrame.join : other: LazyFrame -> leftOn: seq<Expr> -> rightOn: seq<Expr> -> how: JoinType -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
other | LazyFrame | - |
leftOn | seq<Expr> | - |
rightOn | seq<Expr> | - |
how | JoinType | - |
lf | LazyFrame | - |
LazyFrame.joinOn
Perform a join between two LazyFrames.
LazyFrame.joinOn : other: LazyFrame -> on: seq<Expr> -> how: JoinType -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
other | LazyFrame | - |
on | seq<Expr> | - |
how | JoinType | - |
lf | LazyFrame | - |
LazyFrame.melt
Alias for LazyFrame.unpivot.
LazyFrame.melt : index: Selector -> on: Selector -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector | - |
on | Selector | - |
lf | LazyFrame | - |
LazyFrame.meltNamed
Alias for LazyFrame.unpivotNamed.
LazyFrame.meltNamed : index: Selector -> on: Selector -> variableName: string -> valueName: string -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector | - |
on | Selector | - |
variableName | string | - |
valueName | string | - |
lf | LazyFrame | - |
LazyFrame.ofEager
Create a LazyFrame from DataFrame
LazyFrame.ofEager : df: DataFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
df | DataFrame | - |
LazyFrame.orderByAscending
Alias for sortAscending.
LazyFrame.orderByAscending : expr: seq<Expr> -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
expr | seq<Expr> | - |
lf | LazyFrame | - |
LazyFrame.orderByDescending
Alias for sortDescending.
LazyFrame.orderByDescending : expr: seq<Expr> -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
expr | seq<Expr> | - |
lf | LazyFrame | - |
LazyFrame.rename
Rename columns based on a mapping dictionary or key-value pairs.
LazyFrame.rename : mapping: seq<(string * string)> -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
mapping | seq<(string * string)> | - |
lf | LazyFrame | - |
LazyFrame.reverse
Reverse the order of the LazyFrame.
LazyFrame.reverse : lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
lf | LazyFrame | - |
LazyFrame.select
Select columns from LazyFrame.
LazyFrame.select : exprs: seq<'a> -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
exprs | seq<'a> | - |
lf | LazyFrame | - |
LazyFrame.slice
Slice rows starting at offset with length.
LazyFrame.slice : offset: int64 -> length: uint32 -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
offset | int64 | - |
length | uint32 | - |
lf | LazyFrame | - |
LazyFrame.sortAscending
Sort (Order By) the LazyFrame.
LazyFrame.sortAscending : exprs: seq<Expr> -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
exprs | seq<Expr> | - |
lf | LazyFrame | - |
LazyFrame.sortDescending
Sort (Order By) the LazyFrame in descending order.
LazyFrame.sortDescending : exprs: seq<Expr> -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
exprs | seq<Expr> | - |
lf | LazyFrame | - |
LazyFrame.tail
Get the tail of the LazyFrame.
LazyFrame.tail : n: uint32 -> lazyFrame: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
n | uint32 | - |
lazyFrame | LazyFrame | - |
LazyFrame.toAsyncEnumerable
Streams strongly typed F# Records/Tuples/DTOs asynchronously from a LazyFrame.
LazyFrame.toAsyncEnumerable : chunkSize: int64 -> lf: LazyFrame -> IAsyncEnumerable<'T>Parameters
| Name | Type | Description |
|---|---|---|
chunkSize | int64 | - |
lf | LazyFrame | - |
LazyFrame.toRowsAsync
Streams strongly typed F# Records/Tuples/DTOs asynchronously from a LazyFrame.
LazyFrame.toRowsAsync : chunkSize: int64 -> lf: LazyFrame -> IAsyncEnumerable<'T>Parameters
| Name | Type | Description |
|---|---|---|
chunkSize | int64 | - |
lf | LazyFrame | - |
LazyFrame.unnest
Decompose multiple struct columns.
LazyFrame.unnest : columns: 'a -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
columns | 'a | - |
lf | LazyFrame | - |
LazyFrame.unpivot
Unpivot (Melt) the LazyFrame from wide to long format using Selectors. Default column names ("variable", "value") will be used.
LazyFrame.unpivot : index: Selector -> on: Selector -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector | - |
on | Selector | - |
lf | LazyFrame | - |
LazyFrame.unpivotNamed
Unpivot (Melt) the LazyFrame using Selectors with custom variable and value column names.
LazyFrame.unpivotNamed : index: Selector -> on: Selector -> variableName: string -> valueName: string -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
index | Selector | - |
on | Selector | - |
variableName | string | - |
valueName | string | - |
lf | LazyFrame | - |
LazyFrame.withColumn
Add or replace columns in the LazyFrame.
LazyFrame.withColumn : expr: Expr -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
expr | Expr | - |
lf | LazyFrame | - |
LazyFrame.withColumns
Add or replace multiple columns in the LazyFrame.
LazyFrame.withColumns : exprs: seq<Expr> -> lf: LazyFrame -> LazyFrameParameters
| Name | Type | Description |
|---|---|---|
exprs | seq<Expr> | - |
lf | LazyFrame | - |