shift

shift(expr, offset=1, *, fill_expr)

A variant of pl.Expr.shift() that allows filling shifted values using another Polars expression.

When abs(offset) exceeds the total number of rows

Since expressions are evaluated lazily at runtime, their validity cannot be verified during construction. If abs(offset) equals or exceeds the total row count, the result may behave like a full-column replacement using fill_expr=.

Parameters

expr : pl.Expr: A single Polars expression to shift.
offset : int = 1: The number of rows to shift. It must be a non-zero integer. A positive value shifts the column downward (forward), while a negative value shifts it upward (backward).
fill_expr : pl.Expr: Expression used to fill the shifted positions.

Returns

: pl.Expr: A Polars expression with shifted values and custom fill logic.

Examples

DataFrame Context

Shift values downward by 2:

import polars as pl
import turtle_island as ti

pl.Config.set_fmt_table_cell_list_len(10)
df = pl.DataFrame({"x": [1, 2, 3, 4], "y": [5, 6, 7, 8]})
df.with_columns(
    ti.shift(pl.col("x"), 2, fill_expr=pl.col("y")).alias("shifted")
)

shape: (4, 3)

x	y	shifted
i64	i64	i64
1	5	5
2	6	6
3	7	1
4	8	2

Shift values upward by 3:

df.with_columns(
    ti.shift(pl.col("x"), -3, fill_expr=pl.col("y")).alias("shifted")
)

shape: (4, 3)

x	y	shifted
i64	i64	i64
1	5	4
2	6	6
3	7	7
4	8	8

List Namespace Context

Working with Lists as Series

In the list namespace, it may be easier to think of each row as an element in a list. Conceptually, you’re working with a pl.Series, where each row corresponds to one item in the list.

Shift values downward by 2:

df2 = pl.DataFrame(
    {
        "x": [[1, 2, 3, 4], [5, 6, 7, 8]],
        "y": [[9, 10, 11, 12], [13, 14, 15, 16]],
    }
)
df2.with_columns(
    pl.all().list.eval(
        ti.shift(pl.element(), 2, fill_expr=pl.element().add(10))
    )
)

shape: (2, 2)

x	y
list[i64]	list[i64]
[11, 12, 1, 2]	[19, 20, 9, 10]
[15, 16, 5, 6]	[23, 24, 13, 14]