Skip to content

feat: dot-separated field access to struct columns #11796

@baumgold

Description

@baumgold

Is your feature request related to a problem?

Not a problem, per se.

What is the motivation behind your request?

Struct columns currently allow field access in two ways: __getitem__ or __getattr__. For example:

>>> import ibis
... ibis.options.interactive = True
... t = ibis.memtable(
...     {
...         "pos": [
...             {"lat": 10.1, "lon": 30.3},
...             {"lat": 10.2, "lon": 30.2},
...             {"lat": 10.3, "lon": 30.1},
...         ]
...     }
... )
>>> t.pos.lat
┏━━━━━━━━━┓
┃ lat     ┃
┡━━━━━━━━━┩
│ float64 │
├─────────┤
│    10.1 │
│    10.2 │
│    10.3 │
└─────────┘
>>> t["pos"]["lat"]
┏━━━━━━━━━┓
┃ lat     ┃
┡━━━━━━━━━┩
│ float64 │
├─────────┤
│    10.1 │
│    10.2 │
│    10.3 │
└─────────┘

Unfortunately this only allows accessing a single level at once. Sometimes we may want to access multiple levels deep using a dot-delineated string, which is currently unsupported:

>>> t["pos.lat"]
Traceback (most recent call last):
  File "<python-input-5>", line 1, in <module>
    t["pos.lat"]
    ~^^^^^^^^^^^
  File "~/.venv/lib/python3.13/site-packages/ibis/expr/types/relations.py", line 1068, in __getitem__
    return ops.Field(self.op(), what).to_expr()
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "~/.venv/lib/python3.13/site-packages/ibis/common/bases.py", line 72, in __call__
    return cls.__create__(*args, **kwargs)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "~/.venv/lib/python3.13/site-packages/ibis/common/grounds.py", line 120, in __create__
    return super().__create__(**kwargs)
           ~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "~s/.venv/lib/python3.13/site-packages/ibis/expr/operations/relations.py", line 96, in __init__
    raise IbisTypeError(
    ...<2 lines>...
    )
ibis.common.exceptions.IbisTypeError: Column 'pos.lat' is not found in table. Existing columns: 'pos'.

Describe the solution you'd like

Enhance StructColumn.__getitem__ (link) to support dot-delineated column strings. In order to continue supporting columns with names that contain dots, this logic should only kick in if the requested string cannot be found in the StructColumn's names property.

What version of ibis are you running?

11.0.0

What backend(s) are you using, if any?

Trino

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureFeatures or general enhancements

    Type

    No type

    Projects

    Status

    backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions