GH-48344: [Python] Fix Table.from_struct_array for empty ChunkedArray

Project: apache/arrow | Language: C++ | Concepts: error-handling, testing-strategy, api-design | Difficulty: mid
Link: https://github.com/apache/arrow/pull/49869

The problem

Calling Table.from_struct_array() with an empty ChunkedArray of struct type raised a confusing ValueError: Must pass schema, or at least one RecordBatch. This made it impossible to round-trip an empty table through to_struct_array() and back.

Why it's non-trivial

A naive fix might just try to construct a table from the empty ChunkedArray, but without any chunks, there is no schema or data to recover, and the API must still distinguish between valid empty struct arrays and invalid (non-struct) types. The fix must also avoid regressing error messages for invalid input types.

The approach

The main change is in the handling of empty struct ChunkedArray inputs in Table.from_struct_array. The code now ensures that even if there are zero chunks, an appropriate empty batch is created for schema recovery.

Key code changes:

-            return Table.from_batches([
-                RecordBatch.from_struct_array(chunk)
-                for chunk in struct_array.chunks
-            ])
+            chunks = struct_array.chunks or [struct_array.combine_chunks()]
+            return Table.from_batches(
+                [
+                    RecordBatch.from_struct_array(chunk)
+                    for chunk in chunks
+                ]
+            )

• What this does:
  • If struct_array.chunks is empty (i.e., the ChunkedArray has zero chunks), it falls back to [struct_array.combine_chunks()], which produces a single empty struct array with the correct type.
  • This ensures that the downstream call to RecordBatch.from_struct_array(chunk) always has at least one chunk to process, allowing schema recovery even for empty inputs.

Tests added:

def test_table_from_struct_array_for_empty_chunked_array():
    # GH-48344
    struct_type = pa.struct([("ints", pa.int32()), ("floats", pa.float32())])
    empty_chunked_struct_array = pa.chunked_array([], type=struct_type)
    result = pa.Table.from_struct_array(empty_chunked_struct_array)
    expected = pa.Table.from_arrays(
        [
            pa.array([], type=pa.int32()),
            pa.array([], type=pa.float32()),
        ], ["ints", "floats"]
    )
    assert result.equals(expected)
    assert result.schema == expected.schema

• What this does:
  • Verifies that passing an empty struct ChunkedArray to from_struct_array returns an empty table with the correct schema and no rows.

def test_table_from_struct_array_empty_chunked_array_invalid():
    with pytest.raises(TypeError, match="Argument 'struct_array' has incorrect type"):
        pa.Table.from_struct_array(pa.chunked_array([], type=pa.int64()))

• What this does:
  • Confirms that passing an empty non-struct ChunkedArray still raises a TypeError, preserving previous error behavior.

From the review thread

Copilot:

schema(struct_array.type.fields) will raise AttributeError for a zero-chunk ChunkedArray whose type isn't a struct (e.g., pa.chunked_array([], type=pa.int32())). Previously, non-struct chunked inputs would raise a TypeError from RecordBatch.from_struct_array; with zero chunks, this change can regress to an AttributeError with a less helpful message. Consider guarding the schema construction and raising a consistent TypeError when the input type isn't struct.

AlenkaF:

I think simply adding the GH issue number will do here.

Takeaways

• Always handle empty collections explicitly in API design, especially when schema or type information must be preserved.
• When changing error-handling logic, ensure that new code paths do not introduce less informative or inconsistent exceptions for invalid input types.
• Add regression tests for both the fixed and the invalid cases to prevent silent breakage in future refactoring.