`DecimalArray` Logical and Physical type mismatch

[connortsui20]

We have this documentation on DecimalArray:

/// These are just the maximal ranges for each scalar type, but it is perfectly legal to store
/// values with precision that does not match this exactly. For example, a valid DecimalArray with
/// precision=39 may store its values in an `i8` if all of the actual values fit into it.
///
/// Similarly, a `DecimalArray` can be built that stores a set of precision=2 values in a
/// `Buffer<i256>`.

This presents a problem (which is the cause of many of the recent fuzzer crashes): If we try to patch values or use fill_null with a decimal value that is valid according to the logical DType, but is not compatible with the storage DType, then we just panic!

This is highly concerning! The physical type has leaked and now governs what we are allowed to do instead of the logical type.

I see 3 ways forward:

1. Keep this behavior. I think this is a bad idea, as it has completely different behavior from how
   PrimitiveArray works with BitPackedArray, for example.
2. Reallocate the entire array whenever we run into this so that the new storage type can fit the new values.
   I also think this is a bad idea in terms of performance.
3. We instead require that the storage type must always be able to fit any value in the domain of the logical
   DecimalDType.

For 3, by requiring this constraint we no longer need to worry about checking every single value during validation, we can just do a single check in the constructor that the DType fits the storage.

---

We are about to do a huge migration to the new Vortex operator world (where everything is lazily computed / executed), which means we don't necessarily need to do this migration in the old world: We can just make sure we implement the correct behavior in the new world.

In the new world, we need to make sure that the DecimalVector the array tree executes into holds a DVector that can store every possible value given by the DecmialDType's PrecisionScale.

In practice, this will mean finding the smallest integer storage type for the given PrecisionScale. Note we don't necessarily need to use the minimal storage type, though that will be the most helpful for performance.

The more important thing is that we do not allow executing a DecimalArray into a DecimalVector that cannot fit all possible values according to the PrecisionScale.

And also note that we need to handle old files that we were written in the old world with this bad logic.

---

Fuzzer Crashes

• Fuzzing Crash: Sum operation returns null for decimal256 array #5590
• Fuzzing Crash: fill_null on DecimalArray fails when fill value has null decimal_value #5744
• Fuzzing Crash: DecimalArray fill_null with null fill value #5806
• Fuzzing Crash: DecimalArray fill_null panics on null scalar value #5807
• Fuzzing Crash: Sum operation returns null for valid DecimalArray ChunkedArray #5810
• Fuzzing Crash: Decimal Sum Overflow in ChunkedArray #5811

[github-actions]

Related Crash Detected

A crash in the array_ops target has been detected that is related to the DecimalArray logical/physical type mismatch issue.

Crash Details

Crash Location: vortex-array/src/arrays/decimal/compute/fill_null.rs:37

Error Message:

fill value does not fit in array's decimal storage type

Stack Trace:

   6: fill_null
             at ./vortex-array/src/arrays/decimal/compute/fill_null.rs:37:26
   7: invoke<vortex_array::arrays::decimal::vtable::DecimalVTable>
             at ./vortex-array/src/compute/fill_null.rs:92:13
   8: invoke
             at ./vortex-array/src/compute/fill_null.rs:122:42
   9: invoke
             at ./vortex-array/src/compute/mod.rs:149:34
  10: fill_null
             at ./vortex-array/src/compute/fill_null.rs:54:10

Similarities to Original Issue:

• Same root cause: physical storage type (I32) cannot fit values that are valid according to the logical type (Decimal with precision=5, scale=-75)
• Same file and function as other related crashes (Fuzzing Crash: fill_null on DecimalArray fails when fill value has null decimal_value #5744, Fuzzing Crash: DecimalArray fill_null with null fill value #5806, Fuzzing Crash: DecimalArray fill_null panics on null scalar value #5807)
• Demonstrates the core problem: physical type has leaked into operations instead of being governed by logical type

Differences:

• Different line number (37 vs 35 in related issues)
• Different error message: "fill value does not fit" (line 37) vs "ensure non-null fill value" (line 35)
• This crash is at the .cast::<T>() call that checks if the fill value fits the storage type
• The fuzzer generated an unusual decimal with negative scale (-75), which creates edge cases

Analysis

The fuzzer created a DecimalArray with:

• Logical type: Decimal(precision: 5, scale: -75, Nullable)
• Physical storage: I32 (values_type)
• Operation: Take (creates nulls) → FillNull

When fill_null attempts to cast the fill value to fit the I32 storage type, it fails because the decimal value doesn't fit in 32 bits. This is exactly the scenario described in the meta-issue: the physical storage type restricts what operations can be performed, even though the logical type would allow it.

Debug Output

FuzzArrayAction {
    array: DecimalArray {
        dtype: Decimal(
            DecimalDType {
                precision: 5,
                scale: -75,
            },
            Nullable,
        ),
        values: Buffer<u8> {
            length: 4,
            alignment: Alignment(
                4,
            ),
            as_slice: [31, 205, 255, 255],
        },
        values_type: I32,
        validity: AllValid,
        stats_set: ArrayStats {
            inner: RwLock {
                data: StatsSet {
                    values: [],
                },
            },
        },
    },
    actions: [
        (
            Take(
                BitPackedArray {
                    offset: 0,
                    len: 43,
                    dtype: Primitive(
                        U64,
                        Nullable,
                    ),
                    bit_width: 0,
                    packed: Buffer<u8> {
                        length: 0,
                        alignment: Alignment(
                            8,
                        ),
                        as_slice: [],
                    },
                    patches: None,
                    validity: Array(
                        BoolArray {
                            dtype: Bool(
                                NonNullable,
                            ),
                            ...
                        },
                    ),
                    ...
                },
            ),
            ...
        ),
    ],
}

Occurrence Details

• Crash File: crash-c546ae23ec9699279b0e13f2a8f6d69ab20895cf
• Branch: develop
• Commit: 5d50f72
• Crash Artifact: https://github.com/vortex-data/vortex/actions/runs/20543753894/artifacts/4976160209

Reproduction

cargo +nightly fuzz run -D --sanitizer=none array_ops array_ops/crash-c546ae23ec9699279b0e13f2a8f6d69ab20895cf -- -rss_limit_mb=0

---

Auto-detected by fuzzing workflow with Claude analysis

[github-actions]

Related Crash Detected

A crash demonstrating the same root cause was detected in the array_ops target.

Crash Details

Crash Location: vortex-array/src/arrays/decimal/compute/fill_null.rs:37 in the fill_null method

Error Message:

fill value does not fit in array's decimal storage type

Stack Trace:

   6: fill_null
             at ./vortex-array/src/arrays/decimal/compute/fill_null.rs:37:26
   7: invoke<vortex_array::arrays::decimal::vtable::DecimalVTable>
             at ./vortex-array/src/compute/fill_null.rs:92:13
   8: invoke
             at ./vortex-array/src/compute/fill_null.rs:122:42
   9: invoke
             at ./vortex-array/src/compute/mod.rs:149:34
  10: fill_null
             at ./vortex-array/src/compute/fill_null.rs:54:10
  11: run_fuzz_action
             at ./fuzz/src/array/mod.rs:608:33

Similarities to Original Issue:

• Same root cause: DecimalArray logical/physical type mismatch
• Fill value is valid for the logical DecimalDType (precision=39, scale=-53)
• Fill value is I256: -998892667993999963691389465156852180088
• Storage type is also I256, but the cast fails (likely because the value doesn't fit within the precision constraints)
• This is another manifestation of the problem where physical storage type constraints override logical type constraints

Differences:

• This crash happens during the cast operation (line 37) rather than during null value extraction
• The panic message is "fill value does not fit in array's decimal storage type" instead of a null-related error
• However, the root cause is identical: the DecimalArray allows physical/logical type mismatches that cause runtime failures

Debug Output

FuzzArrayAction {
    array: DecimalArray {
        dtype: Decimal(
            DecimalDType {
                precision: 39,
                scale: -53,
            },
            Nullable,
        ),
        values: Buffer<u8> {
            length: 96,
            alignment: Alignment(
                16,
            ),
            as_slice: [44, 1, 52, 255, 127, 159, 156, 85, 141, 146, 73, 163, 37, 123, 121, 84, ...],
        },
        values_type: I256,
        validity: AllValid,
        ...
    },
    actions: [
        ...
        (
            FillNull(
                Scalar {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        NonNullable,
                    ),
                    value: ScalarValue(
                        Decimal(
                            I256(
                                i256(
                                    -998892667993999963691389465156852180088,
                                ),
                            ),
                        ),
                    ),
                },
            ),
            Array(
                DecimalArray {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        NonNullable,
                    ),
                    values: Buffer<u8> {
                        length: 11264,
                        alignment: Alignment(
                            16,
                        ),
                        as_slice: [136, 27, 163, 163, 35, 49, 112, 118, 17, 2, 107, 82, 27, 69, 132, 16, ...],
                    },
                    values_type: I256,
                    validity: NonNullable,
                    ...
                },
            ),
        ),
    ],
}

Occurrence Details

• Crash File: crash-171607a69c9e74234069fa4aaef5b4b31bb9e986
• Branch: develop
• Commit: d67c02c
• Crash Artifact: https://github.com/vortex-data/vortex/actions/runs/20588762356/artifacts/4988953500

Reproduction

cargo +nightly fuzz run -D --sanitizer=none array_ops array_ops/crash-171607a69c9e74234069fa4aaef5b4b31bb9e986 -- -rss_limit_mb=0

---

Auto-detected by fuzzing workflow with Claude analysis

[github-actions]

Related Crash Detected

A crash in the same area was detected in the array_ops target, hitting the exact problem described in this issue.

Crash Details

Crash Location: vortex-array/src/arrays/decimal/compute/fill_null.rs:37 (in the fill_null method)

Error Message:

fill value does not fit in array's decimal storage type

Stack Trace:

   6: fill_null
             at ./vortex-array/src/arrays/decimal/compute/fill_null.rs:37:26
   7: invoke<vortex_array::arrays::decimal::vtable::DecimalVTable>
             at ./vortex-array/src/compute/fill_null.rs:92:13
   8: invoke
             at ./vortex-array/src/compute/fill_null.rs:122:42
   9: invoke
             at ./vortex-array/src/compute/mod.rs:149:34
  10: fill_null
             at ./vortex-array/src/compute/fill_null.rs:54:10
  11: run_fuzz_action
             at ./fuzz/src/array/mod.rs:608:33

Similarities to Original Issue:

• Same root cause: logical dtype vs storage dtype mismatch
• Same file and function: fill_null in vortex-array/src/arrays/decimal/compute/fill_null.rs
• This crash occurs at line 37 (the .cast::<T>() call), demonstrating the exact problem described in this issue

Differences:

• Previous crashes (Fuzzing Crash: fill_null on DecimalArray fails when fill value has null decimal_value #5744, Fuzzing Crash: DecimalArray fill_null with null fill value #5806, Fuzzing Crash: DecimalArray fill_null panics on null scalar value #5807) hit line 34-35 with "top-level fill_null ensure non-null fill value" (null scalar issue)
• This crash hits line 37 with "fill value does not fit in array's decimal storage type" - the cast fails because the fill value is valid for the logical dtype (precision=39, scale=-53) but cannot fit in the physical storage type (I128)

Root Cause Analysis:
This is a textbook example of the issue described in this tracking issue. The fuzzer created:

• A DecimalArray with logical dtype Decimal(precision=39, scale=-53, Nullable)
• Physical storage: I128 (128-bit integers)
• Fill value: i256(-432775966371519351014919162030228426715) - valid for precision=39 but way too large for I128

When fill_null tries to cast this I256 value to I128 at line 36-37:

let fill_value = decimal_value
    .cast::<T>()
    .vortex_expect("fill value does not fit in array's decimal storage type");

The cast fails because the value doesn't fit in I128, causing the panic. The fill value is perfectly valid according to the logical DecimalDType(precision=39, scale=-53), but the physical storage type leaked and prevents the operation from succeeding.

Debug Output

FuzzArrayAction {
    array: DecimalArray {
        dtype: Decimal(
            DecimalDType {
                precision: 39,
                scale: -53,
            },
            Nullable,
        ),
        values: Buffer<u8> {
            length: 16,
            alignment: Alignment(
                16,
            ),
            as_slice: [117, 73, 58, 73, 73, 228, 216, 221, 140, 146, 73, 163, 37, 123, 121, 84, ...],
        },
        values_type: I128,
        validity: AllValid,
        stats_set: ArrayStats {
            inner: RwLock {
                data: StatsSet {
                    values: [
                        (
                            Sum,
                            Exact(
                                ScalarValue(
                                    Primitive(
                                        U64(
                                            3,
                                        ),
                                    ),
                                ),
                            ),
                        ),
                    ],
                },
            },
        },
    },
    actions: [
        ...
        (
            FillNull(
                Scalar {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        NonNullable,
                    ),
                    value: ScalarValue(
                        Decimal(
                            I256(
                                i256(
                                    -432775966371519351014919162030228426715,
                                ),
                            ),
                        ),
                    ),
                },
            ),
            Array(
                DecimalArray {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        NonNullable,
                    ),
                    values: Buffer<u8> {
                        length: 2176,
                        alignment: Alignment(
                            16,
                        ),
                        as_slice: [37, 48, 93, 93, 168, 36, 211, 55, 16, 30, 114, 144, 118, 97, 106, 186, ...],
                    },
                    values_type: I256,
                    validity: NonNullable,
                    stats_set: ArrayStats {
                        inner: RwLock {
                            data: StatsSet {
                                values: [],
                            },
                        },
                    },
                },
            ),
        ),
    ],
}

Note: The array has values_type: I128 but the fill value is I256 with a value too large for I128.

Occurrence Details

• Crash File: crash-902f777d8391195a1b7fd705ca359a3c01180850
• Branch: develop
• Commit: da5d463da6f95e83a5a68e8ac2d0ee3bd05ee06e
• Crash Artifact: https://github.com/vortex-data/vortex/actions/runs/20475321695/artifacts/4969738698

Reproduction

cargo +nightly fuzz run -D --sanitizer=none array_ops array_ops/crash-902f777d8391195a1b7fd705ca359a3c01180850 -- -rss_limit_mb=0

---

Auto-detected by fuzzing workflow with Claude analysis

[github-actions]

Related Crash Detected

A similar crash was detected in the array_ops target, confirming the logical/physical type mismatch issue.

Crash Details

Crash Location: vortex-array/src/arrays/decimal/compute/fill_null.rs:37 in the fill_null method

Error Message:

panicked at vortex-error/src/lib.rs:348:13:
fill value does not fit in array's decimal storage type

Stack Trace:

#0  fill_null at ./vortex-array/src/arrays/decimal/compute/fill_null.rs:37:26
#1  invoke<vortex_array::arrays::decimal::vtable::DecimalVTable> at ./vortex-array/src/compute/fill_null.rs:92:13
#2  invoke at ./vortex-array/src/compute/fill_null.rs:122:42
#3  invoke at ./vortex-array/src/compute/mod.rs:149:34
#4  fill_null at ./vortex-array/src/compute/fill_null.rs:54:10
#5  run_fuzz_action at ./fuzz/src/array/mod.rs:608:33
#6  __libfuzzer_sys_run at ./fuzz/fuzz_targets/array_ops.rs:14:11

Similarities to Original Issue:

• Same root cause: fill value is valid for the logical DecimalDType (precision=39, scale=-53) but doesn't fit in the physical storage type (I256)
• Same function and area: DecimalArray fill_null operation
• Same fundamental problem: physical type constraints leak into logical operations

Differences:

• This crash occurs at line 37 during the .cast::<T>() operation
• Error message is "fill value does not fit in array's decimal storage type" (vs other variants like null fill values in Fuzzing Crash: DecimalArray fill_null panics on null scalar value #5807)
• Storage type is I256, precision=39, scale=-53

Debug Output

FuzzArrayAction {
    array: DecimalArray {
        dtype: Decimal(
            DecimalDType {
                precision: 39,
                scale: -53,
            },
            Nullable,
        ),
        values: Buffer<u8> {
            length: 96,
            alignment: Alignment(
                16,
            ),
            as_slice: [43, 1, 0, 0, 166, 153, 168, 147, 67, 73, 0, 90, 221, 53, 195, 167, ...],
        },
        values_type: I256,
        validity: AllValid,
        stats_set: ArrayStats {
            inner: RwLock {
                data: StatsSet {
                    values: [],
                },
            },
        },
    },
    actions: [
        (
            Take(
                SparseArray {
                    patches: Patches {
                        array_len: 31,
                        offset: 0,
                        indices: PrimitiveArray {
                            dtype: Primitive(
                                U8,
                                NonNullable,
                            ),
                            buffer: Buffer<u8> {
                                length: 3,
                                alignment: Alignment(
                                    1,
                                ),
                                as_slice: [5, 10, 27],
                            },
                            ...
                        },
                        ...
                    },
                    ...
                },
            ),
            FillNull(
                Scalar {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        Nullable,
                    ),
                    value: ScalarValue(
                        Decimal(
                            I128(
                                299,
                            ),
                        ),
                    ),
                },
            ),
        ),
    ],
}

Occurrence Details

• Crash File: crash-ba75ebd2aa27c5d8e6c90168824676be9962fbdf
• Branch: develop
• Commit: da5d463
• Crash Artifact: https://github.com/vortex-data/vortex/actions/runs/12617975893/artifacts/2436988311

Reproduction

cargo +nightly fuzz run -D --sanitizer=none array_ops array_ops/crash-ba75ebd2aa27c5d8e6c90168824676be9962fbdf -- -rss_limit_mb=0

---

Auto-detected by fuzzing workflow with Claude analysis

[github-actions]

Related Crash Detected

A similar crash was detected in the array_ops target.

Crash Details

Crash Location: vortex-array/src/arrays/decimal/compute/fill_null.rs:37 (in fill_null method)

Error Message:

fill value does not fit in array's decimal storage type

Stack Trace:

   6: fill_null
             at ./vortex-array/src/arrays/decimal/compute/fill_null.rs:37:26
   7: invoke<vortex_array::arrays::decimal::vtable::DecimalVTable>
             at ./vortex-array/src/compute/fill_null.rs:92:13
   8: invoke
             at ./vortex-array/src/compute/fill_null.rs:122:42
   9: invoke
             at ./vortex-array/src/compute/mod.rs:149:34
  10: fill_null
             at ./vortex-array/src/compute/fill_null.rs:54:10
  11: run_fuzz_action
             at ./fuzz/src/array/mod.rs:608:33

Similarities to Original Issue:

• Same root cause: logical vs physical type mismatch in DecimalArray
• The fill value (i256(-998892667993999963691389465156852180088)) is valid for the logical dtype (Decimal(precision: 39, scale: -53))
• However, it cannot fit in the physical storage type being used by the array
• This is exactly the problem described in the issue: "If we try to patch values or use fill_null with a decimal value that is valid according to the logical DType, but is not compatible with the storage DType, then we just panic!"

Differences:

• This crash occurs at line 37 (the cast operation) rather than line 35 (null value check)
• The panic message is specifically "fill value does not fit in array's decimal storage type"
• Note: These differences indicate the same root cause manifesting at a different validation point

Debug Output

FuzzArrayAction {
    array: DecimalArray {
        dtype: Decimal(
            DecimalDType {
                precision: 39,
                scale: -53,
            },
            Nullable,
        ),
        values: Buffer<u8> {
            length: 96,
            alignment: Alignment(
                16,
            ),
            as_slice: [44, 1, 52, 255, 127, 159, 156, 85, 141, 146, 73, 163, 37, 123, 121, 84, ...],
        },
        values_type: I256,
        validity: AllValid,
        stats_set: ArrayStats {
            inner: RwLock {
                data: StatsSet {
                    values: [],
                },
            },
        },
    },
    actions: [
        (
            Take(
                SparseArray {
                    patches: Patches {
                        array_len: 315,
                        offset: 0,
                        indices: PrimitiveArray {
                            dtype: Primitive(
                                U16,
                                NonNullable,
                            ),
                            buffer: Buffer<u8> {
                                length: 6,
                                alignment: Alignment(
                                    2,
                                ),
                                as_slice: [5, 0, 237, 0, 12, 1],
                            },
                            ...
                        },
                        values: BitPackedArray {
                            offset: 0,
                            len: 3,
                            dtype: Primitive(
                                U64,
                                Nullable,
                            ),
                            bit_width: 0,
                            ...
                        },
                        ...
                    },
                    fill_value: Scalar {
                        dtype: Primitive(
                            U64,
                            Nullable,
                        ),
                        value: ScalarValue(
                            Null,
                        ),
                    },
                    ...
                },
            ),
            Array(
                DecimalArray {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        Nullable,
                    ),
                    values: Buffer<u8> {
                        length: 10080,
                        alignment: Alignment(
                            16,
                        ),
                        as_slice: [44, 1, 52, 255, 127, 159, 156, 85, 141, 146, 73, 163, 37, 123, 121, 84, ...],
                    },
                    values_type: I256,
                    validity: Array(
                        BoolArray {
                            dtype: Bool(
                                NonNullable,
                            ),
                            bits: BitBuffer {
                                buffer: Buffer<u8> {
                                    length: 40,
                                    alignment: Alignment(
                                        1,
                                    ),
                                    as_slice: [32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
                                },
                                offset: 0,
                                len: 315,
                            },
                            validity: NonNullable,
                            ...
                        },
                    ),
                    ...
                },
            ),
        ),
        (
            Compress(
                Default,
            ),
            Array(
                DecimalArray {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        Nullable,
                    ),
                    values: Buffer<u8> {
                        length: 10080,
                        alignment: Alignment(
                            16,
                        ),
                        as_slice: [44, 1, 52, 255, 127, 159, 156, 85, 141, 146, 73, 163, 37, 123, 121, 84, ...],
                    },
                    values_type: I256,
                    ...
                },
            ),
        ),
        (
            FillNull(
                Scalar {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        NonNullable,
                    ),
                    value: ScalarValue(
                        Decimal(
                            I256(
                                i256(
                                    -998892667993999963691389465156852180088,
                                ),
                            ),
                        ),
                    ),
                },
            ),
            Array(
                DecimalArray {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        NonNullable,
                    ),
                    values: Buffer<u8> {
                        length: 10080,
                        alignment: Alignment(
                            16,
                        ),
                        as_slice: [136, 27, 163, 163, 35, 49, 112, 118, 17, 2, 107, 82, 27, 69, 132, 16, ...],
                    },
                    values_type: I256,
                    validity: NonNullable,
                    ...
                },
            ),
        ),
    ],
}

Occurrence Details

• Crash File: crash-8d887f927311b1ba403abe5e2f1dd74b98948981
• Branch: develop
• Commit: da5d463
• Crash Artifact: https://github.com/vortex-data/vortex/actions/runs/12621035768/artifacts/3093419090

Reproduction

cargo +nightly fuzz run -D --sanitizer=none array_ops array_ops/crash-8d887f927311b1ba403abe5e2f1dd74b98948981 -- -rss_limit_mb=0

---

Auto-detected by fuzzing workflow with Claude analysis

[github-actions]

Related Crash Detected

A crash was detected in the array_ops target that demonstrates the exact problem described in this issue.

Crash Details

Crash Location: vortex-array/src/arrays/decimal/compute/fill_null.rs:37 (the .cast::<T>() call)

Error Message:

fill value does not fit in array's decimal storage type

Stack Trace:

#6  fill_null at ./vortex-array/src/arrays/decimal/compute/fill_null.rs:37:26
#7  invoke<vortex_array::arrays::decimal::vtable::DecimalVTable> at ./vortex-array/src/compute/fill_null.rs:92:13
#8  invoke at ./vortex-array/src/compute/fill_null.rs:122:42
#9  invoke at ./vortex-array/src/compute/mod.rs:149:34
#10 fill_null at ./vortex-array/src/compute/fill_null.rs:54:10
#11 run_fuzz_action at ./fuzz/src/array/mod.rs:608:33

Similarities to Original Issue:

• Same root cause: physical storage type constraints leaking through to logical operations
• Occurs during fill_null operation when fill value is valid for logical type but cannot fit in physical storage
• Array has Decimal(precision=39, scale=-53) with I256 storage
• Fill value is -998892667993999963691389465156852180088 (valid I256, valid for the precision/scale)
• The .cast::<T>() call at line 37 fails because the value doesn't fit the storage type

Differences:

• This crash occurs at line 37 (the cast operation), not line 35 (the null check)
• Error message is about type fitting, not null values
• Note: These differences indicate this is the type mismatch variant of the DecimalArray fill_null issue family

Debug Output

FuzzArrayAction {
    array: DecimalArray {
        dtype: Decimal(
            DecimalDType {
                precision: 39,
                scale: -53,
            },
            Nullable,
        ),
        values: Buffer<u8> {
            length: 96,
            alignment: Alignment(
                16,
            ),
            as_slice: [44, 1, 52, 255, 127, 159, 156, 85, 141, 146, 73, 163, 37, 123, 121, 84, ...],
        },
        values_type: I256,
        validity: AllValid,
        stats_set: ArrayStats {
            inner: RwLock {
                data: StatsSet {
                    values: [],
                },
            },
        },
    },
    actions: [
        (
            Take(
                RunEndArray {
                    ends: PrimitiveArray {
                        dtype: Primitive(
                            U16,
                            NonNullable,
                        ),
                        buffer: Buffer<u8> {
                            length: 18,
                            alignment: Alignment(
                                2,
                            ),
                            as_slice: [5, 0, 6, 0, 230, 0, 231, 0, 237, 0, 238, 0, 98, 1, 99, 1, ...],
                        },
                        validity: NonNullable,
                        stats_set: ArrayStats {
                            inner: RwLock {
                                data: StatsSet {
                                    values: [
                                        (
                                            NullCount,
                                            Exact(
                                                ScalarValue(
                                                    Primitive(
                                                        U64(
                                                            0,
                                                        ),
                                                    ),
                                                ),
                                            ),
                                        ),
                                        (
                                            Min,
                                            Exact(
                                                ScalarValue(
                                                    Primitive(
                                                        U16(
                                                            5,
                                                        ),
                                                    ),
                                                ),
                                            ),
                                        ),
                                        (
                                            Max,
                                            Exact(
                                                ScalarValue(
                                                    Primitive(
                                                        U16(
                                                            401,
                                                        ),
                                                    ),
                                                ),
                                            ),
                                        ),
                                    ],
                                },
                            },
                        },
                    },
                    values: BitPackedArray {
                        offset: 0,
                        len: 9,
                        dtype: Primitive(
                            U64,
                            Nullable,
                        ),
                        bit_width: 0,
                        packed: Buffer<u8> {
                            length: 0,
                            alignment: Alignment(
                                8,
                            ),
                            as_slice: [],
                        },
                        patches: None,
                        validity: Array(
                            BoolArray {
                                dtype: Bool(
                                    NonNullable,
                                ),
                                bits: BitBuffer {
                                    buffer: Buffer<u8> {
                                        length: 2,
                                        alignment: Alignment(
                                            1,
                                        ),
                                        as_slice: [170, 0],
                                    },
                                    offset: 0,
                                    len: 9,
                                },
                                validity: NonNullable,
                                stats_set: ArrayStats {
                                    inner: RwLock {
                                        data: StatsSet {
                                            values: [
                                                (
                                                    Sum,
                                                    Exact(
                                                        ScalarValue(
                                                            Primitive(
                                                                U64(
                                                                    4,
                                                                ),
                                                            ),
                                                        ),
                                                    ),
                                                ),
                                            ],
                                        },
                                    },
                                },
                            },
                        ),
                        stats_set: ArrayStats {
                            inner: RwLock {
                                data: StatsSet {
                                    values: [],
                                },
                            },
                        },
                    },
                    offset: 0,
                    length: 401,
                    stats_set: ArrayStats {
                        inner: RwLock {
                            data: StatsSet {
                                values: [],
                            },
                        },
                    },
                },
            ),
            Array(
                DecimalArray {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        Nullable,
                    ),
                    values: Buffer<u8> {
                        length: 12832,
                        alignment: Alignment(
                            16,
                        ),
                        as_slice: [44, 1, 52, 255, 127, 159, 156, 85, 141, 146, 73, 163, 37, 123, 121, 84, ...],
                    },
                    values_type: I256,
                    validity: Array(
                        BoolArray {
                            dtype: Bool(
                                NonNullable,
                            ),
                            bits: BitBuffer {
                                buffer: Buffer<u8> {
                                    length: 51,
                                    alignment: Alignment(
                                        1,
                                    ),
                                    as_slice: [32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
                                },
                                offset: 0,
                                len: 401,
                            },
                            validity: NonNullable,
                            stats_set: ArrayStats {
                                inner: RwLock {
                                    data: StatsSet {
                                        values: [],
                                    },
                                },
                            },
                        },
                    ),
                    stats_set: ArrayStats {
                        inner: RwLock {
                            data: StatsSet {
                                values: [],
                            },
                        },
                    },
                },
            ),
        ),
        (
            Compress(
                Default,
            ),
            Array(
                DecimalArray {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        Nullable,
                    ),
                    values: Buffer<u8> {
                        length: 12832,
                        alignment: Alignment(
                            16,
                        ),
                        as_slice: [44, 1, 52, 255, 127, 159, 156, 85, 141, 146, 73, 163, 37, 123, 121, 84, ...],
                    },
                    values_type: I256,
                    validity: Array(
                        BoolArray {
                            dtype: Bool(
                                NonNullable,
                            ),
                            bits: BitBuffer {
                                buffer: Buffer<u8> {
                                    length: 51,
                                    alignment: Alignment(
                                        1,
                                    ),
                                    as_slice: [32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
                                },
                                offset: 0,
                                len: 401,
                            },
                            validity: NonNullable,
                            stats_set: ArrayStats {
                                inner: RwLock {
                                    data: StatsSet {
                                        values: [],
                                    },
                                },
                            },
                        },
                    ),
                    stats_set: ArrayStats {
                        inner: RwLock {
                            data: StatsSet {
                                values: [],
                            },
                        },
                    },
                },
            ),
        ),
        (
            FillNull(
                Scalar {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        NonNullable,
                    ),
                    value: ScalarValue(
                        Decimal(
                            I256(
                                i256(
                                    -998892667993999963691389465156852180088,
                                ),
                            ),
                        ),
                    ),
                },
            ),
            Array(
                DecimalArray {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        Nullable,
                    ),
                    values: Buffer<u8> {
                        length: 12832,
                        alignment: Alignment(
                            16,
                        ),
                        as_slice: [44, 1, 52, 255, 127, 159, 156, 85, 141, 146, 73, 163, 37, 123, 121, 84, ...],
                    },
                    values_type: I256,
                    validity: Array(
                        BoolArray {
                            dtype: Bool(
                                NonNullable,
                            ),
                            bits: BitBuffer {
                                buffer: Buffer<u8> {
                                    length: 51,
                                    alignment: Alignment(
                                        1,
                                    ),
                                    as_slice: [32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
                                },
                                offset: 0,
                                len: 401,
                            },
                            validity: NonNullable,
                            stats_set: ArrayStats {
                                inner: RwLock {
                                    data: StatsSet {
                                        values: [],
                                    },
                                },
                            },
                        },
                    ),
                    stats_set: ArrayStats {
                        inner: RwLock {
                            data: StatsSet {
                                values: [],
                            },
                        },
                    },
                },
            ),
        ),
    ],
}

Occurrence Details

• Crash File: crash-a56d7a90bf79f2cf42185e145d2400f08c192616
• Branch: develop
• Commit: da5d463
• Crash Artifact: https://github.com/vortex-data/vortex/actions/runs/20481698623/artifacts/4966094092

Reproduction

cargo +nightly fuzz run -D --sanitizer=none array_ops array_ops/crash-a56d7a90bf79f2cf42185e145d2400f08c192616 -- -rss_limit_mb=0

---

Auto-detected by fuzzing workflow with Claude analysis

[github-actions]

Related Crash Detected

A crash related to the DecimalArray logical/physical type mismatch was detected in the array_ops target.

Crash Details

Crash Location: vortex-array/src/arrays/decimal/compute/fill_null.rs:37:fill_null

Error Message:

fill value does not fit in array's decimal storage type

Stack Trace:

   6: fill_null
             at ./vortex-array/src/arrays/decimal/compute/fill_null.rs:37:26
   7: invoke<vortex_array::arrays::decimal::vtable::DecimalVTable>
             at ./vortex-array/src/compute/fill_null.rs:92:13
   8: invoke
             at ./vortex-array/src/compute/fill_null.rs:122:42
   9: invoke
             at ./vortex-array/src/compute/mod.rs:149:34
  10: fill_null
             at ./vortex-array/src/compute/fill_null.rs:54:10
  11: run_fuzz_action
             at ./fuzz/src/array/mod.rs:608:33

Similarities to Original Issue:

• Same root cause: physical storage type constraints leak into operations that should be governed by logical type
• Same file: vortex-array/src/arrays/decimal/compute/fill_null.rs
• Same operation: fill_null on DecimalArray

Differences:

• Different error line: Line 37 (cast failure) vs line 35 (null value extraction) in other related crashes
• This crash occurs at the cast step where a valid I256 decimal value cannot be cast to the array's storage type
• Error message: "fill value does not fit in array's decimal storage type" (new variant)
• Array details:
  • DecimalArray with precision=39, scale=-53, Nullable
  • Storage type: I256
  • Fill value: Decimal(I256(-998892667993999963691389465156852180088), precision=39, scale=-53, NonNullable)

The crash demonstrates the exact problem described in this issue: a decimal value that is valid for the logical DecimalDType(precision=39, scale=-53) fails because it cannot be cast to the physical storage type (I256), even though both the array and fill value use I256 storage.

Debug Output

FuzzArrayAction {
    array: DecimalArray {
        dtype: Decimal(
            DecimalDType {
                precision: 39,
                scale: -53,
            },
            Nullable,
        ),
        values: Buffer<u8> {
            length: 96,
            alignment: Alignment(
                16,
            ),
            as_slice: [44, 1, 52, 255, 127, 159, 156, 85, 141, 146, 73, 163, 37, 123, 121, 84, ...],
        },
        values_type: I256,
        validity: AllValid,
        stats_set: ArrayStats {
            inner: RwLock {
                data: StatsSet {
                    values: [],
                },
            },
        },
    },
    actions: [
        (
            Take(
                SparseArray {
                    patches: Patches {
                        array_len: 331,
                        offset: 0,
                        indices: PrimitiveArray { dtype: Primitive(U16, NonNullable), ... },
                        values: BitPackedArray { len: 3, dtype: Primitive(U64, Nullable), bit_width: 0, ... },
                    },
                    fill_value: Scalar {
                        dtype: Primitive(U64, Nullable),
                        value: ScalarValue(Null),
                    },
                    ...
                },
            ),
            Array(DecimalArray { dtype: Decimal(precision: 39, scale: -53, Nullable), values_type: I256, ... }),
        ),
        (
            FillNull(
                Scalar {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 39,
                            scale: -53,
                        },
                        NonNullable,
                    ),
                    value: ScalarValue(
                        Decimal(
                            I256(
                                i256(
                                    -998892667993999963691389465156852180088,
                                ),
                            ),
                        ),
                    ),
                },
            ),
            Array(DecimalArray { dtype: Decimal(precision: 39, scale: -53, Nullable), values_type: I256, ... }),
        ),
    ],
}

Occurrence Details

• Crash File: crash-644904bf1248368678f3177f920982cbfcc06a53
• Branch: develop
• Commit: da5d463
• Crash Artifact: https://github.com/vortex-data/vortex/actions/runs//artifacts/

Reproduction

cargo +nightly fuzz run -D --sanitizer=none array_ops array_ops/crash-644904bf1248368678f3177f920982cbfcc06a53 -- -rss_limit_mb=0

---

Auto-detected by fuzzing workflow with Claude analysis

[github-actions]

Related Crash Detected

A similar crash was detected in the array_ops target during fuzzing.

Crash Details

Crash Location: vortex-array/src/arrays/decimal/compute/fill_null.rs:37 in the fill_null method

Error Message:

panicked at vortex-error/src/lib.rs:348:13:
fill value does not fit in array's decimal storage type

Stack Trace:

#6  fill_null at ./vortex-array/src/arrays/decimal/compute/fill_null.rs:37:26
#7  invoke<vortex_array::arrays::decimal::vtable::DecimalVTable> at ./vortex-array/src/compute/fill_null.rs:92:13
#8  invoke at ./vortex-array/src/compute/fill_null.rs:122:42
#9  invoke at ./vortex-array/src/compute/mod.rs:149:34
#10 fill_null at ./vortex-array/src/compute/fill_null.rs:54:10
#11 invoke at ./vortex-array/src/compute/fill_null.rs:133:23
#12 invoke at ./vortex-array/src/compute/mod.rs:149:34
#13 fill_null at ./vortex-array/src/compute/fill_null.rs:54:10
#14 run_fuzz_action at ./fuzz/src/array/mod.rs:608:33

Similarities to Original Issue:

• Same root cause: fill_null operation with a decimal value that is valid for the logical DType (precision=19, scale=-61) but doesn't fit in the physical storage type (I128)
• Same file and function: vortex-array/src/arrays/decimal/compute/fill_null.rs
• This is exactly the logical vs physical type mismatch described in this issue

Differences:

• Specific decimal values differ (as expected from fuzzing)
• The crash happens at line 37 during the cast::<T>() operation
• The fuzzer produced a DecimalArray with I128 storage but a fill value that overflows I128

Debug Output

FuzzArrayAction {
    array: DecimalArray {
        dtype: Decimal(
            DecimalDType {
                precision: 19,
                scale: -61,
            },
            NonNullable,
        ),
        values: Buffer<u8> {
            length: 16,
            alignment: Alignment(
                16,
            ),
            as_slice: [44, 59, 83, 23, 252, 220, 7, 68, 0, 0, 0, 0, 0, 0, 0, 0],
        },
        values_type: I128,
        validity: NonNullable,
        stats_set: ArrayStats {
            inner: RwLock {
                data: StatsSet {
                    values: [],
                },
            },
        },
    },
    actions: [
        // ... (various operations leading to fill_null)
        (
            FillNull(
                Scalar {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 19,
                            scale: -61,
                        },
                        NonNullable,
                    ),
                    value: ScalarValue(
                        Decimal(
                            I128(
                                -9999999999987983606,
                            ),
                        ),
                    ),
                },
            ),
            Array(
                DecimalArray {
                    dtype: Decimal(
                        DecimalDType {
                            precision: 19,
                            scale: -61,
                        },
                        NonNullable,
                    ),
                    values: Buffer<u8> {
                        length: 80,
                        alignment: Alignment(
                            16,
                        ),
                        as_slice: [44, 59, 83, 23, 252, 220, 7, 68, 0, 0, 0, 0, 0, 0, 0, 0, ...],
                    },
                    values_type: I128,
                    validity: NonNullable,
                    stats_set: ArrayStats {
                        inner: RwLock {
                            data: StatsSet {
                                values: [],
                            },
                        },
                    },
                },
            ),
        ),
    ],
}

Occurrence Details

• Crash File: crash-331954b6e083e13d8a8c31672a3adb35b05ef7c1
• Branch: develop
• Commit: da5d463
• Crash Artifact: https://nightly.link/vortex-data/vortex/actions/runs/12611766265/array_ops_crash_331954b6e083e13d8a8c31672a3adb35b05ef7c1.zip

Reproduction

cargo +nightly fuzz run -D --sanitizer=none array_ops array_ops/crash-331954b6e083e13d8a8c31672a3adb35b05ef7c1 -- -rss_limit_mb=0

---

Auto-detected by fuzzing workflow with Claude analysis