Skip to content

py::stride::compute does not accept signed type #1390

@sizmailov

Description

@sizmailov

py::stride used to have size_t to store start/stop/step values. Nowadays it uses ssize_t (issue #776, PR #782).

The issues is that py::slice::compute still accepts size_t* arguments instead of ssize_t*

bool compute(size_t length, size_t *start, size_t *stop, size_t *step,
size_t *slicelength) const {
return PySlice_GetIndicesEx((PYBIND11_SLICE_OBJECT *) m_ptr,
(ssize_t) length, (ssize_t *) start,
(ssize_t *) stop, (ssize_t *) step,
(ssize_t *) slicelength) == 0;
}

C-style pointers casts must be read here as reinterpret_cast<>.

To obtain signed strides user should have something like

ssize_t start, stop, step, slicelength;
if (!slice.compute(
            N,
            reinterpret_cast<size_t*>(&start),
            reinterpret_cast<size_t*>(&stop),
            reinterpret_cast<size_t*>(&step),
            reinterpret_cast<size_t*>(&slicelength)
)){
          throw py::error_already_set();
}
... // use extracted values

So reinterpret_cast<> appears twice: in user code and in the library (in py::slice::compute) converting back and forth ssize_t* and size_t*.

The non-breaking solution would be to add an overload for ssize_t* to py::slice.

bool compute(ssize_t length, ssize_t *start, ssize_t *stop, ssize_t *step, 
              ssize_t *slicelength) const { 
     return PySlice_GetIndicesEx((PYBIND11_SLICE_OBJECT *) m_ptr, 
                                 length, start, 
                                 stop,  step, 
                                 slicelength) == 0; 
 } 

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions