Fix const casting in non-writable numpy-based tensors

[sgonorov]

Fix const casting in non-writable numpy-based tensors.

[Wovchena]

Was it broken? Should be reported as OV bug if so

[sgonorov]

No, we just needed tensor with non-const data for set_tensor, but we were receiving tensor with the const data. So i don't see it as broken in OV right now.

[Wovchena]

It's a shame that a copy is needed just to please the API. I view it as a problem. @praasz, can you take a look

[praasz]

I will check it

[praasz]

The data copy for inputs should be not required. It should be possible use Inputs tensor which are read-only.
The exception on infer() call is error on OV side, investigation in progress.

[AlexanderKalistratov]

The data copy for inputs should be not required. It should be possible use Inputs tensor which are read-only. The exception on infer() call is error on OV side, investigation in progress.

@praasz
FYI: unnecessary non-const access which I found while debugging this issue:
https://github.com/openvinotoolkit/openvino/blob/master/src/inference/src/dev/isync_infer_request.cpp#L289

    OPENVINO_ASSERT(
        std::dynamic_pointer_cast<ov::IRemoteTensor>(tensor._ptr) || tensor->data() != nullptr || is_dynamic,
        "Tensor data equal nullptr!");
}

Here: tensor->data()

Though there should be more places like this. Since fixing this one allow to move forward a bit, but didn't completely solve the issue

[sgonorov]

@AlexanderKalistratov Do we have an issue for that, so i can reference? Or should i create one?

[sgonorov]

@Wovchena do we still need this fix or it's better to wait for it to be resolved on OV side?

[Wovchena]

Wait for OV

[praasz]

The openvinotoolkit/openvino#33120 can be used for verification if that solve issue on GenAI side.

[Copilot]

Pull request overview

This PR fixes issues related to const casting in non-writable numpy-based tensors by creating writable copies of tensors before processing. The changes ensure that read-only numpy arrays (where writeable = False) can be safely used as inputs to various pipeline operations.

Key changes:

• Added defensive tensor copying in C++ code to handle read-only numpy tensors
• Added test coverage to verify read-only tensors work correctly across different pipeline types
• Modified tensor handling to use copy_to() instead of direct assignment or pointer sharing

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/python_tests/test_vlm_pipeline.py	Added test case for VLM pipeline with read-only image tensors
tests/python_tests/test_llm_pipeline_static.py	Added test case for static LLM pipeline with read-only input tensors
tests/python_tests/test_llm_pipeline.py	Added test case for LLM pipeline with read-only input tensors
src/cpp/src/visual_language/phi4mm/classes.cpp	Creates writable copy of image tensor before processing
src/cpp/src/visual_language/inputs_embedder.cpp	Uses memcpy to create independent tensor copies instead of sharing pointers
src/cpp/src/speculative_decoding/speculative_decoding_stateful.cpp	Creates writable copies of input_ids and attention_mask tensors
src/cpp/src/llm/pipeline_static.cpp	Creates writable copies of input_ids and attention_mask tensors
src/cpp/src/image_generation/image_processor.cpp	Creates writable copy of image tensor before processing

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.