Skip to content

Change from error to warning when moving an fp16 pipeline to CPU #763

@apolinario

Description

@apolinario

6c64741 introduced an error moving an fp16 pipeline to CPU, which makes sense as it cannot run on CPU; however I would suggest changing it to a warning as it broke my use-case that I think is a legitimate one

This is the use-case that broke:

  • Load a bunch of different pipelines running different models on CPU (RAM), but never actually run them for inference on CPU
  • Whenever I want to run them for inference, I move them from GPU -> CPU which is an operation that is way faster than if I had to load them from disco every
  • Once the model is ran, the pipeline with the model loaded is sent back to CPU (RAM) to be idle but warm

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions