Skip to content

[Inference Providers] feat. Refactor the NovitaTextToVideoTask using the async API. #1459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

viktor2077
Copy link
Contributor

This PR refactors the NovitaTextToVideoTask using the async API, aiming to allow inference with models that may take over 2 minutes to generate results.

Copy link
Contributor

@SBrandeis SBrandeis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - thank you for your contribution!

I have left a few comments and suggestions about typing and code style

Excited to see Novita's text-to-video service on the platform!

Comment on lines 67 to 75
override preparePayload(params: BodyParams): Record<string, unknown> {
const { num_inference_steps, ...restParameters } = params.args.parameters as Record<string, unknown>;
return {
...omit(params.args, ["inputs", "parameters"]),
...(params.args.parameters as Record<string, unknown>),
...restParameters,
steps: num_inference_steps,
prompt: params.args.inputs,
};
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for casting if you use the proper type here
Also, params.args.parameters can be undefined

Suggested change
override preparePayload(params: BodyParams): Record<string, unknown> {
const { num_inference_steps, ...restParameters } = params.args.parameters as Record<string, unknown>;
return {
...omit(params.args, ["inputs", "parameters"]),
...(params.args.parameters as Record<string, unknown>),
...restParameters,
steps: num_inference_steps,
prompt: params.args.inputs,
};
}
override preparePayload(params: BodyParams<TextToVideoArgs>): Record<string, unknown> {
const { num_inference_steps, ...restParameters } = params.args.parameters ?? {};
return {
...omit(params.args, ["inputs", "parameters"]),
...restParameters,
steps: num_inference_steps,
prompt: params.args.inputs,
};
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @SBrandeis, thanks for this suggestion. And I will encounter the following error if I don't cast the type explicitly:

Property 'num_inference_steps' does not exist on type '{}'.ts(2339)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and I'm refactoring the code to handle cases where params.args.parameters is null or undefined by using an empty object {} as the default value. This ensures the destructuring operation won't fail even if the parameters property doesn't exist:

const { num_inference_steps, ...restParameters } = (params.args.parameters as Record<string, unknown>) ?? {};

@viktor2077
Copy link
Contributor Author

I have left a few comments and suggestions about typing and code style

Thanks @SBrandeis , these suggestions are detailed and helpful!

@hanouticelina hanouticelina added the inference-providers integration of a new or existing Inference Provider label May 20, 2025
@viktor2077 viktor2077 requested a review from SBrandeis May 21, 2025 02:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
inference-providers integration of a new or existing Inference Provider
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants