Providing an invalid webhook URL to `predictions.create` doesn't return any feedback

[carlpeaslee]

How to reproduce:

  await replicate.predictions.create({
    version: "6282abe6a492de4145d7bb601023762212f9ddbbe78278bd6771c8b3b2f2a13b", //vicuna
    input: {
      prompt: "this is definitely developer error  but it would be a nice quality of life improvement to surface the mistake",
    },
    webhook: `htp\\someinvalid,url.corn/webhawks`,
    webhook_events_filter: ['completed', 'logs', 'output', 'start'],
  })

The prediction will successfully run and can be verified in one's dashboard – it just never surfaces any errors or notifications that it couldn't deliver the webhook event.

I think in an ideal world predictions.create would probably:

• error when provided a structurally invalid URL
• still function if given a structurally valid but possibly non-responsive endpoint –– but then surface an error/warning somewhere for the user (either at an errors event endpoint or in the dashboard)

Not urgent just a nice to have that I ran in to today :)

[mattt]

Thanks for sharing this, @carlpeaslee. I think it's totally reasonable for the client library to help guard against this.

What do you think about updating the type definition for webhook parameters to explicitly look for things matching a URL pattern? I don't want to, like, enforce passing URL objects, and attempting to validate inputs at runtime feels wrong (that's more the API's job), but letting TypeScript warn at at build time feels helpful and unobtrusive.

[carlpeaslee]

So I'll give you the actual real example that I experienced:

Which I don't think would get caught by a typescript definition?

I think you're right that this is probably more the API's job, I just couldn't find a repo for your API to leave an issue on :)

(definitely agree that URL objects would be a hassle)

[mattt]

@carlpeaslee Ah, thanks for providing that example. In that case, a runtime check using the built-in URL constructor makes sense.

I think you're right that this is probably more the API's job, I just couldn't find a repo for your API to leave an issue on :)

Oh yeah — sorry to imply that this wasn't the right place to file an issue. Our API isn't open source, so this is as good a place as any. 😄

[mattt]

@carlpeaslee Thanks again for your help with this! I just merged #84, which implements what we discussed. If you update to v0.12.3, you'll get an error the next time you pass an invalid URL.