Format of Uploaded Files

[maticzav]

Hi,

I couldn't find any docs anywhere telling what kind of file format cog (and Replicate API) expects to receive in the input of the prompt. The NodeJS example only includes ... and I don't if python adds any extra headers with open function.

const output = await replicate.run(
  "meronym/speaker-transcription:12483517b558629508e03bed77e9c46c6f6e5756715af2b3a4f741f70be45575",
  {
    input: {
      audio: "..."
    }
  }
);

output = replicate.run(
    "meronym/speaker-transcription:12483517b558629508e03bed77e9c46c6f6e5756715af2b3a4f741f70be45575",
    input={"audio": open("path/to/file", "rb")}
)

Could you help me figure out what's the expected way to upload files using this library?

[nelsonjchen]

I usually pop the web console on the UIs to see. It seems that the website has some sort of file upload functionality and running the prediction passes the uploaded file's URL to it. I'm doing images myself, but I'm guessing this is also the case for audio.

[nelsonjchen]

https://replicate.com/docs/reference/http#predictions.create:~:text=Files%20should%20be%20passed%20as%20data%20URLs%20or%20HTTPS%20URLs.

Ooh, data URLs.

[nelsonjchen]

Data URLs worked for my image case.

It's documented in the API, but it would be nice if the examples also reflected this for files.

[jmcannon]

For what it's worth, I've been banging my head against the wall for several hours trying to pass it file objects. This information should definitely be a lot more prominent. At very least, just put a dummy URL in the examples instead of an ellipsis.

[mattt]

@maticzav Thanks for raising this issue. I agree that this isn't sufficiently documented. The Node.js client is quite a bit newer than the Python client, so there's definitely some work to bring them into parity.

@jmcannon I'm sorry for how frustrating this has been. I'll make a point in the next few days to make it obvious and useful how to pass files as prediction inputs.

[mattt]

@maticzav @nelsonjchen @jmcannon Thanks for your patience on this. I just opened #63, which adds an example of how to pass a file input. Please take a look and let me know if you have any questions.

As I mentioned in that PR, we may want to add an affordance in the library to simplify this process. If you have any ideas for what that should look like, I'd love to hear them. Thanks!

[jmcannon]

Looks great - I think I'd also make it clear that you can just pass it an http url, as well. In my case, that saved an extra step and was super convenient. Maybe it's obvious to others, but I've been working with lots of streams and blobs lately, so it wasn't immediately obvious to me that I could provide a remote uri.