Sampling approval editor

[nathanArseneau]

Is your feature request related to a problem? Please describe.
Currently, the sampling response is hardcoded and cannot be changed.

Describe the solution you'd like
The proposed solution is adding an editor so the user can send customized sampling-approved responses.

The editor should follow the specification

Describe alternatives you've considered
This is a new feature that will significantly improve sampling testing capabilities.

Additional context
Add any other context or screenshots about the feature request here.
Implementation-wise, I suggest having a split view to follow the existing layout of the other pages.

[nathanArseneau]

I can work on this; before I start, I would like to collect ideas on the UX part of the new editor.

[seuros]

I don't understand the goal here.

The inspect is a client, it supposed to receive the sampling from the server, not to emit them.

[olaservo]

@seuros @nathanArseneau I think the goal here is to follow the part of the spec that allows for the user to modify the sampling request, right?

[nathanArseneau]

@seuros, you are correct. The inspector received the sampling request from the server.

The goal is to do point five from the spec

Currently, it is hardcoded here

[olaservo]

Sorry @nathanArseneau I'm still a little confused on what you're proposing to add exactly. Do you mean that instead of returning a stubbed response, you want to add the ability to return a real sampling response?

[nathanArseneau]

yes this is my proposal

[cliffhall]

@nathanArseneau presumably the editor would be pre-populated with the existing stubbed response, but the user could edit that? So the extent of the UI would be a JSON editor and a "Send response" button added to the area outlined in red in your screenshot?

I think it's a good idea. I anticipate the desire for it this week in my own development adventures actually.

[nathanArseneau]

I thought of the JSON editor, but since we have a few fields that can be selected from a list, I ended up with a form.

model - string dropdown can propose some of the most popular models
stopReason - endTurn, stopSequence, maxTokens, or string
role - assistant or user
content.type - text or image
content.mimeType - string dropdown can propose some of the most popular

JSON is more practical to share, while the form is less error-prone when creating the payload.

[cliffhall]

I thought of the JSON editor, but since we have a few fields that can be selected from a list, I ended up with a form.

A form is good. A few questions come to mind:

• How is the model combo populated and are we taking on a maintenance burden keeping it up to date (i.e., should it just be a text field)?
• Does the Role field need to be editable? The expected response here is always from the Assistant, I think.
• Does selection in the Type field change the subsequent available fields (i.e., if you select type of Audio or Image, does the Text field get replaced by MIME type and data fields)?