Skip to content

Conversation

@Cali0707
Copy link
Collaborator

Resolves #427

This PR adds support for a resources_scale tool, so that clients can get and update scale subresources.

@Cali0707 Cali0707 requested a review from manusa November 26, 2025 17:02
Copy link
Collaborator

@nader-ziada nader-ziada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good, one small question

@Cali0707 Cali0707 requested a review from nader-ziada November 26, 2025 18:10
@Cali0707 Cali0707 requested a review from nader-ziada November 27, 2025 19:25
@Cali0707 Cali0707 force-pushed the add-scale-resource-tool branch from 73c27c4 to a2cf933 Compare November 27, 2025 19:30
@nader-ziada
Copy link
Collaborator

/lgtm

@Cali0707
Copy link
Collaborator Author

Cali0707 commented Nov 28, 2025

/run-gevals - this won't work, my branch doesn't have evals/the make targets in it

Cali0707 and others added 3 commits December 1, 2025 10:04
@manusa manusa force-pushed the add-scale-resource-tool branch from a2cf933 to a8e3cc4 Compare December 1, 2025 09:04
@manusa
Copy link
Member

manusa commented Dec 1, 2025

/run-gevals

1 similar comment
@Cali0707
Copy link
Collaborator Author

Cali0707 commented Dec 1, 2025

/run-gevals

@github-actions
Copy link

github-actions bot commented Dec 1, 2025

Gevals MCP Evaluation Results

Summary: 0/24 tasks passed (0.0%)

Metric Result
Tasks Passed 0/24
Assertions Passed 24/72
Overall Failed

View full results

@manusa
Copy link
Member

manusa commented Dec 2, 2025

=== Error ===
failed to run agent: failed to run agent: failed to create chat completion: POST "https://api.openai.com/v1/chat/completions": 401 Unauthorized {
    "message": "You have insufficient permissions for this operation. Missing scopes: model.request. Check that you have the correct role in your organization (Reader, Writer, Owner) and project (Member, Owner), and if you're using a restricted API key, that it has the necessary scopes.",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }

Seems like the gevals results are due to inference provider auth issues

Copy link
Member

@manusa manusa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the gevals improvements (this is a great PR to test its usefulness)
One of the things I do think should be changed is the location of the resources_scale logic to have a clearer separation of concerns.

At the moment, the logic has been placed at the toolsets layer.

However, this layer should only be a wrapper to translate MCP requests into service requests and marshal the service responses appropriately into MCP responses.

Ideally, the scaling logic should be within the pkg/kubernetes module, I'd say that in the resources.go file.

@Cali0707
Copy link
Collaborator Author

Cali0707 commented Dec 2, 2025

/run-gevals

@Cali0707
Copy link
Collaborator Author

Cali0707 commented Dec 2, 2025

@manusa I think I fixed the permissions on the key

I also moved the scaling logic into pkg/kubernetes - lmk what you think!

@Cali0707
Copy link
Collaborator Author

Cali0707 commented Dec 2, 2025

/run-gevals

@github-actions
Copy link

github-actions bot commented Dec 2, 2025

Gevals MCP Evaluation Results

Summary: 21/24 tasks passed (87.5%)

Metric Result
Tasks Passed 21/24
Assertions Passed 70/72
Overall Passed

View full results

@Cali0707
Copy link
Collaborator Author

Cali0707 commented Dec 2, 2025

@manusa the specific scenarios we were targeting here all passed 🎉

Task: scale-deployment
  Difficulty: medium
  → Running agent...
  → Verifying results...
  ✓ Task passed

Task: scale-down-deployment
  Difficulty: medium
  → Running agent...
  → Verifying results...
  ✓ Task passed

... (omitted for brevity)

Task: statefulset-lifecycle
  Difficulty: hard
  → Running agent...
  → Verifying results...
  ✓ Task passed

On main the scale-* are still failing without this tool:

Task: scale-deployment
  Difficulty: medium
  → Running agent...
  → Verifying results...
  ✗ Task failed
    Error: verification script failed with output 'deployment.apps/web-app condition met
Verification failed for scale-deployment
': exit status 1

Task: scale-down-deployment
  Difficulty: medium
  → Running agent...
  → Verifying results...
  ✗ Task failed
    Error: verification script failed with output 'deployment.apps/web-service condition met
Verification failed for scale-down-deployment
': exit status 1

... (omitted for brevity)

Task: statefulset-lifecycle
  Difficulty: hard
  → Running agent...
  → Verifying results...
  ✓ Task passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add scale_resource tool

3 participants