[Feature] GEPA: Add custom component selection logic support

[andressrg]

Summary

This PR implements custom component selection logic for GEPA as requested in issue #8755. It allows users to provide custom strategies for selecting which components (predictors) to optimize at each iteration, enabling more flexible optimization patterns.

Key Features

• Custom Component Selection: New component_selector parameter accepts any object implementing the ReflectionComponentSelector protocol
• Backward Compatibility: Default behavior unchanged - uses RoundRobinReflectionComponentSelector when no custom selector provided
• Multiple Selection Support: Custom selectors can return single or multiple components for simultaneous optimization
• LLM-Driven Selection: Enables context-aware component selection based on optimization state and trajectories
• Full Program Adapter Support: Allows updating all component prompts simultaneously, inspired by DSPy full program adapter approach

Usage Examples

Basic Custom Selector

class AllComponentsSelector:
    """Select all components for simultaneous optimization."""
    
    def __call__(self, state, trajectories, subsample_scores, candidate_idx, candidate):
        return list(candidate.keys())  # Update all components together

gepa = dspy.GEPA(
    metric=your_metric,
    reflection_lm=reflection_lm,
    max_metric_calls=50,
    component_selector=AllComponentsSelector()  # Custom selector
)

Implementation Details

DSPy Changes

• dspy/teleprompt/gepa/gepa.py: Added component_selector parameter and integration with gepa engine
• tests/teleprompt/test_gepa.py: Tests for custom component selection

Core GEPA Library Changes Required

This PR requires corresponding changes to the gepa-ai/gepa repository to add the module_selector parameter to the optimize() function. Related PR: gepa-ai/gepa#52

Testing

• Core functionality: Custom selectors properly invoked and used
• Multiple selection: Simultaneous component optimization
• Default behavior: Backward compatibility maintained

Closes

• Fixes [Feature] GEPA: Allow custom component selection logic #8755

[LakshyAAAgrawal]

Released gepa==0.0.9

[andressrg]

@LakshyAAAgrawal ready to go 🚀

[LakshyAAAgrawal]

@andressrg , thank you so much! The combination of this and instruction proposer is going to be super-powerful!

Could you also take a look at #8767 ? I think at the moment, it will be difficult for anybody to be able to use it fully though.

I believe now that this is implemented, it should also be trivial to fix #8674. It will basically require a component selector that returns all the predictor names all the time.

[LakshyAAAgrawal]

These are independent issues, but at least documentation I think should be part of this PR. Would you be able to create just one example of using custom-instruction proposer, and one example of custom module selector?

[LakshyAAAgrawal]

Released gepa==0.0.10. Please update accordingly.

[andressrg]

Could you also take a look at #8767 ? I think at the moment, it will be difficult for anybody to be able to use it fully though.

for sure!

I believe now that this is implemented, it should also be trivial to fix #8674. It will basically require a component selector that returns all the predictor names all the time.

makes sense! let me do the docs one and come back to you on this one

[LakshyAAAgrawal]

Can you add the implementation of an "all" component selector, that selects all components to be modified.

[andressrg]

Sure.

but here? or the gepa core lib?

thinking it might be better in the gepa one

[LakshyAAAgrawal]

Sure!

[LakshyAAAgrawal]

Hi @andressrg , is this ready to merge? The documentation can come in a separate PR.

[andressrg]

Hi @LakshyAAAgrawal

It's missing this:

add the implementation of an "all" component selector, that selects all components to be modified.

But I've just submitted gepa-ai/gepa#67 🚀

[LakshyAAAgrawal]

Let's finalize this! Released gepa==v0.0.11

[LakshyAAAgrawal]

Looks like we need much tighter tests in GEPA

[LakshyAAAgrawal]

@andressrg I have yanked v0.0.11. Would you be able to fix this issue?

[LakshyAAAgrawal]

Isn't this an error in gepa repo?

[andressrg]

Isn't this an error in gepa repo?

no. Issue was that the component selector that I had in the new test in this repo was not using the __call__

[andressrg]

I think gepa core is correct 👍

[andressrg]

pushed all the updates for dspy that should properly use component_selector

[LakshyAAAgrawal]

Can you update to gepa==0.0.12

[andressrg]

sure

[andressrg]

sorry for the scare there 😅

[LakshyAAAgrawal]

Can we add some unit test to ensure that the next proposed candidate actually updated all components?

[andressrg]

sure. on it

[andressrg]

done 👍

[LakshyAAAgrawal]

This may be a lot of work, but would it be possible to create a unit test similar to https://github.com/stanfordnlp/dspy/blob/main/tests/teleprompt/test_gepa.py#L46-L53 that uses a custom instruction proposer + a custom module selector?

The DictDummyLLM was created by actually running it against a real LLM and saving it.

[andressrg]

This may be a lot of work, but would it be possible to create a unit test similar to https://github.com/stanfordnlp/dspy/blob/main/tests/teleprompt/test_gepa.py#L46-L53 that uses a custom instruction proposer + a custom module selector?

The DictDummyLLM was created by actually running it against a real LLM and saving it.

noted. Will do 👍

not sure I'll have it today tho. But will keep you posted