Skip to content

Commit c22abd8

Browse files
authored
Update guidance with testing guide + agent guidance for tests (#344)
I've been playing around more with guiding the agent to self-review its own test code more; I've found a little improvement by using this guidelines - it correctly used hooks more efficiently following these guidelines & it now provides better guidelines for humans Co-authored-by: Mackenzie Zastrow <[email protected]>
1 parent 51f9897 commit c22abd8

File tree

4 files changed

+597
-318
lines changed

4 files changed

+597
-318
lines changed

.github/agent-sops/task-implementer.sop.md

Lines changed: 60 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,8 @@ Outline the high-level structure of the implementation and create an implementat
140140
Write test cases based on the outlines, following strict TDD principles.
141141

142142
**Constraints:**
143+
144+
- You MUST follow the test patterns and conventions defined in [docs/TESTING.md](../../docs/TESTING.md)
143145
- You MUST validate that the task environment is set up properly
144146
- If you already created a commit, ensure the latest commit matches the expected hash
145147
- If not, ensure the correct branch is checked out
@@ -197,22 +199,53 @@ Write implementation code to pass the tests, focusing on simplicity and correctn
197199
- You MUST otherwise continue automatically after verifying test results
198200
- You MUST follow the Build Output Management practices defined in the Best Practices section
199201

200-
#### 4.3 Review, Refactor, and Optimize
202+
#### 4.3 Review and Refactor Implementation
201203

202-
If the implementation is complete, proceed with review of the implementation to identify opportunities for simplification or improvement.
204+
If the implementation is complete, proceed with a self-review of the implementation code to identify opportunities for simplification or improvement.
203205

204206
**Constraints:**
205-
- You MAY reply to user review threads with a concise response
206-
- You MUST keep your response to less than 3 sentences
207+
207208
- You MUST check that all tasks are complete before proceeding
208209
- if tests fail, you MUST identify the issue and implement a fix
209210
- if builds fail, you MUST identify the issue implement a fix
210211
- You MUST prioritize readability and maintainability over clever optimizations
211212
- You MUST maintain test passing status throughout refactoring
212213
- You SHOULD make note of simplification in your progress notes
213214
- You SHOULD record significant refactorings in your progress notes
215+
- You MUST return to step 4.2 if refactoring reveals additional implementation needs
216+
217+
#### 4.4 Review and Refactor Tests
218+
219+
After reviewing the implementation, review the test code to ensure it follows established patterns and provides adequate coverage.
220+
221+
**Constraints:**
222+
223+
- You MUST review your test code according to the guidelines in [docs/TESTING.md](../../docs/TESTING.md).
224+
- You MUST verify tests conform to the testing documentation standards
225+
- You MUST verify tests are readable and maintainable
226+
- You SHOULD refactor tests that are overly complex or duplicative
227+
- You MUST return to step 4.1 if tests need significant restructuring
228+
229+
**Testing Checklist Verification (REQUIRED):**
230+
231+
You MUST copy the checklist from [docs/TESTING.md](../../docs/TESTING.md) into your progress notes and explicitly verify each item. For each checklist item, you MUST:
232+
233+
1. Copy the checklist item verbatim
234+
2. Mark it as `[x]` (pass) or `[-]` (fail)
235+
3. If failed, provide a brief explanation and fix the issue before proceeding
236+
237+
Example format in your notes:
238+
239+
```markdown
240+
## Testing Checklist Verification
214241

215-
#### 4.4 Validate Implementation
242+
- [x] Do the tests use relevant helpers from `__fixtures__` as noted in the "Test Fixtures Reference" section
243+
- [ ] Are tests asserting on the entire object instead of specific fields? → FAILED: test on line 45 asserts individual properties, refactoring now
244+
```
245+
246+
You MUST NOT proceed to step 4.5 until ALL checklist items pass.
247+
248+
#### 4.5 Validate Implementation
216249

217250
If the implementation meets all requirements and follows established patterns, proceed with this step. Otherwise, return to step 4.2 to fix any issues.
218251

@@ -230,6 +263,24 @@ If the implementation meets all requirements and follows established patterns, p
230263
- You MUST verify that all dependencies are satisfied
231264
- You MUST follow the Build Output Management practices defined in the Best Practices section
232265

266+
#### 4.6 Respond to Review Feedback
267+
268+
If you have received feedback from user reviews or PR comments, address them before proceeding to the commit phase.
269+
270+
**Constraints:**
271+
272+
- You MAY skip this step if no user feedback has been received yet
273+
- You MUST reply to user review threads with a concise response
274+
- You MUST keep your response to less than 3 sentences
275+
- You MUST categorize each piece of feedback as:
276+
- Actionable code changes that can be implemented immediately
277+
- Clarifying questions that require user input
278+
- Suggestions to consider for future iterations
279+
- You MUST implement actionable code changes before proceeding
280+
- You MUST re-run tests after addressing feedback to ensure nothing is broken
281+
- You MUST return to step 4.3 after implementing changes to review the updated code
282+
- You MUST use the handoff_to_user tool if clarification is needed before you can proceed
283+
233284
### 5. Commit and Pull Request Phase
234285

235286
If all tests are passing, draft a conventional commit message, perform the git commit, and create/update the pull request.
@@ -239,7 +290,7 @@ If all tests are passing, draft a conventional commit message, perform the git c
239290
Before creating or updating a PR, you MUST copy the checklist from [docs/PR.md](../../docs/PR.md) into your progress notes and explicitly verify each item. For each checklist item, you MUST:
240291

241292
1. Copy the checklist item verbatim
242-
2. Mark it as `[x]` (pass) or `[ ]` (fail)
293+
2. Mark it as `[x]` (pass) or `[-]` (fail)
243294
3. If failed, revise the PR description until the item passes
244295

245296
Example format in your notes:
@@ -401,6 +452,8 @@ If builds fail during implementation:
401452
- Use appropriate package managers and dependency files for the project type
402453

403454
### Testing Best Practices
455+
456+
- You MUST follow the comprehensive testing guidelines in [docs/TESTING.md](../../docs/TESTING.md)
404457
- Follow TDD principles: RED → GREEN → REFACTOR
405458
- Write tests that fail initially, then implement to make them pass
406459
- Use appropriate testing frameworks for the project type or as specified in DEVELOPMENT.md
@@ -416,7 +469,7 @@ If builds fail during implementation:
416469

417470
### Checklist Verification Pattern
418471

419-
When documentation files contain checklists (e.g., `docs/PR.md`), you MUST:
472+
When documentation files contain checklists (e.g., `docs/TESTING.md`, `docs/PR.md`), you MUST:
420473

421474
1. Copy the entire checklist into your progress notes
422475
2. Explicitly verify each item by marking `[x]` or `[ ]`

0 commit comments

Comments
 (0)