You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Update guidance with testing guide + agent guidance for tests (#344)
I've been playing around more with guiding the agent to self-review its own test code more; I've found a little improvement by using this guidelines - it correctly used hooks more efficiently following these guidelines & it now provides better guidelines for humans
Co-authored-by: Mackenzie Zastrow <[email protected]>
Copy file name to clipboardExpand all lines: .github/agent-sops/task-implementer.sop.md
+60-7Lines changed: 60 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -140,6 +140,8 @@ Outline the high-level structure of the implementation and create an implementat
140
140
Write test cases based on the outlines, following strict TDD principles.
141
141
142
142
**Constraints:**
143
+
144
+
- You MUST follow the test patterns and conventions defined in [docs/TESTING.md](../../docs/TESTING.md)
143
145
- You MUST validate that the task environment is set up properly
144
146
- If you already created a commit, ensure the latest commit matches the expected hash
145
147
- If not, ensure the correct branch is checked out
@@ -197,22 +199,53 @@ Write implementation code to pass the tests, focusing on simplicity and correctn
197
199
- You MUST otherwise continue automatically after verifying test results
198
200
- You MUST follow the Build Output Management practices defined in the Best Practices section
199
201
200
-
#### 4.3 Review, Refactor, and Optimize
202
+
#### 4.3 Reviewand Refactor Implementation
201
203
202
-
If the implementation is complete, proceed with review of the implementation to identify opportunities for simplification or improvement.
204
+
If the implementation is complete, proceed with a self-review of the implementation code to identify opportunities for simplification or improvement.
203
205
204
206
**Constraints:**
205
-
- You MAY reply to user review threads with a concise response
206
-
- You MUST keep your response to less than 3 sentences
207
+
207
208
- You MUST check that all tasks are complete before proceeding
208
209
- if tests fail, you MUST identify the issue and implement a fix
209
210
- if builds fail, you MUST identify the issue implement a fix
210
211
- You MUST prioritize readability and maintainability over clever optimizations
211
212
- You MUST maintain test passing status throughout refactoring
212
213
- You SHOULD make note of simplification in your progress notes
213
214
- You SHOULD record significant refactorings in your progress notes
215
+
- You MUST return to step 4.2 if refactoring reveals additional implementation needs
216
+
217
+
#### 4.4 Review and Refactor Tests
218
+
219
+
After reviewing the implementation, review the test code to ensure it follows established patterns and provides adequate coverage.
220
+
221
+
**Constraints:**
222
+
223
+
- You MUST review your test code according to the guidelines in [docs/TESTING.md](../../docs/TESTING.md).
224
+
- You MUST verify tests conform to the testing documentation standards
225
+
- You MUST verify tests are readable and maintainable
226
+
- You SHOULD refactor tests that are overly complex or duplicative
227
+
- You MUST return to step 4.1 if tests need significant restructuring
228
+
229
+
**Testing Checklist Verification (REQUIRED):**
230
+
231
+
You MUST copy the checklist from [docs/TESTING.md](../../docs/TESTING.md) into your progress notes and explicitly verify each item. For each checklist item, you MUST:
232
+
233
+
1. Copy the checklist item verbatim
234
+
2. Mark it as `[x]` (pass) or `[-]` (fail)
235
+
3. If failed, provide a brief explanation and fix the issue before proceeding
236
+
237
+
Example format in your notes:
238
+
239
+
```markdown
240
+
## Testing Checklist Verification
214
241
215
-
#### 4.4 Validate Implementation
242
+
- [x] Do the tests use relevant helpers from `__fixtures__` as noted in the "Test Fixtures Reference" section
243
+
- [ ] Are tests asserting on the entire object instead of specific fields? → FAILED: test on line 45 asserts individual properties, refactoring now
244
+
```
245
+
246
+
You MUST NOT proceed to step 4.5 until ALL checklist items pass.
247
+
248
+
#### 4.5 Validate Implementation
216
249
217
250
If the implementation meets all requirements and follows established patterns, proceed with this step. Otherwise, return to step 4.2 to fix any issues.
218
251
@@ -230,6 +263,24 @@ If the implementation meets all requirements and follows established patterns, p
230
263
- You MUST verify that all dependencies are satisfied
231
264
- You MUST follow the Build Output Management practices defined in the Best Practices section
232
265
266
+
#### 4.6 Respond to Review Feedback
267
+
268
+
If you have received feedback from user reviews or PR comments, address them before proceeding to the commit phase.
269
+
270
+
**Constraints:**
271
+
272
+
- You MAY skip this step if no user feedback has been received yet
273
+
- You MUST reply to user review threads with a concise response
274
+
- You MUST keep your response to less than 3 sentences
275
+
- You MUST categorize each piece of feedback as:
276
+
- Actionable code changes that can be implemented immediately
277
+
- Clarifying questions that require user input
278
+
- Suggestions to consider for future iterations
279
+
- You MUST implement actionable code changes before proceeding
280
+
- You MUST re-run tests after addressing feedback to ensure nothing is broken
281
+
- You MUST return to step 4.3 after implementing changes to review the updated code
282
+
- You MUST use the handoff_to_user tool if clarification is needed before you can proceed
283
+
233
284
### 5. Commit and Pull Request Phase
234
285
235
286
If all tests are passing, draft a conventional commit message, perform the git commit, and create/update the pull request.
@@ -239,7 +290,7 @@ If all tests are passing, draft a conventional commit message, perform the git c
239
290
Before creating or updating a PR, you MUST copy the checklist from [docs/PR.md](../../docs/PR.md) into your progress notes and explicitly verify each item. For each checklist item, you MUST:
240
291
241
292
1. Copy the checklist item verbatim
242
-
2. Mark it as `[x]` (pass) or `[]` (fail)
293
+
2. Mark it as `[x]` (pass) or `[-]` (fail)
243
294
3. If failed, revise the PR description until the item passes
244
295
245
296
Example format in your notes:
@@ -401,6 +452,8 @@ If builds fail during implementation:
401
452
- Use appropriate package managers and dependency files for the project type
402
453
403
454
### Testing Best Practices
455
+
456
+
- You MUST follow the comprehensive testing guidelines in [docs/TESTING.md](../../docs/TESTING.md)
404
457
- Follow TDD principles: RED → GREEN → REFACTOR
405
458
- Write tests that fail initially, then implement to make them pass
406
459
- Use appropriate testing frameworks for the project type or as specified in DEVELOPMENT.md
@@ -416,7 +469,7 @@ If builds fail during implementation:
416
469
417
470
### Checklist Verification Pattern
418
471
419
-
When documentation files contain checklists (e.g., `docs/PR.md`), you MUST:
472
+
When documentation files contain checklists (e.g., `docs/TESTING.md`, `docs/PR.md`), you MUST:
420
473
421
474
1. Copy the entire checklist into your progress notes
422
475
2. Explicitly verify each item by marking `[x]` or `[ ]`
0 commit comments