Skip to content

Conversation

kshitij-k-osmosys
Copy link
Contributor

@kshitij-k-osmosys kshitij-k-osmosys commented Dec 2, 2024

API PR Checklist

Pre-requisites

  • I have gone through the Contributing guidelines for Submitting a Pull Request (PR) and ensured that this is not a duplicate PR.
  • I have performed preliminary unit testing.
  • I have updated the required api docs as applicable.
  • I have added/updated test cases to the test suite as applicable

PR Details

PR details have been updated as per the given format (see below)

  • PR title adheres to the format specified in guidelines (e.g., feat: add admin login endpoint)
  • Description has been added
  • Related changes have been added (optional)
  • Screenshots have been added (optional)
  • Query request and response examples have been added (as applicable, in case added or updated)
  • Documentation changes have been listed (as applicable)
  • Test suite output is added (as applicable)
  • Pending actions have been added (optional)
  • Any other additional notes have been added (optional)

Additional Information

  • Appropriate label(s) have been added (ready for review should be added if the PR is ready to be reviewed)
  • Assignee(s) and reviewer(s) have been added (optional)

Description:

Create script to schedule adding Pending Summaries To Queue instead of Cron

Related changes:

  • Remove existing cron
  • Create controller for summary module
  • Create endpoint scheduleAddingPendingSummariesToQueue to run scheduler.service function
  • Create bash script to call this endpoint as pre defined intervals
  • Update documentation
  • Update nestWinstonModuleUtilities variable name

Screenshots:
NA

Query request and response:
NA

Documentation changes:

  • Add step to run scheduler script in development-setup.md
  • Add same step in usage-guide.md

Test suite output:
NA

Pending actions:
NA

Additional notes:
NA

Summary by CodeRabbit

  • New Features

    • Introduced a new scheduler.sh script for scheduling summary tasks.
    • Added a SummaryController with a new endpoint for scheduling summaries.
  • Documentation

    • Updated the development setup guide with instructions for running the scheduler script.
    • Enhanced the usage guide to include steps for running the scheduler script after starting the API.
  • Bug Fixes

    • Adjusted logging format to reflect the new application context.
  • Tests

    • Added a test suite for the SummaryController to ensure proper instantiation.

@kshitij-k-osmosys kshitij-k-osmosys added the Ready for review Ready for review label Dec 2, 2024
@kshitij-k-osmosys kshitij-k-osmosys self-assigned this Dec 2, 2024
Copy link
Contributor

coderabbitai bot commented Dec 2, 2024

Caution

Review failed

Rate limit reached, please try again later.

Warning

Rate limit exceeded

@kshitij-k-osmosys has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 6 minutes and 23 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between c5a1010 and d19b012.

Walkthrough

The changes in this pull request involve modifications to the Dockerfile, documentation, and the introduction of a new shell script and controller for managing summary tasks. The Dockerfile now executes a scheduler script alongside the NestJS application. Documentation updates include new instructions for running the scheduler script. A new scheduler.sh script has been added to handle scheduling tasks, and a SummaryController has been introduced to facilitate the addition of pending summaries to a queue.

Changes

File Path Change Summary
apps/api/Dockerfile Changed command from CMD ["node", "dist/main.js"] to CMD ./scheduler.sh & node dist/main.js.
apps/api/docs/development-setup.md Added Step 6 to instruct users to run ./scheduler.sh for setting up the API.
apps/api/docs/usage-guide.md Updated instructions to include running the scheduler script after starting the API and portal.
apps/api/scheduler.sh Introduced a new script for scheduling tasks, sending requests to a specified URL every second.
apps/api/src/config/logger.config.ts Updated logging format from 'OsmosysAssistant' to 'TranscriptSummarizer'.
apps/api/src/modules/summary/schedule/schedule.service.ts Renamed method from addSummaryToQueue to addSummaryToQueueCron and removed @Cron decorator.
apps/api/src/modules/summary/summary.controller.spec.ts Added a new test suite for SummaryController.
apps/api/src/modules/summary/summary.controller.ts Introduced SummaryController with a POST endpoint summary/schedule to handle scheduling pending summaries.
apps/api/src/modules/summary/summary.module.ts Updated SummaryModule to include the new SummaryController.

Possibly related PRs

  • feat: add consumer, producer for summary job #17: The changes in this PR involve adding a consumer and producer for summary jobs, which relates to the new scheduler.sh script introduced in the main PR that manages scheduling tasks for summaries.
  • refactor: add memory use logs, cleanup queue, streams #44: This PR enhances the ScheduleService class, which is relevant to the changes in the main PR that involve executing the scheduler.sh script as part of the Docker container startup process.
  • docs: add usage guide #52: The usage guide updates in this PR include instructions for running the scheduler.sh script, which directly connects to the changes made in the main PR regarding the Dockerfile and the execution of this script.
  • docs: create project onboarding #60: The onboarding document may reference the overall project setup, including the new scheduling functionality introduced in the main PR, which is relevant for new contributors to understand the project's operational flow.

Suggested labels

documentation

Suggested reviewers

  • osm-vishnukyatannawar

🐰 In the meadow, the rabbits play,
With scripts and logs, they hop all day.
A scheduler here, a controller there,
Summaries queued with utmost care.
Hooray for changes, let’s cheer and sway! 🎉


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 11

🧹 Outside diff range and nitpick comments (2)
apps/api/src/config/logger.config.ts (1)

Line range hint 13-16: Consider implementing distributed tracing for request correlation

The TODO comment about tracebackId suggests a need for request correlation across components. Consider implementing proper distributed tracing using a solution like OpenTelemetry.

Would you like me to provide an example implementation using OpenTelemetry for distributed tracing?

apps/api/docs/usage-guide.md (1)

Security concerns identified in scheduler implementation

The architectural change from Cron to script-based scheduling has several security and implementation issues:

  1. The scheduler endpoint (/summary/schedule) lacks authentication guards, while other endpoints are protected with AzureADGuard
  2. The scheduler script makes unauthenticated HTTP calls to the endpoint
  3. No error handling or recovery mechanism in the scheduler script
  4. No rate limiting or concurrency control in the scheduler implementation
🔗 Analysis chain

Line range hint 24-102: Verify complete implementation of scheduling changes

The architectural change from Cron to script-based scheduling appears sound, but several critical components need verification:

  1. The scheduler.sh script implementation
  2. The controller endpoint security
  3. Error handling and recovery in the scheduler

Let's verify the complete implementation:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check for scheduler script
fd -t f "scheduler.sh$"

# Check for controller endpoint security
ast-grep --pattern 'class SummaryController {
  $$$
  @$DECORATOR("summary/schedule")
  scheduleAddingPendingSummariesToQueue() {
    $$$
  }
  $$$
}'

# Check for error handling
rg -A 5 "catch.*error.*scheduler"

Length of output: 236


Script:

#!/bin/bash
# Let's check the scheduler script content
cat apps/api/scheduler.sh

# Check for controller implementation with broader pattern
ast-grep --pattern 'class SummaryController {
  $$$
}'

# Check for any scheduling-related code
rg -A 5 "schedule|cron|queue"

Length of output: 20969


Script:

#!/bin/bash
# Let's check for security decorators and error handling in the schedule service
ast-grep --pattern 'class ScheduleService {
  $$$
}'

# Check for any authentication guards
rg -A 5 "@UseGuards|@Auth"

# Check for error handling in the service
rg -A 5 "catch.*error.*summary"

Length of output: 1221

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between e37716a and 66cfc6c.

📒 Files selected for processing (9)
  • apps/api/Dockerfile (1 hunks)
  • apps/api/docs/development-setup.md (2 hunks)
  • apps/api/docs/usage-guide.md (2 hunks)
  • apps/api/scheduler.sh (1 hunks)
  • apps/api/src/config/logger.config.ts (1 hunks)
  • apps/api/src/modules/summary/schedule/schedule.service.ts (1 hunks)
  • apps/api/src/modules/summary/summary.controller.spec.ts (1 hunks)
  • apps/api/src/modules/summary/summary.controller.ts (1 hunks)
  • apps/api/src/modules/summary/summary.module.ts (2 hunks)
🧰 Additional context used
🪛 Shellcheck (0.10.0)
apps/api/scheduler.sh

[warning] 13-13: last_archive_run appears unused. Verify use (or export if used externally).

(SC2034)

🪛 LanguageTool
apps/api/docs/development-setup.md

[uncategorized] ~102-~102: Possible missing article found.
Context: ...tion that adds all pending summaries to queue for processing.

(AI_HYDRA_LEO_MISSING_A)

🔇 Additional comments (2)
apps/api/src/modules/summary/summary.module.ts (1)

13-13: Verify controller dependencies and consider exporting ScheduleService

The controller addition looks good, but please verify:

  1. That SummaryController has access to all required dependencies
  2. Whether ScheduleService should be exported if it's needed by other modules

Also applies to: 27-27

✅ Verification successful

Let me gather more information about the dependencies and service usage.


Let me check one final thing to see if the controller's constructor is properly injecting the ScheduleService.


Dependencies are properly configured, no need to export ScheduleService

The verification shows that:

  • SummaryController correctly injects ScheduleService in its constructor
  • ScheduleService is properly provided in the SummaryModule
  • ScheduleService is only used within the summary module (no external usage found), so it doesn't need to be exported
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check if ScheduleService is used outside this module
rg "ScheduleService" --type ts -l | grep -v "summary/schedule"

# Verify controller's constructor dependencies
ast-grep --pattern 'class SummaryController {
  constructor($_) {
    $$$
  }
}'

Length of output: 241


Script:

#!/bin/bash
# Check SummaryController's constructor dependencies
ast-grep --pattern 'constructor($_) {
  $$$
}' apps/api/src/modules/summary/summary.controller.ts

# Check if ScheduleService is imported in summary.controller.ts
rg "import.*ScheduleService" apps/api/src/modules/summary/summary.controller.ts

# Check module providers and exports
cat apps/api/src/modules/summary/summary.module.ts

Length of output: 1476


Script:

#!/bin/bash
# Get the full SummaryController implementation to check constructor
cat apps/api/src/modules/summary/summary.controller.ts

Length of output: 437

apps/api/src/config/logger.config.ts (1)

38-38: LGTM: App name updated correctly

The logger name change from 'OsmosysAssistant' to 'TranscriptSummarizer' accurately reflects the application's identity.

@xixas xixas merged commit 57f9208 into main Dec 3, 2024
6 checks passed
@xixas xixas deleted the feat/add-scheduler-script branch December 3, 2024 05:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Ready for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants