Release 0.30.7

[odlbot]

Shankar Ambady

• Add all Contentfile metadata to chunk responses (Add all Contentfile metadata to chunk responses #2075) (aa026304)
• Make embedding generation task use correct run (Make embedding generation task use correct run #2074) (1001aeef)

Chris Chudzicki

• add MITOL_LOGOUT_SUFFIX to github actions (add MITOL_LOGOUT_SUFFIX to github actions #2079) (01fc6b66)
• Accessibility improvements (Accessibility improvements #2071) (0bed63b3)

Nathan Levesque

• Fix user migrations for SCIM (Fix user migrations for SCIM #2078) (736b8105)
• Fix SCIM view tests (Fix SCIM view tests #2073) (798ef523)
• Added SCIM fields to User and populate (Added SCIM fields to User and populate #2062) (6c9a8d20)
• Fix SCIM search API sort and pagination (Fix SCIM search API sort and pagination #2066) (e394d1dd)

Matt Bertrand

• APISIX integration (APISIX integration #2061) (131c04e4)

Arslan Ashraf

• fix: Opensearch container on ARM64 based architecture (fix: opensearch container on ARM64 based architecture #2069) (af38efac)

renovate[bot]

• Update dependency @dnd-kit/sortable to v10 (Update dependency @dnd-kit/sortable to v10 #1974) (aeb59d30)
• Update akhileshns/heroku-deploy digest to e3eb99d (Update akhileshns/heroku-deploy digest to e3eb99d #2068) (be430a1c)
• Update dependency @sentry/nextjs to v9 (Update dependency @sentry/nextjs to v9 #2034) (b3d12422)
• Update dependency @mui/lab to v6.0.0-beta.28 (Update dependency @mui/lab to v6.0.0-beta.28 #2051) (d507c1fb)
• Update dependency tldextract to v5 (Update dependency tldextract to v5 #2031) (d47aa29c)

[gitguardian]

⚠️ GitGuardian has uncovered 5 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
13777606	Triggered	Generic Private Key	131c04e	config/keycloak/tls/tls.key.default	View secret
13777608	Triggered	Generic High Entropy Secret	131c04e	config/keycloak/realms/default-realm.json	View secret
13777609	Triggered	Generic Password	131c04e	config/keycloak/realms/default-realm.json	View secret
13777610	Triggered	Generic High Entropy Secret	131c04e	config/keycloak/realms/default-realm.json	View secret
10259317	Triggered	Generic Password	131c04e	docker-compose.services.yml	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secrets safely. Learn here the best practices.
3. Revoke and rotate these secrets.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[gemini-code-assist]

Hello @odlbot, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

This pull request bumps the version to 0.30.7 and includes several updates and fixes across the project. These include adding content file metadata to chunk responses, fixing embedding generation tasks, adding a logout suffix to GitHub Actions, fixing user migrations and tests for SCIM, accessibility improvements, APISIX integration, ARM64 architecture support for OpenSearch, and dependency updates.

Highlights

• SCIM Fixes: Addresses issues with user migrations, view tests, search API sorting, and pagination for SCIM (System for Cross-domain Identity Management).
• APISIX Integration: Introduces APISIX integration for authentication, including configuration files and documentation.
• Dependency Updates: Updates several dependencies including @dnd-kit/sortable, akhileshns/heroku-deploy, @sentry/nextjs, @mui/lab, and tldextract.
• GitHub Actions: Adds MITOL_LOGOUT_SUFFIX to GitHub Actions workflows for production and release candidate environments.
• Keycloak Integration: Adds Keycloak and APISIX containers for authentication, including default settings, realm configuration, and instructions for making it work.

Changelog

Click here to see the changelog

• .github/workflows/production.yml
  • Updates the akhileshns/heroku-deploy action to the latest digest.
  • Adds MITOL_API_LOGOUT_SUFFIX to the environment variables for the backend release.
• .github/workflows/release-candidate.yml
  • Updates the akhileshns/heroku-deploy action to the latest digest.
  • Adds MITOL_API_LOGOUT_SUFFIX to the environment variables for the backend release.
• .pre-commit-config.yaml
  • Excludes Keycloak TLS and realm configuration files from pre-commit checks.
• .secrets.baseline
  • Excludes Keycloak TLS files from secret scanning.
• Dockerfile
  • Changes mkdir to mkdir -p for staticfiles directory to ensure parent directories are created if they don't exist.
• Dockerfile-litellm
  • Removes the Dockerfile-litellm file.
• README-keycloak.md
  • Adds documentation for Keycloak and APISIX integration, including default settings and instructions for setup.
• RELEASE.rst
  • Adds release notes for version 0.30.7, listing all the included fixes and updates.
• config/apisix/apisix.yaml
  • Adds configuration for APISIX routes and upstreams, including OpenID Connect plugin settings.
• config/apisix/config.yaml
  • Configures APISIX settings, including disabling the admin interface and setting the role to data plane.
• config/apisix/debug.yaml
  • Adds debug configuration for APISIX.
• config/keycloak/providers/README.md
  • Adds a README file for the Keycloak providers directory.
• config/keycloak/realms/default-realm.json
  • Adds a default realm configuration for Keycloak, including users, roles, and client settings.
• config/keycloak/tls/README.md
  • Adds a README file for the Keycloak TLS configuration, including instructions for adding custom certificates.
• config/keycloak/tls/tls.crt
  • Adds a default TLS certificate file.
• config/keycloak/tls/tls.crt.default
  • Adds a default TLS certificate file.
• config/keycloak/tls/tls.key
  • Adds a default TLS key file.
• config/keycloak/tls/tls.key.default
  • Adds a default TLS key file.
• config/litellm_config.yml
  • Removes the litellm config file.
• config/postgres/init-keycloak.sql
  • Adds SQL script to create the Keycloak database and grant privileges.
• docker-compose.litellm.yml
  • Removes the litellm docker compose file.
• docker-compose.opensearch.base.yml
  • Disables SVE (Scalable Vector Extension) for ARM64 architecture in the OpenSearch container.
• docker-compose.services.yml
  • Adds a volume mount for postgres init db scripts
  • Removes restart policy for qdrant
  • Adds Keycloak and APISIX service definitions, including environment variables, ports, and volumes.
• docker-compose.yml
  • Adds env files to docker compose include path
• env/backend.env
  • Changes MITOL_COOKIE_NAME from discussions to mitlearn.
• env/backend.local.example.env
  • Adds APISIX/Keycloak settings to the example environment file.
• env/frontend.env
  • Adds NEXT_PUBLIC_MITOL_API_LOGOUT_SUFFIX to the frontend environment variables.
• env/shared.env
  • Adds MITOL_API_LOGOUT_SUFFIX and comments for apisix/keycloak config
  • Adds ports for nginx, apisix, and keycloak
• env/shared.local.example.env
  • Adds MITOL_NEW_USER_LOGIN_URL to the example environment file.
• frontends/api/src/generated/v0/api.ts
  • Adds edx_block_id as a parameter to the VectorContentFilesSearchApi for retrieving content files.
• frontends/main/Dockerfile.web
  • Adds NEXT_PUBLIC_MITOL_API_LOGOUT_SUFFIX to the frontend Dockerfile.
• frontends/main/package.json
  • Updates @sentry/nextjs dependency to version 9.
• frontends/main/src/app-pages/ChatPage/ChatPage.tsx
  • Adds credentials: 'include' to fetch requests.
• frontends/main/src/app-pages/ChatSyllabusPage/ChatSyllabusPage.tsx
  • Adds credentials: 'include' to fetch requests.
• frontends/main/src/app-pages/DashboardPage/DashboardPage.tsx
  • Adds focus-visible styling to dashboard tabs
• frontends/main/src/common/urls.ts
  • Updates the LOGOUT URL to include the MITOL_API_LOGOUT_SUFFIX.
• frontends/main/src/page-components/AiChat/AiRecommendationBotDrawer.tsx
  • Adds credentials: 'include' to fetch requests.
• frontends/main/src/page-components/LearningResourceExpanded/AiChatSyllabusSlideDown.tsx
  • Adds credentials: 'include' to fetch requests.
• frontends/ol-components/package.json
  • Updates @dnd-kit/sortable dependency to version 10.
  • Updates @mui/lab dependency to version 6.0.0-beta.28
  • Updates @mui/material and @mui/system dependencies.
• frontends/ol-components/src/components/Checkbox/Checkbox.tsx
  • Adds focus-visible styling to checkboxes
• frontends/ol-components/src/components/Checkbox/CheckboxChoiceField.tsx
  • Fixes typescript error
  • Changes label to legend for accessibility
• frontends/ol-components/src/components/SelectField/SelectField.test.tsx
  • Adds tests for Select component
  • Adds test for pointer-open class
• frontends/ol-components/src/components/SelectField/SelectField.tsx
  • Adds pointer-open class to menu if and only if opened via pointer
• frontends/ol-components/src/components/SimpleSelect/SimpleSelect.tsx
  • Removes MenuItem style display none
• frontends/ol-utilities/package.json
  • Updates @dnd-kit/sortable dependency to version 10.
• learning_resources/models.py
  • Adds prefetch related to for_serialization
• main/factories.py
  • Adds scim fields to UserFactory
• main/middleware/apisix_user.py
  • Adds APISIX Middleware for MIT Learn.
• main/middleware/apisix_user_test.py
  • Adds tests for apisix middleware.
• main/settings.py
  • Updates version to 0.30.7
  • Adds ApisixUserMiddleware
  • Adds Keycloak API settings
• openapi/specs/v0.yaml
  • Adds edx_block_id to vector search
• poetry.lock
  • Updates poetry.lock file with dependency changes.
• profiles/admin.py
  • Removes scim fields from ProfileAdmin
• profiles/forms.py
  • Removes scim fields from ProfileForm
• pyproject.toml
  • Updates tldextract dependency to version 5.0.0.
• scim/adapters.py
  • Updates id_field to scim_id
  • Updates meta to use created_on and updated_on
  • Updates to_dict to use scim_external_id
  • Updates from_dict to use scim_username and scim_external_id
  • Updates handle_add to use scim_external_id
• scim/urls.py
  • Adds re_path for users-search
• scim/views.py
  • Adds SearchView for /.search endpoint
• scim/views_test.py
  • Updates user_q to filter by scim_external_id
  • Updates scim_client.put to use user.scim_id
  • Updates scim_client.patch to use user.scim_id
  • Updates _put_operation to use user.scim_id
  • Updates _expected_patch_value to use user.scim_id
  • Adds large_user_set fixture
  • Adds test_user_search
• users/admin.py
  • Adds Users admin
• users/migrations/0004_add_scim_and_timestamp_fields.py
  • Adds scim and timestamp fields
• users/migrations/0005_set_user_scim_id.py
  • Adds set user scim id
• users/models.py
  • Updates models to use AbstractSCIMUserMixin and TimestampedModel
• vector_search/constants.py
  • Adds edx_block_id, content_type, description, url, and file_type to QDRANT_RESOURCE_PARAM_MAP
  • Adds edx_block_id to QDRANT_RESOURCE_PAYLOAD_SCHEMA
• vector_search/serializers.py
  • Adds edx_block_id to ContentFileVectorSearchRequestSerializer
• vector_search/tasks.py
  • Updates start_embed_resources to use next_run if it exists, otherwise use the latest run
• vector_search/tasks_test.py
  • Adds test for embedded content from next run
  • Adds test for embedded content from latest run if next is missing
• vector_search/utils.py
  • Updates _content_file_vector_hits to include all content file metadata

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

A release is born, a version takes flight,
Through code and commits, shining ever so bright.
From fixes and features, a symphony grand,
A digital marvel, crafted by hand.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request updates the version number in the release notes and settings file. The changes are straightforward and seem correct.

Summary of Findings

Merge Readiness

The pull request updates the version number in the release notes and settings file. The changes are straightforward and appear correct. I am unable to approve the pull request, and users should have others review and approve this code before merging.

[ChristopherChudzicki]

Closing this to add #2083