Skip to content

Conversation

@kevalmahajan
Copy link
Member

@kevalmahajan kevalmahajan commented Jul 2, 2025

✨ Feature / Enhancement PR

πŸ”— Epic / Issue

Link to the epic or parent issue:
Closes #159
And partial of #172


πŸš€ Summary (1–2 sentences)

Gateways/mcp servers have status as "Active", "Offline" and "Inactive".
Gateway is turned as offline if health check is failed for the threshold value set and turned back as active once its reachable again.
The corresponding tool is also turned as offline and when reachable is turned to active.

changes in db:
'is_active' column in gateways and tools table is converted to 'status': enabled and reachable columns


πŸ§ͺ Checks

  • make lint passes
  • make test passes
  • CHANGELOG updated (if user-facing)

πŸ““ Notes (optional)

  1. If a gateway is deemed unhealthy during health checks, it is set to offline, when its back online/reachable, it is set to active automatically.
image
  1. All the corresponding tools for the gateway also follow the same states as gateway.
image
  1. Manually deactivate gateways as well as tools, remain deactivated even if they are reachable/accessible. Specific tools can be deactivated if needed, they will remain deactivated, even if the other tools and gateway is healthy and reachable.

  2. When tools are offline, tool calls repond as :
    "Tool 'test-gateway-get-alerts' exists but is currently offline. Please verify if it is running."

image
  1. For migration, use alembic script : alembic/versions/e75490e949b1_add_improved_status_to_tables.py

Signed-off-by: <Keval Mahajan> <[email protected]>
Signed-off-by: <Keval Mahajan> <[email protected]>
Signed-off-by: <Keval Mahajan> <[email protected]>
Signed-off-by: <Keval Mahajan> <[email protected]>
Signed-off-by: <Keval Mahajan> <[email protected]>
Signed-off-by: <Keval Mahajan> <[email protected]>
@crivetimihai
Copy link
Member

crivetimihai commented Jul 2, 2025

Lint

  • make flake8 ruff

Clean (new sqlite db)

  • make test
  • make smoketest
  • make docker-run-ssl-host docker-logs docker-run-ssl-host

Migration

  • make compose-down compose-rm compose-up
docker logs mcp-context-forge-migration-1
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade e4fc04d1a442 -> e75490e949b1, Add improved status to tables (is_active -> status)

App status / login

❌ Logging into the app and checking docker logs mcp-context-forge-gateway-1 leads to internal server error (likely something failed in DB migration)

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.DatatypeMismatch) argument of WHERE must be type boolean, not type json
LINE 3: WHERE tools.status -> 'enabled'
              ^

[SQL: SELECT tools.id, tools.original_name, tools.original_name_slug, tools.url, tools.description, tools.integration_type, tools.request_type, tools.headers, tools.input_schema, tools.annotations, tools.created_at, tools.updated_at, tools.status, tools.jsonpath_filter, tools.auth_type, tools.auth_value, tools.gateway_id, tools.name

Copy link
Member

@crivetimihai crivetimihai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker logs mcp-context-forge-migration-1
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade e4fc04d1a442 -> e75490e949b1, Add improved status to tables (is_active -> status)

@kevalmahajan DB migration executed successfully, but the app doesn't work correctly:

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.DatatypeMismatch) argument of WHERE must be type boolean, not type json
LINE 3: WHERE tools.status -> 'enabled'
              ^

[SQL: SELECT tools.id, tools.original_name, tools.original_name_slug, tools.url, tools.description, tools.integration_type, tools.request_type, tools.headers, tools.input_schema, tools.annotations, tools.created_at, tools.updated_at, tools.status, tools.jsonpath_filter, tools.auth_type, tools.auth_value, tools.gateway_id, tools.name
FROM tools
WHERE tools.status -> %(status_1)s]
[parameters: {'status_1': 'enabled'}]
(Background on this error at: https://sqlalche.me/e/20/f405)

Please also use git commit -s -m to pass DCO.

@crivetimihai crivetimihai self-requested a review July 2, 2025 23:15
@kevalmahajan
Copy link
Member Author

The SQLite database migration is working fine, but there seems to be an issue with the PostgreSQL database migration. I'm currently investigating the problem.

@kevalmahajan
Copy link
Member Author

Instead of storing the JSON object {"enabled": True, "reachable": True} as a single JSON column, we migrated this into separate columns for better compatibility across both databases : sqlite and postgresql. The migration resulted in the following schema:

  • enabled (boolean)
  • reachable (boolean)

This way, we can continue working with both SQLite and PostgreSQL without facing limitations related to their different approaches to JSON storage.

@crivetimihai
Copy link
Member

  • rm mcp.db; make test
  • make venv install serve # After the redis fix, know issue in Fix/redis import lazyΒ #235
  • make smoketest
  • make docker-prod docker-run-ssl-host
  • export MCPGATEWAY_BEARER_TOKEN=$(python3 -m mcpgateway.utils.create_jwt_token --username admin --exp 0 --secret my-test-key); curl -sk -H "Authorization: Bearer $MCPGATEWAY_BEARER_TOKEN" https://localhost:4444/version
  • make docker-stop compose-down compose-rm; docker volume rm mcp-context-forge_pgdata; make compose-up
  File "/app/.venv/lib64/python3.11/site-packages/sqlalchemy/engine/base.py", line 1963, in _exec_single_context
    self.dialect.do_execute(
  File "/app/.venv/lib64/python3.11/site-packages/sqlalchemy/engine/default.py", line 943, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "tools" does not exist

[SQL: ALTER TABLE tools RENAME is_active TO enabled]
(Background on this error at: https://sqlalche.me/e/20/f405)

@madhav165
Copy link
Collaborator

@kevalmahajan Needs to check to see if table exists. Like in the other scripts.

madhav165 and others added 4 commits July 4, 2025 18:31
madhav165 added 2 commits July 4, 2025 21:39
This reverts commit 2ecc431.
Copy link
Collaborator

@madhav165 madhav165 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge PR #240 before this so that alembic migration works correctly with empty as well as populated databases.

Tested with SQL and PostgreSQL - both work along with migrations to this head if the alembic_version of previous database structure is stored in db. PR 240 is for starting that history.

@crivetimihai
Copy link
Member

from previous #240 volume:

  • rm mcp.db; make test
  • make venv install serve # After the redis fix, know issue in Fix/redis import lazyΒ #235
  • make smoketest
  • make docker-prod docker-run-ssl-host
  • export MCPGATEWAY_BEARER_TOKEN=$(python3 -m mcpgateway.utils.create_jwt_token --username admin --exp 0 --secret my-test-key); curl -sk -H "Authorization: Bearer $MCPGATEWAY_BEARER_TOKEN" https://localhost:4444/version
  • make docker-stop compose-down compose-rm; make compose-up # from previous Start alembic revision historyΒ #240 volume

Copy link
Member

@crivetimihai crivetimihai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merged after #240, huge pr, tested - will need more cleanup after to remove db migration task in docker-compose and helm as this is now automatic.

@crivetimihai crivetimihai merged commit b9d3af4 into main Jul 4, 2025
22 of 23 checks passed
@crivetimihai crivetimihai deleted the reactivate_gateway branch July 4, 2025 16:36
vk-playground pushed a commit to vk-playground/mcp-context-forge that referenced this pull request Sep 14, 2025
…ties (IBM#215)

* changes implemented reachable and enabled as parameters for status of gateways and tools

Signed-off-by: <Keval Mahajan> <[email protected]>

* Updated test cases as per new changes for status in gateways and tools

Signed-off-by: <Keval Mahajan> <[email protected]>

* minor changes

Signed-off-by: <Keval Mahajan> <[email protected]>

* black and isort

Signed-off-by: <Keval Mahajan> <[email protected]>

* Alembic script for db migration

Signed-off-by: <Keval Mahajan> <[email protected]>

* offline tools rpc response

Signed-off-by: <Keval Mahajan> <[email protected]>

* offline tools rpc response message

Signed-off-by: <Keval Mahajan> <[email protected]>

* minor changes

Signed-off-by: <Keval Mahajan> <[email protected]>

* minor flake8 issues

Signed-off-by: <Keval Mahajan> <[email protected]>

* db migration for postgresql

Signed-off-by: <Keval Mahajan> <[email protected]>

* Check for existing tables in migration script
Signed-off-by: Madhav Kandukuri <[email protected]>

* db migration from status to enabled and reachable

Signed-off-by: <Keval Mahajan> <[email protected]>

* reachable value is updated as necessary

Signed-off-by: <Keval Mahajan> <[email protected]>

* Updated test cases and linting

Signed-off-by: Keval Mahajan <[email protected]>

* Use bootstrap script
Signed-off-by: Madhav Kandukuri <[email protected]>

* linting

Signed-off-by: Keval Mahajan <[email protected]>

* Use bootstrap script in main for table creation
Signed-off-by: Madhav Kandukuri <[email protected]>

* Revert "Use bootstrap script in main for table creation"

This reverts commit 20fcceb.

* Revert "linting"

This reverts commit 2ecc431.

* Revert "Use bootstrap script"

This reverts commit d579e8e.

---------

Signed-off-by: <Keval Mahajan> <[email protected]>
Signed-off-by: Keval Mahajan <[email protected]>
Co-authored-by: Madhav Kandukuri <[email protected]>
vk-playground pushed a commit to vk-playground/mcp-context-forge that referenced this pull request Sep 14, 2025
…ties (IBM#215)

* changes implemented reachable and enabled as parameters for status of gateways and tools

Signed-off-by: <Keval Mahajan> <[email protected]>

* Updated test cases as per new changes for status in gateways and tools

Signed-off-by: <Keval Mahajan> <[email protected]>

* minor changes

Signed-off-by: <Keval Mahajan> <[email protected]>

* black and isort

Signed-off-by: <Keval Mahajan> <[email protected]>

* Alembic script for db migration

Signed-off-by: <Keval Mahajan> <[email protected]>

* offline tools rpc response

Signed-off-by: <Keval Mahajan> <[email protected]>

* offline tools rpc response message

Signed-off-by: <Keval Mahajan> <[email protected]>

* minor changes

Signed-off-by: <Keval Mahajan> <[email protected]>

* minor flake8 issues

Signed-off-by: <Keval Mahajan> <[email protected]>

* db migration for postgresql

Signed-off-by: <Keval Mahajan> <[email protected]>

* Check for existing tables in migration script
Signed-off-by: Madhav Kandukuri <[email protected]>

* db migration from status to enabled and reachable

Signed-off-by: <Keval Mahajan> <[email protected]>

* reachable value is updated as necessary

Signed-off-by: <Keval Mahajan> <[email protected]>

* Updated test cases and linting

Signed-off-by: Keval Mahajan <[email protected]>

* Use bootstrap script
Signed-off-by: Madhav Kandukuri <[email protected]>

* linting

Signed-off-by: Keval Mahajan <[email protected]>

* Use bootstrap script in main for table creation
Signed-off-by: Madhav Kandukuri <[email protected]>

* Revert "Use bootstrap script in main for table creation"

This reverts commit 20fcceb.

* Revert "linting"

This reverts commit 2ecc431.

* Revert "Use bootstrap script"

This reverts commit d579e8e.

---------

Signed-off-by: <Keval Mahajan> <[email protected]>
Signed-off-by: Keval Mahajan <[email protected]>
Co-authored-by: Madhav Kandukuri <[email protected]>
vk-playground pushed a commit to vk-playground/mcp-context-forge that referenced this pull request Sep 16, 2025
…ties (IBM#215)

* changes implemented reachable and enabled as parameters for status of gateways and tools

Signed-off-by: <Keval Mahajan> <[email protected]>

* Updated test cases as per new changes for status in gateways and tools

Signed-off-by: <Keval Mahajan> <[email protected]>

* minor changes

Signed-off-by: <Keval Mahajan> <[email protected]>

* black and isort

Signed-off-by: <Keval Mahajan> <[email protected]>

* Alembic script for db migration

Signed-off-by: <Keval Mahajan> <[email protected]>

* offline tools rpc response

Signed-off-by: <Keval Mahajan> <[email protected]>

* offline tools rpc response message

Signed-off-by: <Keval Mahajan> <[email protected]>

* minor changes

Signed-off-by: <Keval Mahajan> <[email protected]>

* minor flake8 issues

Signed-off-by: <Keval Mahajan> <[email protected]>

* db migration for postgresql

Signed-off-by: <Keval Mahajan> <[email protected]>

* Check for existing tables in migration script
Signed-off-by: Madhav Kandukuri <[email protected]>

* db migration from status to enabled and reachable

Signed-off-by: <Keval Mahajan> <[email protected]>

* reachable value is updated as necessary

Signed-off-by: <Keval Mahajan> <[email protected]>

* Updated test cases and linting

Signed-off-by: Keval Mahajan <[email protected]>

* Use bootstrap script
Signed-off-by: Madhav Kandukuri <[email protected]>

* linting

Signed-off-by: Keval Mahajan <[email protected]>

* Use bootstrap script in main for table creation
Signed-off-by: Madhav Kandukuri <[email protected]>

* Revert "Use bootstrap script in main for table creation"

This reverts commit 20fcceb.

* Revert "linting"

This reverts commit 2ecc431.

* Revert "Use bootstrap script"

This reverts commit d579e8e.

---------

Signed-off-by: <Keval Mahajan> <[email protected]>
Signed-off-by: Keval Mahajan <[email protected]>
Co-authored-by: Madhav Kandukuri <[email protected]>
crivetimihai added a commit that referenced this pull request Oct 19, 2025
…mplates

The pagination templates were using 'tool.gatewayReachable' which doesn't
exist in the serialized JSON. The correct field name is 'tool.reachable'.

This caused all tools to show as 'Offline' with the message 'The host
gateway for this tool is unreachable' even when the MCP Server status
was Active.

Root Cause:
- ToolRead schema has 'reachable: bool' field
- BaseModelWithConfigDict uses alias_generator=to_camel_case
- Since 'reachable' has no underscores, it stays as 'reachable' in JSON
- Templates incorrectly expected 'gatewayReachable' instead

Changes:
- tools_partial.html: tool.gatewayReachable β†’ tool.reachable (1 occurrence)
- tools_with_pagination.html: tool.gatewayReachable β†’ tool.reachable (2 occurrences)

The 'reachable' field was originally added in PR #215 for gateway/tool
health monitoring and works correctly throughout the codebase except
for these template references introduced in the pagination PR.

Fixes incorrect status display in Tools UI pagination.

Signed-off-by: Mihai Criveti <[email protected]>
crivetimihai added a commit that referenced this pull request Oct 19, 2025
* Tools pagination

Signed-off-by: Mihai Criveti <[email protected]>

* Tools pagination still broken

Signed-off-by: Mihai Criveti <[email protected]>

* Tools pagination still broken

Signed-off-by: Mihai Criveti <[email protected]>

* copied from rest-api-and-ui-pagination-v2

Signed-off-by: Madhav Kandukuri <[email protected]>

* Use jsonable_encoder for tools_pydantic
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix console error from tools_partial
Signed-off-by: Madhav Kandukuri <[email protected]>

* Multiple changes
Signed-off-by: Madhav Kandukuri <[email protected]>ok, d

* Swapping working but styling needs update
Signed-off-by: Madhav Kandukuri <[email protected]>

* Some working versions
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix everything
Signed-off-by: Madhav Kandukuri <[email protected]>

* Use tools_partial for first load
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix minor bug
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix failing tests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Remove extra logging
Signed-off-by: Madhav Kandukuri <[email protected]>

* add uv.lock from main

Signed-off-by: Madhav Kandukuri <[email protected]>

* Fixes revision in latest alembic script
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix one doctest
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix doctest
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix some doctests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Minor updates to pagination doctests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Mock db in doctests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix doctests in pagination
Signed-off-by: Madhav Kandukuri <[email protected]>

* Add trailing line in tools_with_pagination
Signed-off-by: Madhav Kandukuri <[email protected]>

* pre-commit fixes
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix key navigation in pagination
Signed-off-by: Madhav Kandukuri <[email protected]>

* pylint fix
Signed-off-by: Madhav Kandukuri <[email protected]>

* fix: Update pagination migration to follow correct alembic chain

The pagination indexes migration now correctly follows e5a59c16e041
(unique constraints for prompts/resources) which was merged to main
after this PR branch was created.

Migration chain:
- e5a59c16e041 (main)
- g1a2b3c4d5e6 (pagination indexes) <- this PR
- 9aaa90ad26d9 (output schema)
- 8a2934be50c0 (rest pass api)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Restore correct down_revision for output_schema migration

The output_schema migration (9aaa90ad26d9) should point to 9c99ec6872ed,
not to g1a2b3c4d5e6, to avoid creating a circular dependency.

Correct migration chain:
- 9c99ec6872ed (token usage logs fix)
- 9aaa90ad26d9 (output schema) <- restored
- 8a2934be50c0 (rest pass api)
- e5a59c16e041 (unique constraints)
- g1a2b3c4d5e6 (pagination indexes) <- this PR

Signed-off-by: Mihai Criveti <[email protected]>

* docs: Update CHANGELOG.md with REST API and UI pagination feature

Add comprehensive documentation for PR #1277 pagination feature including:
- Paginated REST API endpoints with backward compatibility
- Database indexes for efficient pagination queries
- HTMX-based UI pagination with keyboard navigation
- 11 new pagination configuration environment variables
- Pagination utilities module for offset and cursor-based pagination
- 1,089+ lines of test coverage

Addresses #1224

Signed-off-by: Mihai Criveti <[email protected]>

* fix: PostgreSQL foreign key handling in e5a59c16e041 migration

The unique constraint migration was failing on PostgreSQL during upgrades
from 0.8.0 because it attempted to drop tables that have foreign key
dependencies without first dropping those constraints.

Changes:
- Detect PostgreSQL and query information_schema for incoming FKs
- Drop foreign key constraints before dropping/recreating tables
- Recreate foreign key constraints after table recreation
- Apply same fix to both upgrade() and downgrade() functions

This fixes the error:
  'cannot drop table prompts because other objects depend on it'

Affected tables: prompts, resources, a2a_agents
Dependent tables: server_prompt_association, prompt_metrics, etc.

Tested on: SQLite (unchanged behavior), PostgreSQL (now works)

Addresses upgrade path from 0.8.0 -> 0.9.0 (unreleased)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Use correct column name in email_api_tokens pagination index

The email_api_tokens table has a column 'user_email', not 'owner_email'.
The pagination migration was trying to create an index on a non-existent
column, causing PostgreSQL migrations to fail.

Changes:
- Renamed index from 'ix_email_api_tokens_owner_created_at' to
  'ix_email_api_tokens_user_email_created_at'
- Updated both upgrade() and downgrade() functions
- Index now correctly references the 'user_email' column

Error fixed:
  'column "owner_email" does not exist'

Table schema reference:
  mcpgateway/alembic/versions/cfc3d6aa0fb2 (line 208)
  defines email_api_tokens.user_email

Addresses upgrade path from 0.8.0 -> 0.9.0 (unreleased)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Use correct column name in email_auth_events pagination index

The email_auth_events table has a column 'user_email', not 'email'.
The pagination migration was trying to create an index on a non-existent
column, causing PostgreSQL migrations to fail.

Changes:
- Renamed index from 'ix_email_auth_events_user_timestamp' to
  'ix_email_auth_events_user_email_timestamp'
- Changed column from 'email' to 'user_email' in index definition
- Updated both upgrade() and downgrade() functions

Error fixed:
  'column "email" does not exist'

Table schema reference:
  mcpgateway/alembic/versions/cfc3d6aa0fb2 (line 100)
  defines email_auth_events.user_email

This is the second column name fix for the pagination migration.
First fix was for email_api_tokens (owner_email -> user_email).

Addresses upgrade path from 0.8.0 -> 0.9.0 (unreleased)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Correct field name for tool reachability status in pagination templates

The pagination templates were using 'tool.gatewayReachable' which doesn't
exist in the serialized JSON. The correct field name is 'tool.reachable'.

This caused all tools to show as 'Offline' with the message 'The host
gateway for this tool is unreachable' even when the MCP Server status
was Active.

Root Cause:
- ToolRead schema has 'reachable: bool' field
- BaseModelWithConfigDict uses alias_generator=to_camel_case
- Since 'reachable' has no underscores, it stays as 'reachable' in JSON
- Templates incorrectly expected 'gatewayReachable' instead

Changes:
- tools_partial.html: tool.gatewayReachable β†’ tool.reachable (1 occurrence)
- tools_with_pagination.html: tool.gatewayReachable β†’ tool.reachable (2 occurrences)

The 'reachable' field was originally added in PR #215 for gateway/tool
health monitoring and works correctly throughout the codebase except
for these template references introduced in the pagination PR.

Fixes incorrect status display in Tools UI pagination.

Signed-off-by: Mihai Criveti <[email protected]>

---------

Signed-off-by: Mihai Criveti <[email protected]>
Signed-off-by: Madhav Kandukuri <[email protected]>
Co-authored-by: Mihai Criveti <[email protected]>
crivetimihai added a commit that referenced this pull request Oct 19, 2025
* Tools pagination

Signed-off-by: Mihai Criveti <[email protected]>

* Tools pagination still broken

Signed-off-by: Mihai Criveti <[email protected]>

* Tools pagination still broken

Signed-off-by: Mihai Criveti <[email protected]>

* copied from rest-api-and-ui-pagination-v2

Signed-off-by: Madhav Kandukuri <[email protected]>

* Use jsonable_encoder for tools_pydantic
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix console error from tools_partial
Signed-off-by: Madhav Kandukuri <[email protected]>

* Multiple changes
Signed-off-by: Madhav Kandukuri <[email protected]>ok, d

* Swapping working but styling needs update
Signed-off-by: Madhav Kandukuri <[email protected]>

* Some working versions
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix everything
Signed-off-by: Madhav Kandukuri <[email protected]>

* Use tools_partial for first load
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix minor bug
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix failing tests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Remove extra logging
Signed-off-by: Madhav Kandukuri <[email protected]>

* add uv.lock from main

Signed-off-by: Madhav Kandukuri <[email protected]>

* Fixes revision in latest alembic script
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix one doctest
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix doctest
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix some doctests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Minor updates to pagination doctests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Mock db in doctests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix doctests in pagination
Signed-off-by: Madhav Kandukuri <[email protected]>

* Add trailing line in tools_with_pagination
Signed-off-by: Madhav Kandukuri <[email protected]>

* pre-commit fixes
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix key navigation in pagination
Signed-off-by: Madhav Kandukuri <[email protected]>

* pylint fix
Signed-off-by: Madhav Kandukuri <[email protected]>

* fix: Update pagination migration to follow correct alembic chain

The pagination indexes migration now correctly follows e5a59c16e041
(unique constraints for prompts/resources) which was merged to main
after this PR branch was created.

Migration chain:
- e5a59c16e041 (main)
- g1a2b3c4d5e6 (pagination indexes) <- this PR
- 9aaa90ad26d9 (output schema)
- 8a2934be50c0 (rest pass api)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Restore correct down_revision for output_schema migration

The output_schema migration (9aaa90ad26d9) should point to 9c99ec6872ed,
not to g1a2b3c4d5e6, to avoid creating a circular dependency.

Correct migration chain:
- 9c99ec6872ed (token usage logs fix)
- 9aaa90ad26d9 (output schema) <- restored
- 8a2934be50c0 (rest pass api)
- e5a59c16e041 (unique constraints)
- g1a2b3c4d5e6 (pagination indexes) <- this PR

Signed-off-by: Mihai Criveti <[email protected]>

* docs: Update CHANGELOG.md with REST API and UI pagination feature

Add comprehensive documentation for PR #1277 pagination feature including:
- Paginated REST API endpoints with backward compatibility
- Database indexes for efficient pagination queries
- HTMX-based UI pagination with keyboard navigation
- 11 new pagination configuration environment variables
- Pagination utilities module for offset and cursor-based pagination
- 1,089+ lines of test coverage

Addresses #1224

Signed-off-by: Mihai Criveti <[email protected]>

* fix: PostgreSQL foreign key handling in e5a59c16e041 migration

The unique constraint migration was failing on PostgreSQL during upgrades
from 0.8.0 because it attempted to drop tables that have foreign key
dependencies without first dropping those constraints.

Changes:
- Detect PostgreSQL and query information_schema for incoming FKs
- Drop foreign key constraints before dropping/recreating tables
- Recreate foreign key constraints after table recreation
- Apply same fix to both upgrade() and downgrade() functions

This fixes the error:
  'cannot drop table prompts because other objects depend on it'

Affected tables: prompts, resources, a2a_agents
Dependent tables: server_prompt_association, prompt_metrics, etc.

Tested on: SQLite (unchanged behavior), PostgreSQL (now works)

Addresses upgrade path from 0.8.0 -> 0.9.0 (unreleased)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Use correct column name in email_api_tokens pagination index

The email_api_tokens table has a column 'user_email', not 'owner_email'.
The pagination migration was trying to create an index on a non-existent
column, causing PostgreSQL migrations to fail.

Changes:
- Renamed index from 'ix_email_api_tokens_owner_created_at' to
  'ix_email_api_tokens_user_email_created_at'
- Updated both upgrade() and downgrade() functions
- Index now correctly references the 'user_email' column

Error fixed:
  'column "owner_email" does not exist'

Table schema reference:
  mcpgateway/alembic/versions/cfc3d6aa0fb2 (line 208)
  defines email_api_tokens.user_email

Addresses upgrade path from 0.8.0 -> 0.9.0 (unreleased)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Use correct column name in email_auth_events pagination index

The email_auth_events table has a column 'user_email', not 'email'.
The pagination migration was trying to create an index on a non-existent
column, causing PostgreSQL migrations to fail.

Changes:
- Renamed index from 'ix_email_auth_events_user_timestamp' to
  'ix_email_auth_events_user_email_timestamp'
- Changed column from 'email' to 'user_email' in index definition
- Updated both upgrade() and downgrade() functions

Error fixed:
  'column "email" does not exist'

Table schema reference:
  mcpgateway/alembic/versions/cfc3d6aa0fb2 (line 100)
  defines email_auth_events.user_email

This is the second column name fix for the pagination migration.
First fix was for email_api_tokens (owner_email -> user_email).

Addresses upgrade path from 0.8.0 -> 0.9.0 (unreleased)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Correct field name for tool reachability status in pagination templates

The pagination templates were using 'tool.gatewayReachable' which doesn't
exist in the serialized JSON. The correct field name is 'tool.reachable'.

This caused all tools to show as 'Offline' with the message 'The host
gateway for this tool is unreachable' even when the MCP Server status
was Active.

Root Cause:
- ToolRead schema has 'reachable: bool' field
- BaseModelWithConfigDict uses alias_generator=to_camel_case
- Since 'reachable' has no underscores, it stays as 'reachable' in JSON
- Templates incorrectly expected 'gatewayReachable' instead

Changes:
- tools_partial.html: tool.gatewayReachable β†’ tool.reachable (1 occurrence)
- tools_with_pagination.html: tool.gatewayReachable β†’ tool.reachable (2 occurrences)

The 'reachable' field was originally added in PR #215 for gateway/tool
health monitoring and works correctly throughout the codebase except
for these template references introduced in the pagination PR.

Fixes incorrect status display in Tools UI pagination.

Signed-off-by: Mihai Criveti <[email protected]>

---------

Signed-off-by: Mihai Criveti <[email protected]>
Signed-off-by: Madhav Kandukuri <[email protected]>
Co-authored-by: Mihai Criveti <[email protected]>
p4yl04d3r pushed a commit to p4yl04d3r/mcp-context-forge that referenced this pull request Nov 19, 2025
* Tools pagination

Signed-off-by: Mihai Criveti <[email protected]>

* Tools pagination still broken

Signed-off-by: Mihai Criveti <[email protected]>

* Tools pagination still broken

Signed-off-by: Mihai Criveti <[email protected]>

* copied from rest-api-and-ui-pagination-v2

Signed-off-by: Madhav Kandukuri <[email protected]>

* Use jsonable_encoder for tools_pydantic
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix console error from tools_partial
Signed-off-by: Madhav Kandukuri <[email protected]>

* Multiple changes
Signed-off-by: Madhav Kandukuri <[email protected]>ok, d

* Swapping working but styling needs update
Signed-off-by: Madhav Kandukuri <[email protected]>

* Some working versions
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix everything
Signed-off-by: Madhav Kandukuri <[email protected]>

* Use tools_partial for first load
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix minor bug
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix failing tests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Remove extra logging
Signed-off-by: Madhav Kandukuri <[email protected]>

* add uv.lock from main

Signed-off-by: Madhav Kandukuri <[email protected]>

* Fixes revision in latest alembic script
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix one doctest
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix doctest
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix some doctests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Minor updates to pagination doctests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Mock db in doctests
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix doctests in pagination
Signed-off-by: Madhav Kandukuri <[email protected]>

* Add trailing line in tools_with_pagination
Signed-off-by: Madhav Kandukuri <[email protected]>

* pre-commit fixes
Signed-off-by: Madhav Kandukuri <[email protected]>

* Fix key navigation in pagination
Signed-off-by: Madhav Kandukuri <[email protected]>

* pylint fix
Signed-off-by: Madhav Kandukuri <[email protected]>

* fix: Update pagination migration to follow correct alembic chain

The pagination indexes migration now correctly follows e5a59c16e041
(unique constraints for prompts/resources) which was merged to main
after this PR branch was created.

Migration chain:
- e5a59c16e041 (main)
- g1a2b3c4d5e6 (pagination indexes) <- this PR
- 9aaa90ad26d9 (output schema)
- 8a2934be50c0 (rest pass api)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Restore correct down_revision for output_schema migration

The output_schema migration (9aaa90ad26d9) should point to 9c99ec6872ed,
not to g1a2b3c4d5e6, to avoid creating a circular dependency.

Correct migration chain:
- 9c99ec6872ed (token usage logs fix)
- 9aaa90ad26d9 (output schema) <- restored
- 8a2934be50c0 (rest pass api)
- e5a59c16e041 (unique constraints)
- g1a2b3c4d5e6 (pagination indexes) <- this PR

Signed-off-by: Mihai Criveti <[email protected]>

* docs: Update CHANGELOG.md with REST API and UI pagination feature

Add comprehensive documentation for PR IBM#1277 pagination feature including:
- Paginated REST API endpoints with backward compatibility
- Database indexes for efficient pagination queries
- HTMX-based UI pagination with keyboard navigation
- 11 new pagination configuration environment variables
- Pagination utilities module for offset and cursor-based pagination
- 1,089+ lines of test coverage

Addresses IBM#1224

Signed-off-by: Mihai Criveti <[email protected]>

* fix: PostgreSQL foreign key handling in e5a59c16e041 migration

The unique constraint migration was failing on PostgreSQL during upgrades
from 0.8.0 because it attempted to drop tables that have foreign key
dependencies without first dropping those constraints.

Changes:
- Detect PostgreSQL and query information_schema for incoming FKs
- Drop foreign key constraints before dropping/recreating tables
- Recreate foreign key constraints after table recreation
- Apply same fix to both upgrade() and downgrade() functions

This fixes the error:
  'cannot drop table prompts because other objects depend on it'

Affected tables: prompts, resources, a2a_agents
Dependent tables: server_prompt_association, prompt_metrics, etc.

Tested on: SQLite (unchanged behavior), PostgreSQL (now works)

Addresses upgrade path from 0.8.0 -> 0.9.0 (unreleased)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Use correct column name in email_api_tokens pagination index

The email_api_tokens table has a column 'user_email', not 'owner_email'.
The pagination migration was trying to create an index on a non-existent
column, causing PostgreSQL migrations to fail.

Changes:
- Renamed index from 'ix_email_api_tokens_owner_created_at' to
  'ix_email_api_tokens_user_email_created_at'
- Updated both upgrade() and downgrade() functions
- Index now correctly references the 'user_email' column

Error fixed:
  'column "owner_email" does not exist'

Table schema reference:
  mcpgateway/alembic/versions/cfc3d6aa0fb2 (line 208)
  defines email_api_tokens.user_email

Addresses upgrade path from 0.8.0 -> 0.9.0 (unreleased)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Use correct column name in email_auth_events pagination index

The email_auth_events table has a column 'user_email', not 'email'.
The pagination migration was trying to create an index on a non-existent
column, causing PostgreSQL migrations to fail.

Changes:
- Renamed index from 'ix_email_auth_events_user_timestamp' to
  'ix_email_auth_events_user_email_timestamp'
- Changed column from 'email' to 'user_email' in index definition
- Updated both upgrade() and downgrade() functions

Error fixed:
  'column "email" does not exist'

Table schema reference:
  mcpgateway/alembic/versions/cfc3d6aa0fb2 (line 100)
  defines email_auth_events.user_email

This is the second column name fix for the pagination migration.
First fix was for email_api_tokens (owner_email -> user_email).

Addresses upgrade path from 0.8.0 -> 0.9.0 (unreleased)

Signed-off-by: Mihai Criveti <[email protected]>

* fix: Correct field name for tool reachability status in pagination templates

The pagination templates were using 'tool.gatewayReachable' which doesn't
exist in the serialized JSON. The correct field name is 'tool.reachable'.

This caused all tools to show as 'Offline' with the message 'The host
gateway for this tool is unreachable' even when the MCP Server status
was Active.

Root Cause:
- ToolRead schema has 'reachable: bool' field
- BaseModelWithConfigDict uses alias_generator=to_camel_case
- Since 'reachable' has no underscores, it stays as 'reachable' in JSON
- Templates incorrectly expected 'gatewayReachable' instead

Changes:
- tools_partial.html: tool.gatewayReachable β†’ tool.reachable (1 occurrence)
- tools_with_pagination.html: tool.gatewayReachable β†’ tool.reachable (2 occurrences)

The 'reachable' field was originally added in PR IBM#215 for gateway/tool
health monitoring and works correctly throughout the codebase except
for these template references introduced in the pagination PR.

Fixes incorrect status display in Tools UI pagination.

Signed-off-by: Mihai Criveti <[email protected]>

---------

Signed-off-by: Mihai Criveti <[email protected]>
Signed-off-by: Madhav Kandukuri <[email protected]>
Co-authored-by: Mihai Criveti <[email protected]>
Signed-off-by: p4yl04d3r <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request]: Add auto activation of mcp-server, when it goes up back again

4 participants