diff --git a/CHANGELOG.md b/CHANGELOG.md index 36a73474f..679f1ba62 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. ### Added +- Added catalogs route support to enable federated hierarchical catalog browsing and navigation in the STAC API. [#547](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/547) + ### Changed ### Fixed diff --git a/README.md b/README.md index f2a7f498e..106b8d2d8 100644 --- a/README.md +++ b/README.md @@ -92,6 +92,7 @@ This project is built on the following technologies: STAC, stac-fastapi, FastAPI - [Technologies](#technologies) - [Table of Contents](#table-of-contents) - [Collection Search Extensions](#collection-search-extensions) + - [Catalogs Route](#catalogs-route) - [Documentation & Resources](#documentation--resources) - [SFEOS STAC Viewer](#sfeos-stac-viewer) - [Package Structure](#package-structure) @@ -168,6 +169,8 @@ SFEOS provides enhanced collection search capabilities through two primary route - **GET/POST `/collections`**: The standard STAC endpoint with extended query parameters - **GET/POST `/collections-search`**: A custom endpoint that supports the same parameters, created to avoid conflicts with the STAC Transactions extension if enabled (which uses POST `/collections` for collection creation) +The `/collections-search` endpoint follows the [STAC API Collection Search Endpoint](https://github.com/Healy-Hyperspatial/stac-api-extensions-collection-search-endpoint) specification, which provides a dedicated, conflict-free mechanism for advanced collection searching. + These endpoints support advanced collection discovery features including: - **Sorting**: Sort collections by sortable fields using the `sortby` parameter @@ -227,6 +230,96 @@ These extensions make it easier to build user interfaces that display and naviga > **Important**: Adding keyword fields to make text fields sortable can significantly increase the index size, especially for large text fields. Consider the storage implications when deciding which fields to make sortable. +## Catalogs Route + +SFEOS supports federated hierarchical catalog browsing through the `/catalogs` endpoint, enabling users to navigate through STAC catalog structures in a tree-like fashion. This extension allows for organized discovery and browsing of collections and sub-catalogs. + +This implementation follows the [STAC API Catalogs Extension](https://github.com/Healy-Hyperspatial/stac-api-extensions-catalogs) specification, which enables a Federated STAC API architecture with a "Hub and Spoke" structure. + +### Features + +- **Hierarchical Navigation**: Browse catalogs and sub-catalogs in a parent-child relationship structure +- **Collection Discovery**: Access collections within specific catalog contexts +- **STAC API Compliance**: Follows STAC specification for catalog objects and linking +- **Flexible Querying**: Support for standard STAC API query parameters when browsing collections within catalogs + +### Endpoints + +- **GET `/catalogs`**: Retrieve the root catalog and its child catalogs +- **POST `/catalogs`**: Create a new catalog (requires appropriate permissions) +- **GET `/catalogs/{catalog_id}`**: Retrieve a specific catalog and its children +- **DELETE `/catalogs/{catalog_id}`**: Delete a catalog (optionally cascade delete all collections) +- **GET `/catalogs/{catalog_id}/collections`**: Retrieve collections within a specific catalog +- **POST `/catalogs/{catalog_id}/collections`**: Create a new collection within a specific catalog +- **GET `/catalogs/{catalog_id}/collections/{collection_id}`**: Retrieve a specific collection within a catalog +- **GET `/catalogs/{catalog_id}/collections/{collection_id}/items`**: Retrieve items within a collection in a catalog context +- **GET `/catalogs/{catalog_id}/collections/{collection_id}/items/{item_id}`**: Retrieve a specific item within a catalog context + +### Usage Examples + +```bash +# Get root catalog +curl "http://localhost:8081/catalogs" + +# Get specific catalog +curl "http://localhost:8081/catalogs/earth-observation" + +# Get collections in a catalog +curl "http://localhost:8081/catalogs/earth-observation/collections" + +# Create a new collection within a catalog +curl -X POST "http://localhost:8081/catalogs/earth-observation/collections" \ + -H "Content-Type: application/json" \ + -d '{ + "id": "landsat-9", + "type": "Collection", + "stac_version": "1.0.0", + "description": "Landsat 9 satellite imagery collection", + "title": "Landsat 9", + "license": "MIT", + "extent": { + "spatial": {"bbox": [[-180, -90, 180, 90]]}, + "temporal": {"interval": [["2021-09-27T00:00:00Z", null]]} + } + }' + +# Get specific collection within a catalog +curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2" + +# Get items in a collection within a catalog +curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2/items" + +# Get specific item within a catalog +curl "http://localhost:8081/catalogs/earth-observation/collections/sentinel-2/items/S2A_20231015_123456" + +# Delete a catalog (collections remain intact) +curl -X DELETE "http://localhost:8081/catalogs/earth-observation" + +# Delete a catalog and all its collections (cascade delete) +curl -X DELETE "http://localhost:8081/catalogs/earth-observation?cascade=true" +``` + +### Delete Catalog Parameters + +The DELETE endpoint supports the following query parameter: + +- **`cascade`** (boolean, default: `false`): + - If `false`: Only deletes the catalog. Collections linked to the catalog remain in the database but lose their catalog link. + - If `true`: Deletes the catalog AND all collections linked to it. Use with caution as this is a destructive operation. + +### Response Structure + +Catalog responses include: +- **Catalog metadata**: ID, title, description, and other catalog properties +- **Child catalogs**: Links to sub-catalogs for hierarchical navigation +- **Collections**: Links to collections contained within the catalog +- **STAC links**: Properly formatted STAC API links for navigation + +This feature enables building user interfaces that provide organized, hierarchical browsing of STAC collections, making it easier for users to discover and navigate through large collections organized by theme, provider, or any other categorization scheme. + +> **Configuration**: The catalogs route can be enabled or disabled by setting the `ENABLE_CATALOGS_ROUTE` environment variable to `true` or `false`. By default, this endpoint is **disabled**. + + ## Package Structure This project is organized into several packages, each with a specific purpose: @@ -360,6 +453,7 @@ You can customize additional settings in your `.env` file: | `ENABLE_COLLECTIONS_SEARCH` | Enable collection search extensions (sort, fields, free text search, structured filtering, and datetime filtering) on the core `/collections` endpoint. | `true` | Optional | | `ENABLE_COLLECTIONS_SEARCH_ROUTE` | Enable the custom `/collections-search` endpoint (both GET and POST methods). When disabled, the custom endpoint will not be available, but collection search extensions will still be available on the core `/collections` endpoint if `ENABLE_COLLECTIONS_SEARCH` is true. | `false` | Optional | | `ENABLE_TRANSACTIONS_EXTENSIONS` | Enables or disables the Transactions and Bulk Transactions API extensions. This is useful for deployments where mutating the catalog via the API should be prevented. If set to `true`, the POST `/collections` route for search will be unavailable in the API. | `true` | Optional | +| `ENABLE_CATALOGS_ROUTE` | Enable the `/catalogs` endpoint for federated hierarchical catalog browsing and navigation. When enabled, provides access to federated STAC API architecture with hub-and-spoke pattern. | `false` | Optional | | `STAC_GLOBAL_COLLECTION_MAX_LIMIT` | Configures the maximum number of STAC collections that can be returned in a single search request. | N/A | Optional | | `STAC_DEFAULT_COLLECTION_LIMIT` | Configures the default number of STAC collections returned when no limit parameter is specified in the request. | `300` | Optional | | `STAC_GLOBAL_ITEM_MAX_LIMIT` | Configures the maximum number of STAC items that can be returned in a single search request. | N/A | Optional | diff --git a/compose.yml b/compose.yml index 2c1d7be38..00fb5938b 100644 --- a/compose.yml +++ b/compose.yml @@ -23,6 +23,7 @@ services: - BACKEND=elasticsearch - DATABASE_REFRESH=true - ENABLE_COLLECTIONS_SEARCH_ROUTE=true + - ENABLE_CATALOGS_ROUTE=true - REDIS_ENABLE=true - REDIS_HOST=redis - REDIS_PORT=6379 @@ -62,6 +63,7 @@ services: - BACKEND=opensearch - STAC_FASTAPI_RATE_LIMIT=200/minute - ENABLE_COLLECTIONS_SEARCH_ROUTE=true + - ENABLE_CATALOGS_ROUTE=true - REDIS_ENABLE=true - REDIS_HOST=redis - REDIS_PORT=6379 diff --git a/stac_fastapi/core/stac_fastapi/core/base_database_logic.py b/stac_fastapi/core/stac_fastapi/core/base_database_logic.py index 105fdf925..4f5434535 100644 --- a/stac_fastapi/core/stac_fastapi/core/base_database_logic.py +++ b/stac_fastapi/core/stac_fastapi/core/base_database_logic.py @@ -138,3 +138,39 @@ async def delete_collection( ) -> None: """Delete a collection from the database.""" pass + + @abc.abstractmethod + async def get_all_catalogs( + self, + token: Optional[str], + limit: int, + request: Any = None, + sort: Optional[List[Dict[str, Any]]] = None, + ) -> Tuple[List[Dict[str, Any]], Optional[str], Optional[int]]: + """Retrieve a list of catalogs from the database, supporting pagination. + + Args: + token (Optional[str]): The pagination token. + limit (int): The number of results to return. + request (Any, optional): The FastAPI request object. Defaults to None. + sort (Optional[List[Dict[str, Any]]], optional): Optional sort parameter. Defaults to None. + + Returns: + A tuple of (catalogs, next pagination token if any, optional count). + """ + pass + + @abc.abstractmethod + async def create_catalog(self, catalog: Dict, refresh: bool = False) -> None: + """Create a catalog in the database.""" + pass + + @abc.abstractmethod + async def find_catalog(self, catalog_id: str) -> Dict: + """Find a catalog in the database.""" + pass + + @abc.abstractmethod + async def delete_catalog(self, catalog_id: str, refresh: bool = False) -> None: + """Delete a catalog from the database.""" + pass diff --git a/stac_fastapi/core/stac_fastapi/core/core.py b/stac_fastapi/core/stac_fastapi/core/core.py index e099e3d48..d04b66062 100644 --- a/stac_fastapi/core/stac_fastapi/core/core.py +++ b/stac_fastapi/core/stac_fastapi/core/core.py @@ -24,7 +24,11 @@ from stac_fastapi.core.base_settings import ApiBaseSettings from stac_fastapi.core.datetime_utils import format_datetime_range from stac_fastapi.core.models.links import PagingLinks -from stac_fastapi.core.serializers import CollectionSerializer, ItemSerializer +from stac_fastapi.core.serializers import ( + CatalogSerializer, + CollectionSerializer, + ItemSerializer, +) from stac_fastapi.core.session import Session from stac_fastapi.core.utilities import filter_fields, get_bool_env from stac_fastapi.extensions.core.transaction import AsyncBaseTransactionsClient @@ -81,12 +85,24 @@ class CoreClient(AsyncBaseCoreClient): collection_serializer: Type[CollectionSerializer] = attr.ib( default=CollectionSerializer ) + catalog_serializer: Type[CatalogSerializer] = attr.ib(default=CatalogSerializer) post_request_model = attr.ib(default=BaseSearchPostRequest) stac_version: str = attr.ib(default=STAC_VERSION) landing_page_id: str = attr.ib(default="stac-fastapi") title: str = attr.ib(default="stac-fastapi") description: str = attr.ib(default="stac-fastapi") + def extension_is_enabled(self, extension_name: str) -> bool: + """Check if an extension is enabled by checking self.extensions. + + Args: + extension_name: Name of the extension class to check for. + + Returns: + True if the extension is in self.extensions, False otherwise. + """ + return any(ext.__class__.__name__ == extension_name for ext in self.extensions) + def _landing_page( self, base_url: str, @@ -150,6 +166,7 @@ async def landing_page(self, **kwargs) -> stac_types.LandingPage: API landing page, serving as an entry point to the API. """ request: Request = kwargs["request"] + base_url = get_base_url(request) landing_page = self._landing_page( base_url=base_url, @@ -207,6 +224,16 @@ async def landing_page(self, **kwargs) -> stac_types.LandingPage: ] ) + if self.extension_is_enabled("CatalogsExtension"): + landing_page["links"].append( + { + "rel": "catalogs", + "type": "application/json", + "title": "Catalogs", + "href": urljoin(base_url, "catalogs"), + } + ) + # Add OpenAPI URL landing_page["links"].append( { diff --git a/stac_fastapi/core/stac_fastapi/core/extensions/__init__.py b/stac_fastapi/core/stac_fastapi/core/extensions/__init__.py index 9216e8ec0..f74d53ae3 100644 --- a/stac_fastapi/core/stac_fastapi/core/extensions/__init__.py +++ b/stac_fastapi/core/stac_fastapi/core/extensions/__init__.py @@ -1,5 +1,6 @@ """elasticsearch extensions modifications.""" +from .catalogs import CatalogsExtension from .collections_search import CollectionsSearchEndpointExtension from .query import Operator, QueryableTypes, QueryExtension @@ -8,4 +9,5 @@ "QueryableTypes", "QueryExtension", "CollectionsSearchEndpointExtension", + "CatalogsExtension", ] diff --git a/stac_fastapi/core/stac_fastapi/core/extensions/catalogs.py b/stac_fastapi/core/stac_fastapi/core/extensions/catalogs.py new file mode 100644 index 000000000..872fc54d1 --- /dev/null +++ b/stac_fastapi/core/stac_fastapi/core/extensions/catalogs.py @@ -0,0 +1,855 @@ +"""Catalogs extension.""" + +import logging +from typing import List, Optional, Type +from urllib.parse import urlencode, urlparse + +import attr +from fastapi import APIRouter, FastAPI, HTTPException, Query, Request +from fastapi.responses import JSONResponse +from stac_pydantic import Collection +from starlette.responses import Response +from typing_extensions import TypedDict + +from stac_fastapi.core.models import Catalog +from stac_fastapi.sfeos_helpers.mappings import COLLECTIONS_INDEX +from stac_fastapi.types import stac as stac_types +from stac_fastapi.types.core import BaseCoreClient +from stac_fastapi.types.extension import ApiExtension + +logger = logging.getLogger(__name__) + + +class Catalogs(TypedDict, total=False): + """Catalogs endpoint response. + + Similar to Collections but for catalogs. + """ + + catalogs: List[Catalog] + links: List[dict] + numberMatched: Optional[int] + numberReturned: Optional[int] + + +@attr.s +class CatalogsExtension(ApiExtension): + """Catalogs Extension. + + The Catalogs extension adds a /catalogs endpoint that returns a list of all catalogs + in the database, similar to how /collections returns a list of collections. + """ + + client: BaseCoreClient = attr.ib(default=None) + settings: dict = attr.ib(default=attr.Factory(dict)) + conformance_classes: List[str] = attr.ib(default=attr.Factory(list)) + router: APIRouter = attr.ib(default=attr.Factory(APIRouter)) + response_class: Type[Response] = attr.ib(default=JSONResponse) + + def register(self, app: FastAPI, settings=None) -> None: + """Register the extension with a FastAPI application. + + Args: + app: target FastAPI application. + settings: extension settings (unused for now). + """ + self.settings = settings or {} + + self.router.add_api_route( + path="/catalogs", + endpoint=self.catalogs, + methods=["GET"], + response_model=Catalogs, + response_class=self.response_class, + summary="Get All Catalogs", + description="Returns a list of all catalogs in the database.", + tags=["Catalogs"], + ) + + # Add endpoint for creating catalogs + self.router.add_api_route( + path="/catalogs", + endpoint=self.create_catalog, + methods=["POST"], + response_model=Catalog, + response_class=self.response_class, + status_code=201, + summary="Create Catalog", + description="Create a new STAC catalog.", + tags=["Catalogs"], + ) + + # Add endpoint for getting individual catalogs + self.router.add_api_route( + path="/catalogs/{catalog_id}", + endpoint=self.get_catalog, + methods=["GET"], + response_model=Catalog, + response_class=self.response_class, + summary="Get Catalog", + description="Get a specific STAC catalog by ID.", + tags=["Catalogs"], + ) + + # Add endpoint for getting collections in a catalog + self.router.add_api_route( + path="/catalogs/{catalog_id}/collections", + endpoint=self.get_catalog_collections, + methods=["GET"], + response_model=stac_types.Collections, + response_class=self.response_class, + summary="Get Catalog Collections", + description="Get collections linked from a specific catalog.", + tags=["Catalogs"], + ) + + # Add endpoint for creating collections in a catalog + self.router.add_api_route( + path="/catalogs/{catalog_id}/collections", + endpoint=self.create_catalog_collection, + methods=["POST"], + response_model=stac_types.Collection, + response_class=self.response_class, + status_code=201, + summary="Create Catalog Collection", + description="Create a new collection and link it to a specific catalog.", + tags=["Catalogs"], + ) + + # Add endpoint for deleting a catalog + self.router.add_api_route( + path="/catalogs/{catalog_id}", + endpoint=self.delete_catalog, + methods=["DELETE"], + response_class=self.response_class, + status_code=204, + summary="Delete Catalog", + description="Delete a catalog. Optionally cascade delete all collections in the catalog.", + tags=["Catalogs"], + ) + + # Add endpoint for getting a specific collection in a catalog + self.router.add_api_route( + path="/catalogs/{catalog_id}/collections/{collection_id}", + endpoint=self.get_catalog_collection, + methods=["GET"], + response_model=stac_types.Collection, + response_class=self.response_class, + summary="Get Catalog Collection", + description="Get a specific collection from a catalog.", + tags=["Catalogs"], + ) + + # Add endpoint for getting items in a collection within a catalog + self.router.add_api_route( + path="/catalogs/{catalog_id}/collections/{collection_id}/items", + endpoint=self.get_catalog_collection_items, + methods=["GET"], + response_model=stac_types.ItemCollection, + response_class=self.response_class, + summary="Get Catalog Collection Items", + description="Get items from a collection in a catalog.", + tags=["Catalogs"], + ) + + # Add endpoint for getting a specific item in a collection within a catalog + self.router.add_api_route( + path="/catalogs/{catalog_id}/collections/{collection_id}/items/{item_id}", + endpoint=self.get_catalog_collection_item, + methods=["GET"], + response_model=stac_types.Item, + response_class=self.response_class, + summary="Get Catalog Collection Item", + description="Get a specific item from a collection in a catalog.", + tags=["Catalogs"], + ) + + app.include_router(self.router, tags=["Catalogs"]) + + async def catalogs( + self, + request: Request, + limit: Optional[int] = Query( + 10, + ge=1, + description=( + "The maximum number of catalogs to return (page size). Defaults to 10." + ), + ), + token: Optional[str] = Query( + None, + description="Pagination token for the next page of results", + ), + ) -> Catalogs: + """Get all catalogs with pagination support. + + Args: + request: Request object. + limit: The maximum number of catalogs to return (page size). Defaults to 10. + token: Pagination token for the next page of results. + + Returns: + Catalogs object containing catalogs and pagination links. + """ + base_url = str(request.base_url) + + # Get all catalogs from database with pagination + catalogs, next_token, _ = await self.client.database.get_all_catalogs( + token=token, + limit=limit, + request=request, + sort=[{"field": "id", "direction": "asc"}], + ) + + # Convert database catalogs to STAC format + catalog_stac_objects = [] + for catalog in catalogs: + catalog_stac = self.client.catalog_serializer.db_to_stac(catalog, request) + catalog_stac_objects.append(catalog_stac) + + # Create pagination links + links = [ + {"rel": "root", "type": "application/json", "href": base_url}, + {"rel": "parent", "type": "application/json", "href": base_url}, + {"rel": "self", "type": "application/json", "href": str(request.url)}, + ] + + # Add next link if there are more pages + if next_token: + query_params = {"limit": limit, "token": next_token} + next_link = { + "rel": "next", + "href": f"{base_url}catalogs?{urlencode(query_params)}", + "type": "application/json", + "title": "Next page of catalogs", + } + links.append(next_link) + + # Return Catalogs object with catalogs + return Catalogs( + catalogs=catalog_stac_objects, + links=links, + numberReturned=len(catalog_stac_objects), + ) + + async def create_catalog(self, catalog: Catalog, request: Request) -> Catalog: + """Create a new catalog. + + Args: + catalog: The catalog to create. + request: Request object. + + Returns: + The created catalog. + """ + # Convert STAC catalog to database format + db_catalog = self.client.catalog_serializer.stac_to_db(catalog, request) + + # Convert to dict and ensure type is set to Catalog + db_catalog_dict = db_catalog.model_dump() + db_catalog_dict["type"] = "Catalog" + + # Create the catalog in the database with refresh to ensure immediate availability + await self.client.database.create_catalog(db_catalog_dict, refresh=True) + + # Return the created catalog + return catalog + + async def get_catalog(self, catalog_id: str, request: Request) -> Catalog: + """Get a specific catalog by ID. + + Args: + catalog_id: The ID of the catalog to retrieve. + request: Request object. + + Returns: + The requested catalog. + """ + try: + # Get the catalog from the database + db_catalog = await self.client.database.find_catalog(catalog_id) + + # Convert to STAC format + catalog = self.client.catalog_serializer.db_to_stac(db_catalog, request) + + return catalog + except Exception: + raise HTTPException( + status_code=404, detail=f"Catalog {catalog_id} not found" + ) + + async def delete_catalog( + self, + catalog_id: str, + request: Request, + cascade: bool = Query( + False, + description="If true, delete all collections linked to this catalog. If false, only delete the catalog.", + ), + ) -> None: + """Delete a catalog. + + Args: + catalog_id: The ID of the catalog to delete. + request: Request object. + cascade: If true, delete all collections linked to this catalog. + If false, only delete the catalog. + + Returns: + None (204 No Content) + + Raises: + HTTPException: If the catalog is not found. + """ + try: + # Get the catalog to verify it exists and get its collections + db_catalog = await self.client.database.find_catalog(catalog_id) + catalog = self.client.catalog_serializer.db_to_stac(db_catalog, request) + + # Extract collection IDs from catalog links + collection_ids = [] + if hasattr(catalog, "links") and catalog.links: + for link in catalog.links: + rel = ( + link.get("rel") + if hasattr(link, "get") + else getattr(link, "rel", None) + ) + if rel == "child": + href = ( + link.get("href", "") + if hasattr(link, "get") + else getattr(link, "href", "") + ) + if href and "/collections/" in href: + # Extract collection ID from href + collection_id = href.split("/collections/")[-1].split("/")[ + 0 + ] + if collection_id: + collection_ids.append(collection_id) + + if cascade: + # Delete each collection + for coll_id in collection_ids: + try: + await self.client.database.delete_collection(coll_id) + logger.info( + f"Deleted collection {coll_id} as part of cascade delete for catalog {catalog_id}" + ) + except Exception as e: + error_msg = str(e) + if "not found" in error_msg.lower(): + logger.debug( + f"Collection {coll_id} not found, skipping (may have been deleted elsewhere)" + ) + else: + logger.warning( + f"Failed to delete collection {coll_id}: {e}" + ) + else: + # Remove catalog link from each collection (orphan them) + for coll_id in collection_ids: + try: + collection = await self.client.get_collection( + coll_id, request=request + ) + # Remove the catalog link from the collection + if hasattr(collection, "links"): + collection.links = [ + link + for link in collection.links + if not ( + getattr(link, "rel", None) == "catalog" + and catalog_id in getattr(link, "href", "") + ) + ] + elif isinstance(collection, dict): + collection["links"] = [ + link + for link in collection.get("links", []) + if not ( + link.get("rel") == "catalog" + and catalog_id in link.get("href", "") + ) + ] + + # Update the collection in the database + collection_dict = ( + collection.model_dump(mode="json") + if hasattr(collection, "model_dump") + else collection + ) + collection_db = ( + self.client.database.collection_serializer.stac_to_db( + collection_dict, request + ) + ) + await self.client.database.client.index( + index=COLLECTIONS_INDEX, + id=coll_id, + body=collection_db.model_dump() + if hasattr(collection_db, "model_dump") + else collection_db, + refresh=True, + ) + logger.info(f"Removed catalog link from collection {coll_id}") + except Exception as e: + error_msg = str(e) + if "not found" in error_msg.lower(): + logger.debug( + f"Collection {coll_id} not found, skipping (may have been deleted elsewhere)" + ) + else: + logger.warning( + f"Failed to remove catalog link from collection {coll_id}: {e}" + ) + + # Delete the catalog + await self.client.database.delete_catalog(catalog_id) + logger.info(f"Deleted catalog {catalog_id}") + + except Exception as e: + error_msg = str(e) + if "not found" in error_msg.lower(): + raise HTTPException( + status_code=404, detail=f"Catalog {catalog_id} not found" + ) + logger.error(f"Error deleting catalog {catalog_id}: {e}") + raise HTTPException( + status_code=500, + detail=f"Failed to delete catalog: {str(e)}", + ) + + async def get_catalog_collections( + self, catalog_id: str, request: Request + ) -> stac_types.Collections: + """Get collections linked from a specific catalog. + + Args: + catalog_id: The ID of the catalog. + request: Request object. + + Returns: + Collections object containing collections linked from the catalog. + """ + try: + # Get the catalog from the database + db_catalog = await self.client.database.find_catalog(catalog_id) + + # Convert to STAC format to access links + catalog = self.client.catalog_serializer.db_to_stac(db_catalog, request) + + # Extract collection IDs from catalog links + # + # FRAGILE IMPLEMENTATION WARNING: + # This approach relies on parsing URL patterns to determine catalog-collection relationships. + # This is fragile and will break if: + # - URLs don't follow the expected /collections/{id} pattern + # - Base URLs contain /collections/ in other segments + # - Relative links are used instead of absolute URLs + # - Links have trailing slashes or query parameters + # + # TODO: In a future version, this should be replaced with a proper database relationship + # (e.g., parent_catalog_id field on Collection documents) + # + collection_ids = [] + if hasattr(catalog, "links") and catalog.links: + base_url = str(request.base_url).rstrip("/") + base_path = urlparse(base_url).path.rstrip("/") + + for link in catalog.links: + rel = ( + link.get("rel") + if hasattr(link, "get") + else getattr(link, "rel", None) + ) + if rel in ["child", "item"]: + # Extract collection ID from href using proper URL parsing + href = ( + link.get("href", "") + if hasattr(link, "get") + else getattr(link, "href", "") + ) + if href: + try: + parsed_url = urlparse(href) + path = parsed_url.path.rstrip("/") + + # Resolve relative URLs against base URL + if not href.startswith(("http://", "https://")): + full_path = ( + f"{base_path}{path}" if path else base_path + ) + else: + # For absolute URLs, ensure they belong to our base domain + if parsed_url.netloc != urlparse(base_url).netloc: + continue + full_path = path + + # Look for collections endpoint at the end of the path + # This prevents false positives when /collections/ appears in base URL + collections_pattern = "/collections/" + if collections_pattern in full_path: + # Find the LAST occurrence of /collections/ to avoid base URL conflicts + last_collections_pos = full_path.rfind( + collections_pattern + ) + if last_collections_pos != -1: + # Extract everything after the last /collections/ + after_collections = full_path[ + last_collections_pos + + len(collections_pattern) : + ] + + # Handle cases where there might be additional path segments + # We only want the immediate collection ID + collection_id = ( + after_collections.split("/")[0] + if after_collections + else None + ) + + if ( + collection_id + and collection_id not in collection_ids + ): + collection_ids.append(collection_id) + + except Exception: + # If URL parsing fails, skip this link + continue + + # Fetch the collections + collections = [] + for coll_id in collection_ids: + try: + collection = await self.client.get_collection( + coll_id, request=request + ) + collections.append(collection) + except HTTPException as e: + # Only skip collections that are not found (404) + if e.status_code == 404: + logger.debug(f"Collection {coll_id} not found, skipping") + continue + else: + # Re-raise other HTTP exceptions (5xx server errors, etc.) + logger.error(f"HTTP error retrieving collection {coll_id}: {e}") + raise + except Exception as e: + # Log unexpected errors and re-raise them + logger.error( + f"Unexpected error retrieving collection {coll_id}: {e}" + ) + raise + + # Return in Collections format + base_url = str(request.base_url) + return stac_types.Collections( + collections=collections, + links=[ + {"rel": "root", "type": "application/json", "href": base_url}, + {"rel": "parent", "type": "application/json", "href": base_url}, + { + "rel": "self", + "type": "application/json", + "href": f"{base_url}catalogs/{catalog_id}/collections", + }, + ], + ) + + except HTTPException: + # Re-raise HTTP exceptions as-is + raise + except Exception as e: + logger.error( + f"Error retrieving collections for catalog {catalog_id}: {e}", + exc_info=True, + ) + raise HTTPException( + status_code=404, detail=f"Catalog {catalog_id} not found" + ) + + async def create_catalog_collection( + self, catalog_id: str, collection: Collection, request: Request + ) -> stac_types.Collection: + """Create a new collection and link it to a specific catalog. + + Args: + catalog_id: The ID of the catalog to link the collection to. + collection: The collection to create. + request: Request object. + + Returns: + The created collection. + + Raises: + HTTPException: If the catalog is not found or collection creation fails. + """ + try: + # Verify the catalog exists + await self.client.database.find_catalog(catalog_id) + + # Create the collection using the same pattern as TransactionsClient.create_collection + # This handles the Collection model from stac_pydantic correctly + collection_dict = collection.model_dump(mode="json") + + # Add a link from the collection back to its parent catalog BEFORE saving to database + base_url = str(request.base_url) + catalog_link = { + "rel": "catalog", + "type": "application/json", + "href": f"{base_url}catalogs/{catalog_id}", + "title": catalog_id, + } + + # Add the catalog link to the collection dict + if "links" not in collection_dict: + collection_dict["links"] = [] + + # Check if the catalog link already exists + catalog_href = catalog_link["href"] + link_exists = any( + link.get("href") == catalog_href and link.get("rel") == "catalog" + for link in collection_dict.get("links", []) + ) + + if not link_exists: + collection_dict["links"].append(catalog_link) + + # Now convert to database format (this will process the links) + collection_db = self.client.database.collection_serializer.stac_to_db( + collection_dict, request + ) + await self.client.database.create_collection( + collection=collection_db, refresh=True + ) + + # Convert back to STAC format for the response + created_collection = self.client.database.collection_serializer.db_to_stac( + collection_db, + request, + extensions=[ + type(ext).__name__ for ext in self.client.database.extensions + ], + ) + + # Update the catalog to include a link to the new collection + await self._add_collection_to_catalog_links( + catalog_id, collection.id, request + ) + + return created_collection + + except HTTPException as e: + # Re-raise HTTP exceptions (e.g., catalog not found, collection validation errors) + raise e + except Exception as e: + # Check if this is a "not found" error from find_catalog + error_msg = str(e) + if "not found" in error_msg.lower(): + raise HTTPException(status_code=404, detail=error_msg) + + # Handle unexpected errors + logger.error(f"Error creating collection in catalog {catalog_id}: {e}") + raise HTTPException( + status_code=500, + detail=f"Failed to create collection in catalog: {str(e)}", + ) + + async def _add_collection_to_catalog_links( + self, catalog_id: str, collection_id: str, request: Request + ) -> None: + """Add a collection link to a catalog. + + This helper method updates a catalog's links to include a reference + to a collection by reindexing the updated catalog document. + + Args: + catalog_id: The ID of the catalog to update. + collection_id: The ID of the collection to link. + request: Request object for base URL construction. + """ + try: + # Get the current catalog + db_catalog = await self.client.database.find_catalog(catalog_id) + catalog = self.client.catalog_serializer.db_to_stac(db_catalog, request) + + # Create the collection link + base_url = str(request.base_url) + collection_link = { + "rel": "child", + "href": f"{base_url}collections/{collection_id}", + "type": "application/json", + "title": collection_id, + } + + # Add the link to the catalog if it doesn't already exist + catalog_links = ( + catalog.get("links") + if isinstance(catalog, dict) + else getattr(catalog, "links", None) + ) + if not catalog_links: + catalog_links = [] + if isinstance(catalog, dict): + catalog["links"] = catalog_links + else: + catalog.links = catalog_links + + # Check if the collection link already exists + collection_href = collection_link["href"] + link_exists = any( + ( + link.get("href") + if hasattr(link, "get") + else getattr(link, "href", None) + ) + == collection_href + for link in catalog_links + ) + + if not link_exists: + catalog_links.append(collection_link) + + # Update the catalog in the database by reindexing it + # Convert back to database format + updated_db_catalog = self.client.catalog_serializer.stac_to_db( + catalog, request + ) + updated_db_catalog_dict = ( + updated_db_catalog.model_dump() + if hasattr(updated_db_catalog, "model_dump") + else updated_db_catalog + ) + updated_db_catalog_dict["type"] = "Catalog" + + # Use the same approach as create_catalog to update the document + await self.client.database.client.index( + index=COLLECTIONS_INDEX, + id=catalog_id, + body=updated_db_catalog_dict, + refresh=True, + ) + + logger.info( + f"Updated catalog {catalog_id} to include link to collection {collection_id}" + ) + + except Exception as e: + logger.error( + f"Failed to update catalog {catalog_id} links: {e}", exc_info=True + ) + # Don't fail the entire operation if link update fails + # The collection was created successfully, just the catalog link is missing + + async def get_catalog_collection( + self, catalog_id: str, collection_id: str, request: Request + ) -> stac_types.Collection: + """Get a specific collection from a catalog. + + Args: + catalog_id: The ID of the catalog. + collection_id: The ID of the collection. + request: Request object. + + Returns: + The requested collection. + """ + # Verify the catalog exists + try: + await self.client.database.find_catalog(catalog_id) + except Exception: + raise HTTPException( + status_code=404, detail=f"Catalog {catalog_id} not found" + ) + + # Delegate to the core client's get_collection method + return await self.client.get_collection( + collection_id=collection_id, request=request + ) + + async def get_catalog_collection_items( + self, + catalog_id: str, + collection_id: str, + request: Request, + bbox: Optional[List[float]] = None, + datetime: Optional[str] = None, + limit: Optional[int] = None, + sortby: Optional[str] = None, + filter_expr: Optional[str] = None, + filter_lang: Optional[str] = None, + token: Optional[str] = None, + query: Optional[str] = None, + fields: Optional[List[str]] = None, + ) -> stac_types.ItemCollection: + """Get items from a collection in a catalog. + + Args: + catalog_id: The ID of the catalog. + collection_id: The ID of the collection. + request: Request object. + bbox: Optional bounding box filter. + datetime: Optional datetime or interval filter. + limit: Optional page size. + sortby: Optional sort specification. + filter_expr: Optional filter expression. + filter_lang: Optional filter language. + token: Optional pagination token. + query: Optional query string. + fields: Optional fields to include or exclude. + + Returns: + ItemCollection containing items from the collection. + """ + # Verify the catalog exists + try: + await self.client.database.find_catalog(catalog_id) + except Exception: + raise HTTPException( + status_code=404, detail=f"Catalog {catalog_id} not found" + ) + + # Delegate to the core client's item_collection method with all parameters + return await self.client.item_collection( + collection_id=collection_id, + request=request, + bbox=bbox, + datetime=datetime, + limit=limit, + sortby=sortby, + filter_expr=filter_expr, + filter_lang=filter_lang, + token=token, + query=query, + fields=fields, + ) + + async def get_catalog_collection_item( + self, catalog_id: str, collection_id: str, item_id: str, request: Request + ) -> stac_types.Item: + """Get a specific item from a collection in a catalog. + + Args: + catalog_id: The ID of the catalog. + collection_id: The ID of the collection. + item_id: The ID of the item. + request: Request object. + + Returns: + The requested item. + """ + # Verify the catalog exists + try: + await self.client.database.find_catalog(catalog_id) + except Exception: + raise HTTPException( + status_code=404, detail=f"Catalog {catalog_id} not found" + ) + + # Delegate to the core client's get_item method + return await self.client.get_item( + item_id=item_id, collection_id=collection_id, request=request + ) diff --git a/stac_fastapi/core/stac_fastapi/core/models/__init__.py b/stac_fastapi/core/stac_fastapi/core/models/__init__.py index d0748bcca..ce88bc57b 100644 --- a/stac_fastapi/core/stac_fastapi/core/models/__init__.py +++ b/stac_fastapi/core/stac_fastapi/core/models/__init__.py @@ -1 +1,27 @@ -"""stac_fastapi.elasticsearch.models module.""" +"""STAC models.""" + +from typing import Any, Dict, List, Optional + +from pydantic import BaseModel + + +class Catalog(BaseModel): + """STAC Catalog model.""" + + type: str = "Catalog" + stac_version: str + id: str + title: Optional[str] = None + description: Optional[str] = None + links: List[Dict[str, Any]] + stac_extensions: Optional[List[str]] = None + + +class PartialCatalog(BaseModel): + """Partial STAC Catalog model for updates.""" + + id: str + title: Optional[str] = None + description: Optional[str] = None + links: Optional[List[Dict[str, Any]]] = None + stac_extensions: Optional[List[str]] = None diff --git a/stac_fastapi/core/stac_fastapi/core/serializers.py b/stac_fastapi/core/stac_fastapi/core/serializers.py index 0e156ce4c..f935bff53 100644 --- a/stac_fastapi/core/stac_fastapi/core/serializers.py +++ b/stac_fastapi/core/stac_fastapi/core/serializers.py @@ -10,6 +10,7 @@ from starlette.requests import Request from stac_fastapi.core.datetime_utils import now_to_rfc3339_str +from stac_fastapi.core.models import Catalog from stac_fastapi.core.models.links import CollectionLinks from stac_fastapi.core.utilities import get_bool_env, get_excluded_from_items from stac_fastapi.types import stac as stac_types @@ -225,3 +226,57 @@ def db_to_stac( # Return the stac_types.Collection object return stac_types.Collection(**collection) + + +class CatalogSerializer(Serializer): + """Serialization methods for STAC catalogs.""" + + @classmethod + def stac_to_db(cls, catalog: Catalog, request: Request) -> Catalog: + """ + Transform STAC Catalog to database-ready STAC catalog. + + Args: + catalog: the STAC Catalog object to be transformed + request: the API request + + Returns: + Catalog: The database-ready STAC Catalog object. + """ + catalog = deepcopy(catalog) + catalog.links = resolve_links(catalog.links, str(request.base_url)) + return catalog + + @classmethod + def db_to_stac( + cls, catalog: dict, request: Request, extensions: Optional[List[str]] = [] + ) -> Catalog: + """Transform database model to STAC catalog. + + Args: + catalog (dict): The catalog data in dictionary form, extracted from the database. + request (Request): the API request + extensions: A list of the extension class names (`ext.__name__`) or all enabled STAC API extensions. + + Returns: + Catalog: The STAC catalog object. + """ + # Avoid modifying the input dict in-place + catalog = deepcopy(catalog) + + # Set defaults + catalog.setdefault("type", "Catalog") + catalog.setdefault("stac_extensions", []) + catalog.setdefault("stac_version", "") + catalog.setdefault("title", "") + catalog.setdefault("description", "") + + # Create the catalog links - for now, just resolve existing links + original_links = catalog.get("links", []) + if original_links: + catalog["links"] = resolve_links(original_links, str(request.base_url)) + else: + catalog["links"] = [] + + # Return the Catalog object + return Catalog(**catalog) diff --git a/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/app.py b/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/app.py index bcb870f39..bb2db9377 100644 --- a/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/app.py +++ b/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/app.py @@ -23,6 +23,7 @@ EsAggregationExtensionGetRequest, EsAggregationExtensionPostRequest, ) +from stac_fastapi.core.extensions.catalogs import CatalogsExtension from stac_fastapi.core.extensions.collections_search import ( CollectionsSearchEndpointExtension, ) @@ -65,11 +66,13 @@ ENABLE_COLLECTIONS_SEARCH_ROUTE = get_bool_env( "ENABLE_COLLECTIONS_SEARCH_ROUTE", default=False ) +ENABLE_CATALOGS_ROUTE = get_bool_env("ENABLE_CATALOGS_ROUTE", default=False) logger.info("TRANSACTIONS_EXTENSIONS is set to %s", TRANSACTIONS_EXTENSIONS) logger.info("ENABLE_COLLECTIONS_SEARCH is set to %s", ENABLE_COLLECTIONS_SEARCH) logger.info( "ENABLE_COLLECTIONS_SEARCH_ROUTE is set to %s", ENABLE_COLLECTIONS_SEARCH_ROUTE ) +logger.info("ENABLE_CATALOGS_ROUTE is set to %s", ENABLE_CATALOGS_ROUTE) settings = ElasticsearchSettings() session = Session.create_from_settings(settings) @@ -202,6 +205,22 @@ extensions.append(collections_search_endpoint_ext) +if ENABLE_CATALOGS_ROUTE: + catalogs_extension = CatalogsExtension( + client=CoreClient( + database=database_logic, + session=session, + post_request_model=collection_search_post_request_model, + landing_page_id=os.getenv("STAC_FASTAPI_LANDING_PAGE_ID", "stac-fastapi"), + ), + settings=settings, + conformance_classes=[ + "https://api.stacspec.org/v1.0.0-beta.1/catalogs-endpoint", + ], + ) + extensions.append(catalogs_extension) + + database_logic.extensions = [type(ext).__name__ for ext in extensions] post_request_model = create_post_request_model(search_extensions) diff --git a/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/database_logic.py b/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/database_logic.py index 35b6ae31a..759156073 100644 --- a/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/database_logic.py +++ b/stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/database_logic.py @@ -15,8 +15,13 @@ from fastapi import HTTPException from starlette.requests import Request +import stac_fastapi.sfeos_helpers.filter as filter_module from stac_fastapi.core.base_database_logic import BaseDatabaseLogic -from stac_fastapi.core.serializers import CollectionSerializer, ItemSerializer +from stac_fastapi.core.serializers import ( + CatalogSerializer, + CollectionSerializer, + ItemSerializer, +) from stac_fastapi.core.utilities import MAX_LIMIT, bbox2polygon, get_bool_env from stac_fastapi.elasticsearch.config import AsyncElasticsearchSettings from stac_fastapi.elasticsearch.config import ( @@ -27,7 +32,6 @@ PartialItem, PatchOperation, ) -from stac_fastapi.sfeos_helpers import filter as filter_module from stac_fastapi.sfeos_helpers.database import ( add_bbox_shape_to_collection, apply_collections_bbox_filter_shared, @@ -144,6 +148,7 @@ def __attrs_post_init__(self): collection_serializer: Type[CollectionSerializer] = attr.ib( default=CollectionSerializer ) + catalog_serializer: Type[CatalogSerializer] = attr.ib(default=CatalogSerializer) extensions: List[str] = attr.ib(default=attr.Factory(list)) @@ -1749,3 +1754,132 @@ async def delete_collections(self) -> None: body={"query": {"match_all": {}}}, wait_for_completion=True, ) + + """CATALOGS LOGIC""" + + async def get_all_catalogs( + self, + token: Optional[str], + limit: int, + request: Any = None, + sort: Optional[List[Dict[str, Any]]] = None, + ) -> Tuple[List[Dict[str, Any]], Optional[str], Optional[int]]: + """Retrieve a list of catalogs from Elasticsearch, supporting pagination. + + Args: + token (Optional[str]): The pagination token. + limit (int): The number of results to return. + request (Any, optional): The FastAPI request object. Defaults to None. + sort (Optional[List[Dict[str, Any]]], optional): Optional sort parameter. Defaults to None. + + Returns: + A tuple of (catalogs, next pagination token if any, optional count). + """ + # Define sortable fields for catalogs + sortable_fields = ["id"] + + # Format the sort parameter + formatted_sort = [] + if sort: + for item in sort: + field = item.get("field") + direction = item.get("direction", "asc") + if field and field in sortable_fields: + formatted_sort.append({field: {"order": direction}}) + + if not formatted_sort: + formatted_sort = [{"id": {"order": "asc"}}] + + body = { + "sort": formatted_sort, + "size": limit, + "query": {"term": {"type": "Catalog"}}, + } + + # Handle search_after token + search_after = None + if token: + try: + search_after = token.split("|") + if len(search_after) != len(formatted_sort): + search_after = None + except Exception: + search_after = None + + if search_after is not None: + body["search_after"] = search_after + + # Search for catalogs in collections index + response = await self.client.search( + index=COLLECTIONS_INDEX, + body=body, + ) + + hits = response["hits"]["hits"] + catalogs = [hit["_source"] for hit in hits] + + next_token = None + if len(hits) == limit: + next_token_values = hits[-1].get("sort") + if next_token_values: + next_token = "|".join(str(val) for val in next_token_values) + + # Get the total count + matched = ( + response["hits"]["total"]["value"] + if response["hits"]["total"]["relation"] == "eq" + else None + ) + + return catalogs, next_token, matched + + async def create_catalog(self, catalog: Dict, refresh: bool = False) -> None: + """Create a catalog in Elasticsearch. + + Args: + catalog (Dict): The catalog document to create. + refresh (bool): Whether to refresh the index after creation. + """ + await self.client.index( + index=COLLECTIONS_INDEX, + id=catalog.get("id"), + body=catalog, + refresh=refresh, + ) + + async def find_catalog(self, catalog_id: str) -> Dict: + """Find a catalog in Elasticsearch by ID. + + Args: + catalog_id (str): The ID of the catalog to find. + + Returns: + Dict: The catalog document. + + Raises: + NotFoundError: If the catalog is not found. + """ + try: + response = await self.client.get( + index=COLLECTIONS_INDEX, + id=catalog_id, + ) + # Verify it's a catalog + if response["_source"].get("type") != "Catalog": + raise NotFoundError(f"Catalog {catalog_id} not found") + return response["_source"] + except ESNotFoundError: + raise NotFoundError(f"Catalog {catalog_id} not found") + + async def delete_catalog(self, catalog_id: str, refresh: bool = False) -> None: + """Delete a catalog from Elasticsearch. + + Args: + catalog_id (str): The ID of the catalog to delete. + refresh (bool): Whether to refresh the index after deletion. + """ + await self.client.delete( + index=COLLECTIONS_INDEX, + id=catalog_id, + refresh=refresh, + ) diff --git a/stac_fastapi/opensearch/stac_fastapi/opensearch/app.py b/stac_fastapi/opensearch/stac_fastapi/opensearch/app.py index 181d8a7aa..7339eab55 100644 --- a/stac_fastapi/opensearch/stac_fastapi/opensearch/app.py +++ b/stac_fastapi/opensearch/stac_fastapi/opensearch/app.py @@ -23,6 +23,7 @@ EsAggregationExtensionGetRequest, EsAggregationExtensionPostRequest, ) +from stac_fastapi.core.extensions.catalogs import CatalogsExtension from stac_fastapi.core.extensions.collections_search import ( CollectionsSearchEndpointExtension, ) @@ -65,11 +66,13 @@ ENABLE_COLLECTIONS_SEARCH_ROUTE = get_bool_env( "ENABLE_COLLECTIONS_SEARCH_ROUTE", default=False ) +ENABLE_CATALOGS_ROUTE = get_bool_env("ENABLE_CATALOGS_ROUTE", default=False) logger.info("TRANSACTIONS_EXTENSIONS is set to %s", TRANSACTIONS_EXTENSIONS) logger.info("ENABLE_COLLECTIONS_SEARCH is set to %s", ENABLE_COLLECTIONS_SEARCH) logger.info( "ENABLE_COLLECTIONS_SEARCH_ROUTE is set to %s", ENABLE_COLLECTIONS_SEARCH_ROUTE ) +logger.info("ENABLE_CATALOGS_ROUTE is set to %s", ENABLE_CATALOGS_ROUTE) settings = OpensearchSettings() session = Session.create_from_settings(settings) @@ -201,6 +204,22 @@ extensions.append(collections_search_endpoint_ext) +if ENABLE_CATALOGS_ROUTE: + catalogs_extension = CatalogsExtension( + client=CoreClient( + database=database_logic, + session=session, + post_request_model=collection_search_post_request_model, + landing_page_id=os.getenv("STAC_FASTAPI_LANDING_PAGE_ID", "stac-fastapi"), + ), + settings=settings, + conformance_classes=[ + "https://api.stacspec.org/v1.0.0-beta.1/catalogs-endpoint", + ], + ) + extensions.append(catalogs_extension) + + database_logic.extensions = [type(ext).__name__ for ext in extensions] post_request_model = create_post_request_model(search_extensions) diff --git a/stac_fastapi/opensearch/stac_fastapi/opensearch/database_logic.py b/stac_fastapi/opensearch/stac_fastapi/opensearch/database_logic.py index 05aac1763..14529ac37 100644 --- a/stac_fastapi/opensearch/stac_fastapi/opensearch/database_logic.py +++ b/stac_fastapi/opensearch/stac_fastapi/opensearch/database_logic.py @@ -15,6 +15,7 @@ from opensearchpy.helpers.search import Search from starlette.requests import Request +import stac_fastapi.sfeos_helpers.filter as filter_module from stac_fastapi.core.base_database_logic import BaseDatabaseLogic from stac_fastapi.core.serializers import CollectionSerializer, ItemSerializer from stac_fastapi.core.utilities import MAX_LIMIT, bbox2polygon, get_bool_env @@ -27,7 +28,6 @@ AsyncOpensearchSettings as AsyncSearchSettings, ) from stac_fastapi.opensearch.config import OpensearchSettings as SyncSearchSettings -from stac_fastapi.sfeos_helpers import filter as filter_module from stac_fastapi.sfeos_helpers.database import ( add_bbox_shape_to_collection, apply_collections_bbox_filter_shared, @@ -1723,3 +1723,132 @@ async def delete_collections(self) -> None: body={"query": {"match_all": {}}}, wait_for_completion=True, ) + + """CATALOGS LOGIC""" + + async def get_all_catalogs( + self, + token: Optional[str], + limit: int, + request: Any = None, + sort: Optional[List[Dict[str, Any]]] = None, + ) -> Tuple[List[Dict[str, Any]], Optional[str], Optional[int]]: + """Retrieve a list of catalogs from OpenSearch, supporting pagination. + + Args: + token (Optional[str]): The pagination token. + limit (int): The number of results to return. + request (Any, optional): The FastAPI request object. Defaults to None. + sort (Optional[List[Dict[str, Any]]], optional): Optional sort parameter. Defaults to None. + + Returns: + A tuple of (catalogs, next pagination token if any, optional count). + """ + # Define sortable fields for catalogs + sortable_fields = ["id"] + + # Format the sort parameter + formatted_sort = [] + if sort: + for item in sort: + field = item.get("field") + direction = item.get("direction", "asc") + if field and field in sortable_fields: + formatted_sort.append({field: {"order": direction}}) + + if not formatted_sort: + formatted_sort = [{"id": {"order": "asc"}}] + + body = { + "sort": formatted_sort, + "size": limit, + "query": {"term": {"type": "Catalog"}}, + } + + # Handle search_after token + search_after = None + if token: + try: + search_after = token.split("|") + if len(search_after) != len(formatted_sort): + search_after = None + except Exception: + search_after = None + + if search_after is not None: + body["search_after"] = search_after + + # Search for catalogs in collections index + response = await self.client.search( + index=COLLECTIONS_INDEX, + body=body, + ) + + hits = response["hits"]["hits"] + catalogs = [hit["_source"] for hit in hits] + + next_token = None + if len(hits) == limit: + next_token_values = hits[-1].get("sort") + if next_token_values: + next_token = "|".join(str(val) for val in next_token_values) + + # Get the total count + matched = ( + response["hits"]["total"]["value"] + if response["hits"]["total"]["relation"] == "eq" + else None + ) + + return catalogs, next_token, matched + + async def create_catalog(self, catalog: Dict, refresh: bool = False) -> None: + """Create a catalog in OpenSearch. + + Args: + catalog (Dict): The catalog document to create. + refresh (bool): Whether to refresh the index after creation. + """ + await self.client.index( + index=COLLECTIONS_INDEX, + id=catalog.get("id"), + body=catalog, + refresh=refresh, + ) + + async def find_catalog(self, catalog_id: str) -> Dict: + """Find a catalog in OpenSearch by ID. + + Args: + catalog_id (str): The ID of the catalog to find. + + Returns: + Dict: The catalog document. + + Raises: + NotFoundError: If the catalog is not found. + """ + try: + response = await self.client.get( + index=COLLECTIONS_INDEX, + id=catalog_id, + ) + # Verify it's a catalog + if response["_source"].get("type") != "Catalog": + raise NotFoundError(f"Catalog {catalog_id} not found") + return response["_source"] + except exceptions.NotFoundError: + raise NotFoundError(f"Catalog {catalog_id} not found") + + async def delete_catalog(self, catalog_id: str, refresh: bool = False) -> None: + """Delete a catalog from OpenSearch. + + Args: + catalog_id (str): The ID of the catalog to delete. + refresh (bool): Whether to refresh the index after deletion. + """ + await self.client.delete( + index=COLLECTIONS_INDEX, + id=catalog_id, + refresh=refresh, + ) diff --git a/stac_fastapi/tests/api/test_api.py b/stac_fastapi/tests/api/test_api.py index 38d7e5978..76830a06b 100644 --- a/stac_fastapi/tests/api/test_api.py +++ b/stac_fastapi/tests/api/test_api.py @@ -50,6 +50,16 @@ "POST /collections/{collection_id}/aggregate", "GET /collections-search", "POST /collections-search", + "GET /catalogs", + "POST /catalogs", + "GET /catalogs/{catalog_id}", + "DELETE /catalogs/{catalog_id}", + "GET /catalogs/{catalog_id}/collections", + "POST /catalogs/{catalog_id}/collections", + "GET /catalogs/{catalog_id}/collections/{collection_id}", + "GET /catalogs/{catalog_id}/collections/{collection_id}/items", + "GET /catalogs/{catalog_id}/collections/{collection_id}/items/{item_id}", + "", } diff --git a/stac_fastapi/tests/conftest.py b/stac_fastapi/tests/conftest.py index b461e7221..1a6a2b662 100644 --- a/stac_fastapi/tests/conftest.py +++ b/stac_fastapi/tests/conftest.py @@ -39,6 +39,7 @@ from stac_fastapi.types.config import Settings os.environ.setdefault("ENABLE_COLLECTIONS_SEARCH_ROUTE", "true") +os.environ.setdefault("ENABLE_CATALOGS_ROUTE", "false") if os.getenv("BACKEND", "elasticsearch").lower() == "opensearch": from stac_fastapi.opensearch.app import app_config @@ -396,3 +397,63 @@ def build_test_app(): # Create and return the app api = StacApi(**test_config) return api.app + + +def build_test_app_with_catalogs(): + """Build a test app with catalogs extension enabled.""" + from stac_fastapi.core.extensions.catalogs import CatalogsExtension + + # Get the base config + test_config = app_config.copy() + + # Get database and settings (already imported above) + test_database = DatabaseLogic() + test_settings = AsyncSettings() + + # Add catalogs extension + catalogs_extension = CatalogsExtension( + client=CoreClient( + database=test_database, + session=None, + landing_page_id=os.getenv("STAC_FASTAPI_LANDING_PAGE_ID", "stac-fastapi"), + ), + settings=test_settings, + conformance_classes=[ + "https://api.stacspec.org/v1.0.0-beta.1/catalogs-endpoint", + ], + ) + + # Add to extensions if not already present + if not any(isinstance(ext, CatalogsExtension) for ext in test_config["extensions"]): + test_config["extensions"].append(catalogs_extension) + + # Update client with new extensions + test_config["client"] = CoreClient( + database=test_database, + session=None, + extensions=test_config["extensions"], + post_request_model=test_config["search_post_request_model"], + landing_page_id=os.getenv("STAC_FASTAPI_LANDING_PAGE_ID", "stac-fastapi"), + ) + + # Create and return the app + api = StacApi(**test_config) + return api.app + + +@pytest_asyncio.fixture(scope="session") +async def catalogs_app(): + """Fixture to get the FastAPI app with catalogs extension enabled.""" + return build_test_app_with_catalogs() + + +@pytest_asyncio.fixture(scope="session") +async def catalogs_app_client(catalogs_app): + """Fixture to get an async client for the app with catalogs extension enabled.""" + await create_index_templates() + await create_collection_index() + + async with AsyncClient( + transport=ASGITransport(app=catalogs_app), base_url="http://test-server" + ) as c: + yield c diff --git a/stac_fastapi/tests/data/test_catalog.json b/stac_fastapi/tests/data/test_catalog.json new file mode 100644 index 000000000..49a59f8d4 --- /dev/null +++ b/stac_fastapi/tests/data/test_catalog.json @@ -0,0 +1,25 @@ +{ + "id": "test-catalog", + "type": "Catalog", + "stac_version": "1.0.0", + "title": "Test Catalog", + "description": "A test catalog for STAC API testing", + "links": [ + { + "rel": "self", + "href": "http://test-server/catalogs/test-catalog", + "type": "application/json" + }, + { + "rel": "root", + "href": "http://test-server/catalogs", + "type": "application/json" + }, + { + "rel": "child", + "href": "http://test-server/collections/placeholder-collection", + "type": "application/json", + "title": "Placeholder Collection" + } + ] +} diff --git a/stac_fastapi/tests/extensions/test_catalogs.py b/stac_fastapi/tests/extensions/test_catalogs.py new file mode 100644 index 000000000..7c9960e73 --- /dev/null +++ b/stac_fastapi/tests/extensions/test_catalogs.py @@ -0,0 +1,598 @@ +import uuid + +import pytest + + +@pytest.mark.asyncio +async def test_get_root_catalog(catalogs_app_client, load_test_data): + """Test getting the catalogs list.""" + resp = await catalogs_app_client.get("/catalogs") + assert resp.status_code == 200 + + catalogs_response = resp.json() + assert "catalogs" in catalogs_response + assert "links" in catalogs_response + assert "numberReturned" in catalogs_response + + # Check for required pagination links + links = catalogs_response["links"] + link_rels = [link["rel"] for link in links] + assert "self" in link_rels + assert "root" in link_rels + assert "parent" in link_rels + + +@pytest.mark.asyncio +async def test_create_catalog(catalogs_app_client, load_test_data): + """Test creating a new catalog.""" + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert resp.status_code == 201 + + created_catalog = resp.json() + assert created_catalog["id"] == test_catalog["id"] + assert created_catalog["type"] == "Catalog" + assert created_catalog["title"] == test_catalog["title"] + + +@pytest.mark.asyncio +async def test_get_catalog(catalogs_app_client, load_test_data): + """Test getting a specific catalog.""" + # First create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + + # Now get it back + resp = await catalogs_app_client.get(f"/catalogs/{test_catalog['id']}") + assert resp.status_code == 200 + + catalog = resp.json() + assert catalog["id"] == test_catalog["id"] + assert catalog["title"] == test_catalog["title"] + assert catalog["description"] == test_catalog["description"] + + +@pytest.mark.asyncio +async def test_get_nonexistent_catalog(catalogs_app_client): + """Test getting a catalog that doesn't exist.""" + resp = await catalogs_app_client.get("/catalogs/nonexistent-catalog") + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_get_catalog_collections(catalogs_app_client, load_test_data, ctx): + """Test getting collections linked from a catalog.""" + # First create a catalog with a link to the test collection + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + # Update the catalog links to point to the actual test collection + for link in test_catalog["links"]: + if link["rel"] == "child": + link["href"] = f"http://test-server/collections/{ctx.collection['id']}" + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + + # Now get collections from the catalog + resp = await catalogs_app_client.get(f"/catalogs/{test_catalog['id']}/collections") + assert resp.status_code == 200 + + collections_response = resp.json() + assert "collections" in collections_response + assert "links" in collections_response + + # Should contain the test collection + collection_ids = [col["id"] for col in collections_response["collections"]] + assert ctx.collection["id"] in collection_ids + + +@pytest.mark.asyncio +async def test_get_catalog_collections_nonexistent_catalog(catalogs_app_client): + """Test getting collections from a catalog that doesn't exist.""" + resp = await catalogs_app_client.get("/catalogs/nonexistent-catalog/collections") + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_root_catalog_with_multiple_catalogs(catalogs_app_client, load_test_data): + """Test that catalogs response includes multiple catalog objects.""" + # Create multiple catalogs + catalog_ids = [] + for i in range(3): + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}-{i}" + test_catalog["title"] = f"Test Catalog {i}" + + resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert resp.status_code == 201 + catalog_ids.append(test_catalog["id"]) + + # Get catalogs list + resp = await catalogs_app_client.get("/catalogs") + assert resp.status_code == 200 + + catalogs_response = resp.json() + returned_catalogs = catalogs_response["catalogs"] + returned_ids = [catalog["id"] for catalog in returned_catalogs] + + # Should have all created catalogs in the response + for catalog_id in catalog_ids: + assert catalog_id in returned_ids + + +@pytest.mark.asyncio +async def test_get_catalog_collection(catalogs_app_client, load_test_data, ctx): + """Test getting a specific collection from a catalog.""" + # First create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + + # Get a specific collection through the catalog route + resp = await catalogs_app_client.get( + f"/catalogs/{test_catalog['id']}/collections/{ctx.collection['id']}" + ) + assert resp.status_code == 200 + + collection = resp.json() + assert collection["id"] == ctx.collection["id"] + assert collection["type"] == "Collection" + assert "links" in collection + + +@pytest.mark.asyncio +async def test_get_catalog_collection_nonexistent_catalog(catalogs_app_client, ctx): + """Test getting a collection from a catalog that doesn't exist.""" + resp = await catalogs_app_client.get( + f"/catalogs/nonexistent-catalog/collections/{ctx.collection['id']}" + ) + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_get_catalog_collection_nonexistent_collection( + catalogs_app_client, load_test_data +): + """Test getting a collection that doesn't exist from a catalog.""" + # First create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + + # Try to get a nonexistent collection + resp = await catalogs_app_client.get( + f"/catalogs/{test_catalog['id']}/collections/nonexistent-collection" + ) + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_get_catalog_collection_items(catalogs_app_client, load_test_data, ctx): + """Test getting items from a collection in a catalog.""" + # First create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + + # Get items from a collection through the catalog route + resp = await catalogs_app_client.get( + f"/catalogs/{test_catalog['id']}/collections/{ctx.collection['id']}/items" + ) + assert resp.status_code == 200 + + items_response = resp.json() + assert items_response["type"] == "FeatureCollection" + assert "features" in items_response + assert "links" in items_response + # Should contain the test item + assert len(items_response["features"]) > 0 + + +@pytest.mark.asyncio +async def test_get_catalog_collection_items_nonexistent_catalog( + catalogs_app_client, ctx +): + """Test getting items from a collection in a catalog that doesn't exist.""" + resp = await catalogs_app_client.get( + f"/catalogs/nonexistent-catalog/collections/{ctx.collection['id']}/items" + ) + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_get_catalog_collection_items_nonexistent_collection( + catalogs_app_client, load_test_data +): + """Test getting items from a collection that doesn't exist in a catalog.""" + # First create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + + # Try to get items from a nonexistent collection + resp = await catalogs_app_client.get( + f"/catalogs/{test_catalog['id']}/collections/nonexistent-collection/items" + ) + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_get_catalog_collection_item(catalogs_app_client, load_test_data, ctx): + """Test getting a specific item from a collection in a catalog.""" + # First create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + + # Get a specific item through the catalog route + resp = await catalogs_app_client.get( + f"/catalogs/{test_catalog['id']}/collections/{ctx.collection['id']}/items/{ctx.item['id']}" + ) + assert resp.status_code == 200 + + item = resp.json() + assert item["id"] == ctx.item["id"] + assert item["type"] == "Feature" + assert "properties" in item + assert "geometry" in item + + +@pytest.mark.asyncio +async def test_get_catalog_collection_item_nonexistent_catalog( + catalogs_app_client, ctx +): + """Test getting an item from a collection in a catalog that doesn't exist.""" + resp = await catalogs_app_client.get( + f"/catalogs/nonexistent-catalog/collections/{ctx.collection['id']}/items/{ctx.item['id']}" + ) + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_get_catalog_collection_item_nonexistent_collection( + catalogs_app_client, load_test_data, ctx +): + """Test getting an item from a collection that doesn't exist in a catalog.""" + # First create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + + # Try to get an item from a nonexistent collection + resp = await catalogs_app_client.get( + f"/catalogs/{test_catalog['id']}/collections/nonexistent-collection/items/{ctx.item['id']}" + ) + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_get_catalog_collection_item_nonexistent_item( + catalogs_app_client, load_test_data, ctx +): + """Test getting an item that doesn't exist from a collection in a catalog.""" + # First create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + + # Try to get a nonexistent item + resp = await catalogs_app_client.get( + f"/catalogs/{test_catalog['id']}/collections/{ctx.collection['id']}/items/nonexistent-item" + ) + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_catalogs_pagination_limit(catalogs_app_client, load_test_data): + """Test that pagination limit parameter works for catalogs endpoint.""" + # Create multiple catalogs + catalog_ids = [] + for i in range(5): + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}-{i}" + test_catalog["title"] = f"Test Catalog {i}" + + resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert resp.status_code == 201 + catalog_ids.append(test_catalog["id"]) + + # Test with limit=2 + resp = await catalogs_app_client.get("/catalogs?limit=2") + assert resp.status_code == 200 + + catalogs_response = resp.json() + returned_catalogs = catalogs_response["catalogs"] + + # Should only return 2 catalogs + assert len(returned_catalogs) == 2 + assert catalogs_response["numberReturned"] == 2 + + +@pytest.mark.asyncio +async def test_catalogs_pagination_default_limit(catalogs_app_client, load_test_data): + """Test that pagination uses default limit when no limit parameter is provided.""" + # Create multiple catalogs + catalog_ids = [] + for i in range(15): + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}-{i}" + test_catalog["title"] = f"Test Catalog {i}" + + resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert resp.status_code == 201 + catalog_ids.append(test_catalog["id"]) + + # Test without limit parameter (should default to 10) + resp = await catalogs_app_client.get("/catalogs") + assert resp.status_code == 200 + + catalogs_response = resp.json() + returned_catalogs = catalogs_response["catalogs"] + + # Should return default limit of 10 catalogs + assert len(returned_catalogs) == 10 + assert catalogs_response["numberReturned"] == 10 + + +@pytest.mark.asyncio +async def test_catalogs_pagination_limit_validation(catalogs_app_client): + """Test that pagination limit parameter validation works.""" + # Test with limit=0 (should be invalid) + resp = await catalogs_app_client.get("/catalogs?limit=0") + assert resp.status_code == 400 # Validation error returns 400 for Query parameters + + +@pytest.mark.asyncio +async def test_catalogs_pagination_token_parameter(catalogs_app_client, load_test_data): + """Test that pagination token parameter is accepted (even if token is invalid).""" + # Create a catalog first + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert resp.status_code == 201 + + # Test with token parameter (even if invalid, should be accepted) + resp = await catalogs_app_client.get("/catalogs?token=invalid_token") + assert resp.status_code == 200 + + catalogs_response = resp.json() + assert "catalogs" in catalogs_response + assert "links" in catalogs_response + + +@pytest.mark.asyncio +async def test_create_catalog_collection(catalogs_app_client, load_test_data, ctx): + """Test creating a collection within a catalog.""" + # First create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + + # Remove placeholder collection links so we start with a clean catalog + test_catalog["links"] = [ + link for link in test_catalog.get("links", []) if link.get("rel") != "child" + ] + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + catalog_id = test_catalog["id"] + + # Create a new collection within the catalog + test_collection = load_test_data("test_collection.json") + test_collection["id"] = f"test-collection-{uuid.uuid4()}" + + resp = await catalogs_app_client.post( + f"/catalogs/{catalog_id}/collections", json=test_collection + ) + assert resp.status_code == 201 + + created_collection = resp.json() + assert created_collection["id"] == test_collection["id"] + assert created_collection["type"] == "Collection" + + # Verify the collection was created by getting it directly + get_resp = await catalogs_app_client.get(f"/collections/{test_collection['id']}") + assert get_resp.status_code == 200 + assert get_resp.json()["id"] == test_collection["id"] + + # Verify the collection has a catalog link to the catalog + collection_data = get_resp.json() + collection_links = collection_data.get("links", []) + catalog_link = None + for link in collection_links: + if link.get("rel") == "catalog" and f"/catalogs/{catalog_id}" in link.get( + "href", "" + ): + catalog_link = link + break + + assert ( + catalog_link is not None + ), f"Collection should have catalog link to /catalogs/{catalog_id}" + assert catalog_link["type"] == "application/json" + assert catalog_link["href"].endswith(f"/catalogs/{catalog_id}") + + # Verify the catalog has a child link to the collection + catalog_resp = await catalogs_app_client.get(f"/catalogs/{catalog_id}") + assert catalog_resp.status_code == 200 + catalog_data = catalog_resp.json() + catalog_links = catalog_data.get("links", []) + collection_child_link = None + for link in catalog_links: + if link.get( + "rel" + ) == "child" and f"/collections/{test_collection['id']}" in link.get( + "href", "" + ): + collection_child_link = link + break + + assert ( + collection_child_link is not None + ), f"Catalog should have child link to collection /collections/{test_collection['id']}" + assert collection_child_link["type"] == "application/json" + assert collection_child_link["href"].endswith( + f"/collections/{test_collection['id']}" + ) + + # Verify the catalog now includes the collection in its collections endpoint + catalog_resp = await catalogs_app_client.get(f"/catalogs/{catalog_id}/collections") + assert catalog_resp.status_code == 200 + + collections_response = catalog_resp.json() + collection_ids = [col["id"] for col in collections_response["collections"]] + assert test_collection["id"] in collection_ids + + +@pytest.mark.asyncio +async def test_create_catalog_collection_nonexistent_catalog( + catalogs_app_client, load_test_data +): + """Test creating a collection in a catalog that doesn't exist.""" + test_collection = load_test_data("test_collection.json") + test_collection["id"] = f"test-collection-{uuid.uuid4()}" + + resp = await catalogs_app_client.post( + "/catalogs/nonexistent-catalog/collections", json=test_collection + ) + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_delete_catalog(catalogs_app_client, load_test_data): + """Test deleting a catalog without cascade.""" + # Create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + test_catalog["links"] = [ + link for link in test_catalog.get("links", []) if link.get("rel") != "child" + ] + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + catalog_id = test_catalog["id"] + + # Verify catalog exists + get_resp = await catalogs_app_client.get(f"/catalogs/{catalog_id}") + assert get_resp.status_code == 200 + + # Delete the catalog + delete_resp = await catalogs_app_client.delete(f"/catalogs/{catalog_id}") + assert delete_resp.status_code == 204 + + # Verify catalog is deleted + get_resp = await catalogs_app_client.get(f"/catalogs/{catalog_id}") + assert get_resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_delete_catalog_cascade(catalogs_app_client, load_test_data): + """Test deleting a catalog with cascade delete of collections.""" + # Create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + test_catalog["links"] = [ + link for link in test_catalog.get("links", []) if link.get("rel") != "child" + ] + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + catalog_id = test_catalog["id"] + + # Create a collection in the catalog + test_collection = load_test_data("test_collection.json") + test_collection["id"] = f"test-collection-{uuid.uuid4()}" + + coll_resp = await catalogs_app_client.post( + f"/catalogs/{catalog_id}/collections", json=test_collection + ) + assert coll_resp.status_code == 201 + collection_id = test_collection["id"] + + # Verify collection exists + get_coll_resp = await catalogs_app_client.get(f"/collections/{collection_id}") + assert get_coll_resp.status_code == 200 + + # Delete the catalog with cascade=true + delete_resp = await catalogs_app_client.delete( + f"/catalogs/{catalog_id}?cascade=true" + ) + assert delete_resp.status_code == 204 + + # Verify catalog is deleted + get_resp = await catalogs_app_client.get(f"/catalogs/{catalog_id}") + assert get_resp.status_code == 404 + + # Verify collection is also deleted (cascade delete) + get_coll_resp = await catalogs_app_client.get(f"/collections/{collection_id}") + assert get_coll_resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_delete_catalog_no_cascade(catalogs_app_client, load_test_data): + """Test deleting a catalog without cascade (collections remain).""" + # Create a catalog + test_catalog = load_test_data("test_catalog.json") + test_catalog["id"] = f"test-catalog-{uuid.uuid4()}" + test_catalog["links"] = [ + link for link in test_catalog.get("links", []) if link.get("rel") != "child" + ] + + create_resp = await catalogs_app_client.post("/catalogs", json=test_catalog) + assert create_resp.status_code == 201 + catalog_id = test_catalog["id"] + + # Create a collection in the catalog + test_collection = load_test_data("test_collection.json") + test_collection["id"] = f"test-collection-{uuid.uuid4()}" + + coll_resp = await catalogs_app_client.post( + f"/catalogs/{catalog_id}/collections", json=test_collection + ) + assert coll_resp.status_code == 201 + collection_id = test_collection["id"] + + # Delete the catalog with cascade=false (default) + delete_resp = await catalogs_app_client.delete(f"/catalogs/{catalog_id}") + assert delete_resp.status_code == 204 + + # Verify catalog is deleted + get_resp = await catalogs_app_client.get(f"/catalogs/{catalog_id}") + assert get_resp.status_code == 404 + + # Verify collection still exists (no cascade delete) + get_coll_resp = await catalogs_app_client.get(f"/collections/{collection_id}") + assert get_coll_resp.status_code == 200 + + # Verify the catalog link was removed from the collection + collection_data = get_coll_resp.json() + collection_links = collection_data.get("links", []) + catalog_link = None + for link in collection_links: + if link.get("rel") == "catalog" and catalog_id in link.get("href", ""): + catalog_link = link + break + + assert ( + catalog_link is None + ), "Collection should not have catalog link after catalog deletion"