feat(ui): another go at gallery (#3791)

* feat(ui): migrate listImages to RTK query using createEntityAdapter

- see comments in `endpoints/images.ts` for explanation of the caching
- so far, only manually updating `all` images when new image is generated. no other manual cache updates are implemented, but will be needed.
- fixed some weirdness with loading state components (like the spinners in gallery)
- added `useThumbnailFallback` for `IAIDndImage`, this displays the tiny webp thumbnail while the full-size images load
- comment out some old thunk related stuff in gallerySlice, which is no longer needed

* feat(ui): add manual cache updates for board changes (wip)

- update RTK Query caches when adding/removing single image to/from board
- work more on migrating all image-related operations to RTK Query

* update AddImagesToBoardContext so that it works when user uses context menu + modal

* handle case where no image is selected

* get assets working for main list and boards - dnd only

* feat(ui): migrate image uploads to RTK Query

- minor refactor of `ImageUploader` and `useImageUploadButton` hooks, simplify some logic
- style filesystem upload overlay to match existing UI
- replace all old `imageUploaded` thunks with `uploadImage` RTK Query calls, update associated logic including canvas related uploads
- simplify `PostUploadAction`s that only need to display user input

* feat(ui): remove `receivedPageOfImages` thunks

* feat(ui): remove `receivedImageUrls` thunk

* feat(ui): finish removing all images thunks

stuff now broken:
- image usage
- delete board images
- on first load, no image selected

* feat(ui): simplify `updateImage` cache manipulation

- we don't actually ever change categories, so we can remove a lot of logic

* feat(ui): simplify canvas autosave

- instead of using a network request to set the canvas generation as not intermediate, we can just do that in the graph

* feat(ui): simplify & handle edge cases in cache updates

* feat(db, api): support `board_id='none'` for `get_many` images queries

This allows us to get all images that are not on a board.

* chore(ui): regen types

* feat(ui): add `All Assets`, `No Board` boards

Restructure boards:
- `all images` is all images
- `all assets` is all assets
- `no board` is all images/assets without a board set
- user boards may have images and assets

Update caching logic
- much simpler without every board having sub-views of images and assets
- update drag and drop operations for all possible interactions

* chore(ui): regen types

* feat(ui): move download to top of context menu

* feat(ui): improve drop overlay styles

* fix(ui): fix image not selected on first load

- listen for first load of all images board, then select the first image

* feat(ui): refactor board deletion

api changes:
- add route to list all image names for a board. this is required to handle board + image deletion. we need to know every image in the board to determine the image usage across the app. this is fetched only when the delete board and images modal is opened so it's as efficient as it can be.
- update the delete board route to respond with a list of deleted `board_images` and `images`, as image names. this is needed to perform accurate clientside state & cache updates after deleting.

db changes:
- remove unused `board_images` service method to get paginated images dtos for a board. this is now done thru the list images endpoint & images service. needs a small logic change on `images.delete_images_on_board`

ui changes:
- simplify the delete board modal - no context, just minor prop drilling. this is feasible for boards only because the components that need to trigger and manipulate the modal are very close together in the tree
- add cache updates for `deleteBoard` & `deleteBoardAndImages` mutations
- the only thing we cannot do directly is on `deleteBoardAndImages`, update the `No Board` board. we'd need to insert image dtos that we may not have loaded. instead, i am just invalidating the tags for that `listImages` cache. so when you `deleteBoardAndImages`, the `No Board` will re-fetch the initial image limit. i think this is more efficient than e.g. fetching all image dtos to insert then inserting them.
- handle image usage for `deleteBoardAndImages`
- update all (i think/hope) the little bits and pieces in the UI to accomodate these changes

* fix(ui): fix board selection logic

* feat(ui): add delete board modal loading state

* fix(ui): use thumbnails for board cover images

* fix(ui): fix race condition with board selection

when selecting a board that doesn't have any images loaded, we need to wait until the images haveloaded before selecting the first image.

this logic is debounced to ~1000ms.

* feat(ui): name 'No Board' correctly, change icon

* fix(ui): do not cache listAllImageNames query

if we cache it, we can end up with stale image usage during deletion.

we could of course manually update the cache as we are doing elsewhere. but because this is a relatively infrequent network request, i'd like to trade increased cache mgmt complexity here for increased resource usage.

* feat(ui): reduce drag preview opacity, remove border

* fix(ui): fix incorrect queryArg used in `deleteImage` and `updateImage` cache updates

* fix(ui): fix doubled open in new tab

* fix(ui): fix new generations not getting added to 'No Board'

* fix(ui): fix board id not changing on new image when autosave enabled

* fix(ui): context menu when selection is 0

need to revise how context menu is triggered later, when we approach multi select

* fix(ui): fix deleting does not update counts for all images and all assets

* fix(ui): fix all assets board name in boards list collapse button

* fix(ui): ensure we never go under 0 for total board count

* fix(ui): fix text overflow on board names

---------

Co-authored-by: Mary Hipp <[email protected]>

Author: psychedelicious

invokeai/app/api/routers/board_images.py

@@ -24,11 +24,14 @@ async def create_board_image(
 ):
     """Creates a board_image"""
     try:
-        result = ApiDependencies.invoker.services.board_images.add_image_to_board(board_id=board_id, image_name=image_name)
+        result = ApiDependencies.invoker.services.board_images.add_image_to_board(
+            board_id=board_id, image_name=image_name
+        )
         return result
     except Exception as e:
         raise HTTPException(status_code=500, detail="Failed to add to board")
-    
+
+
 @board_images_router.delete(
     "/",
     operation_id="remove_board_image",
@@ -43,27 +46,10 @@ async def remove_board_image(
 ):
     """Deletes a board_image"""
     try:
-        result = ApiDependencies.invoker.services.board_images.remove_image_from_board(board_id=board_id, image_name=image_name)
+        result = ApiDependencies.invoker.services.board_images.remove_image_from_board(
+            board_id=board_id, image_name=image_name
+        )
         return result
     except Exception as e:
         raise HTTPException(status_code=500, detail="Failed to update board")
 
-
-
-@board_images_router.get(
-    "/{board_id}",
-    operation_id="list_board_images",
-    response_model=OffsetPaginatedResults[ImageDTO],
-)
-async def list_board_images(
-    board_id: str = Path(description="The id of the board"),
-    offset: int = Query(default=0, description="The page offset"),
-    limit: int = Query(default=10, description="The number of boards per page"),
-) -> OffsetPaginatedResults[ImageDTO]:
-    """Gets a list of images for a board"""
-
-    results = ApiDependencies.invoker.services.board_images.get_images_for_board(
-        board_id,
-    )
-    return results
-

invokeai/app/api/routers/boards.py

@@ -1,16 +1,28 @@
 from typing import Optional, Union
+
 from fastapi import Body, HTTPException, Path, Query
 from fastapi.routing import APIRouter
+from pydantic import BaseModel, Field
+
 from invokeai.app.services.board_record_storage import BoardChanges
 from invokeai.app.services.image_record_storage import OffsetPaginatedResults
 from invokeai.app.services.models.board_record import BoardDTO
 
-
 from ..dependencies import ApiDependencies
 
 boards_router = APIRouter(prefix="/v1/boards", tags=["boards"])
 
 
+class DeleteBoardResult(BaseModel):
+    board_id: str = Field(description="The id of the board that was deleted.")
+    deleted_board_images: list[str] = Field(
+        description="The image names of the board-images relationships that were deleted."
+    )
+    deleted_images: list[str] = Field(
+        description="The names of the images that were deleted."
+    )
+
+
 @boards_router.post(
     "/",
     operation_id="create_board",
@@ -69,25 +81,42 @@ async def update_board(
         raise HTTPException(status_code=500, detail="Failed to update board")
 
 
-@boards_router.delete("/{board_id}", operation_id="delete_board")
+@boards_router.delete(
+    "/{board_id}", operation_id="delete_board", response_model=DeleteBoardResult
+)
 async def delete_board(
     board_id: str = Path(description="The id of board to delete"),
     include_images: Optional[bool] = Query(
         description="Permanently delete all images on the board", default=False
     ),
-) -> None:
+) -> DeleteBoardResult:
     """Deletes a board"""
     try:
         if include_images is True:
+            deleted_images = ApiDependencies.invoker.services.board_images.get_all_board_image_names_for_board(
+                board_id=board_id
+            )
             ApiDependencies.invoker.services.images.delete_images_on_board(
                 board_id=board_id
             )
             ApiDependencies.invoker.services.boards.delete(board_id=board_id)
+            return DeleteBoardResult(
+                board_id=board_id,
+                deleted_board_images=[],
+                deleted_images=deleted_images,
+            )
         else:
+            deleted_board_images = ApiDependencies.invoker.services.board_images.get_all_board_image_names_for_board(
+                board_id=board_id
+            )
             ApiDependencies.invoker.services.boards.delete(board_id=board_id)
+            return DeleteBoardResult(
+                board_id=board_id,
+                deleted_board_images=deleted_board_images,
+                deleted_images=[],
+            )
     except Exception as e:
-        # TODO: Does this need any exception handling at all?
-        pass
+        raise HTTPException(status_code=500, detail="Failed to delete board")
 
 
 @boards_router.get(
@@ -115,3 +144,19 @@ async def list_boards(
             status_code=400,
             detail="Invalid request: Must provide either 'all' or both 'offset' and 'limit'",
         )
+
+
+@boards_router.get(
+    "/{board_id}/image_names",
+    operation_id="list_all_board_image_names",
+    response_model=list[str],
+)
+async def list_all_board_image_names(
+    board_id: str = Path(description="The id of the board"),
+) -> list[str]:
+    """Gets a list of images for a board"""
+
+    image_names = ApiDependencies.invoker.services.board_images.get_all_board_image_names_for_board(
+        board_id,
+    )
+    return image_names

invokeai/app/api/routers/images.py

@@ -245,16 +245,16 @@ async def get_image_urls(
 )
 async def list_image_dtos(
     image_origin: Optional[ResourceOrigin] = Query(
-        default=None, description="The origin of images to list"
+        default=None, description="The origin of images to list."
     ),
     categories: Optional[list[ImageCategory]] = Query(
-        default=None, description="The categories of image to include"
+        default=None, description="The categories of image to include."
     ),
     is_intermediate: Optional[bool] = Query(
-        default=None, description="Whether to list intermediate images"
+        default=None, description="Whether to list intermediate images."
     ),
     board_id: Optional[str] = Query(
-        default=None, description="The board id to filter by"
+        default=None, description="The board id to filter by. Use 'none' to find images without a board."
     ),
     offset: int = Query(default=0, description="The page offset"),
     limit: int = Query(default=10, description="The number of images per page"),

invokeai/app/services/board_image_record_storage.py

@@ -32,11 +32,11 @@ def remove_image_from_board(
         pass
 
     @abstractmethod
-    def get_images_for_board(
+    def get_all_board_image_names_for_board(
         self,
         board_id: str,
-    ) -> OffsetPaginatedResults[ImageRecord]:
-        """Gets images for a board."""
+    ) -> list[str]:
+        """Gets all board images for a board, as a list of the image names."""
         pass
 
     @abstractmethod
@@ -211,6 +211,26 @@ def get_images_for_board(
             items=images, offset=offset, limit=limit, total=count
         )
 
+    def get_all_board_image_names_for_board(self, board_id: str) -> list[str]:
+        try:
+            self._lock.acquire()
+            self._cursor.execute(
+                """--sql
+                SELECT image_name
+                FROM board_images
+                WHERE board_id = ?;
+                """,
+                (board_id,),
+            )
+            result = cast(list[sqlite3.Row], self._cursor.fetchall())
+            image_names = list(map(lambda r: r[0], result))
+            return image_names
+        except sqlite3.Error as e:
+            self._conn.rollback()
+            raise e
+        finally:
+            self._lock.release()
+
     def get_board_for_image(
         self,
         image_name: str,

invokeai/app/services/board_images.py

@@ -38,11 +38,11 @@ def remove_image_from_board(
         pass
 
     @abstractmethod
-    def get_images_for_board(
+    def get_all_board_image_names_for_board(
         self,
         board_id: str,
-    ) -> OffsetPaginatedResults[ImageDTO]:
-        """Gets images for a board."""
+    ) -> list[str]:
+        """Gets all board images for a board, as a list of the image names."""
         pass
 
     @abstractmethod
@@ -98,30 +98,13 @@ def remove_image_from_board(
     ) -> None:
         self._services.board_image_records.remove_image_from_board(board_id, image_name)
 
-    def get_images_for_board(
+    def get_all_board_image_names_for_board(
         self,
         board_id: str,
-    ) -> OffsetPaginatedResults[ImageDTO]:
-        image_records = self._services.board_image_records.get_images_for_board(
+    ) -> list[str]:
+        return self._services.board_image_records.get_all_board_image_names_for_board(
             board_id
         )
-        image_dtos = list(
-            map(
-                lambda r: image_record_to_dto(
-                    r,
-                    self._services.urls.get_image_url(r.image_name),
-                    self._services.urls.get_image_url(r.image_name, True),
-                    board_id,
-                ),
-                image_records.items,
-            )
-        )
-        return OffsetPaginatedResults[ImageDTO](
-            items=image_dtos,
-            offset=image_records.offset,
-            limit=image_records.limit,
-            total=image_records.total,
-        )
 
     def get_board_for_image(
         self,
@@ -136,7 +119,7 @@ def board_record_to_dto(
 ) -> BoardDTO:
     """Converts a board record to a board DTO."""
     return BoardDTO(
-        **board_record.dict(exclude={'cover_image_name'}),
+        **board_record.dict(exclude={"cover_image_name"}),
         cover_image_name=cover_image_name,
         image_count=image_count,
     )

invokeai/app/services/image_record_storage.py

@@ -10,7 +10,10 @@
 
 from invokeai.app.models.image import ImageCategory, ResourceOrigin
 from invokeai.app.services.models.image_record import (
-    ImageRecord, ImageRecordChanges, deserialize_image_record)
+    ImageRecord,
+    ImageRecordChanges,
+    deserialize_image_record,
+)
 
 T = TypeVar("T", bound=BaseModel)
 
@@ -377,11 +380,15 @@ def get_many(
 
                 query_params.append(is_intermediate)
 
-            if board_id is not None:
+            # board_id of "none" is reserved for images without a board
+            if board_id == "none":
+                query_conditions += """--sql
+                AND board_images.board_id IS NULL
+                """
+            elif board_id is not None:
                 query_conditions += """--sql
                 AND board_images.board_id = ?
                 """
-
                 query_params.append(board_id)
 
             query_pagination = """--sql

invokeai/app/services/images.py

@@ -11,7 +11,6 @@
                                        InvalidOriginException, ResourceOrigin)
 from invokeai.app.services.board_image_record_storage import \
     BoardImageRecordStorageBase
-from invokeai.app.services.graph import Graph
 from invokeai.app.services.image_file_storage import (
     ImageFileDeleteException, ImageFileNotFoundException,
     ImageFileSaveException, ImageFileStorageBase)
@@ -385,16 +384,14 @@ def delete(self, image_name: str):
 
     def delete_images_on_board(self, board_id: str):
         try:
-            images = self._services.board_image_records.get_images_for_board(board_id)
-            image_name_list = list(
-                map(
-                    lambda r: r.image_name,
-                    images.items,
+            image_names = (
+                self._services.board_image_records.get_all_board_image_names_for_board(
+                    board_id
                 )
             )
-            for image_name in image_name_list:
+            for image_name in image_names:
                 self._services.image_files.delete(image_name)
-            self._services.image_records.delete_many(image_name_list)
+            self._services.image_records.delete_many(image_names)
         except ImageRecordDeleteException:
             self._services.logger.error(f"Failed to delete image records")
             raise

invokeai/frontend/web/src/app/components/App.tsx

@@ -15,7 +15,6 @@ import InvokeTabs from 'features/ui/components/InvokeTabs';
 import ParametersDrawer from 'features/ui/components/ParametersDrawer';
 import i18n from 'i18n';
 import { ReactNode, memo, useEffect } from 'react';
-import DeleteBoardImagesModal from '../../features/gallery/components/Boards/DeleteBoardImagesModal';
 import UpdateImageBoardModal from '../../features/gallery/components/Boards/UpdateImageBoardModal';
 import GlobalHotkeys from './GlobalHotkeys';
 import Toaster from './Toaster';
@@ -84,7 +83,6 @@ const App = ({ config = DEFAULT_CONFIG, headerComponent }: Props) => {
       </Grid>
       <DeleteImageModal />
       <UpdateImageBoardModal />
-      <DeleteBoardImagesModal />
       <Toaster />
       <GlobalHotkeys />
     </>

invokeai/frontend/web/src/app/components/ImageDnd/DragPreview.tsx

@@ -15,10 +15,7 @@ const STYLES: ChakraProps['sx'] = {
   maxH: BOX_SIZE,
   shadow: 'dark-lg',
   borderRadius: 'lg',
-  borderWidth: 2,
-  borderStyle: 'dashed',
-  borderColor: 'base.100',
-  opacity: 0.5,
+  opacity: 0.3,
   bg: 'base.800',
   color: 'base.50',
   _dark: {

invokeai/frontend/web/src/app/components/ImageDnd/ImageDndContext.tsx

@@ -28,6 +28,7 @@ const ImageDndContext = (props: ImageDndContextProps) => {
   const dispatch = useAppDispatch();
 
   const handleDragStart = useCallback((event: DragStartEvent) => {
+    console.log('dragStart', event.active.data.current);
     const activeData = event.active.data.current;
     if (!activeData) {
       return;
@@ -37,15 +38,16 @@ const ImageDndContext = (props: ImageDndContextProps) => {
 
   const handleDragEnd = useCallback(
     (event: DragEndEvent) => {
+      console.log('dragEnd', event.active.data.current);
       const activeData = event.active.data.current;
       const overData = event.over?.data.current;
-      if (!activeData || !overData) {
+      if (!activeDragData || !overData) {
         return;
       }
-      dispatch(dndDropped({ overData, activeData }));
+      dispatch(dndDropped({ overData, activeData: activeDragData }));
       setActiveDragData(null);
     },
-    [dispatch]
+    [activeDragData, dispatch]
   );
 
   const mouseSensor = useSensor(MouseSensor, {