add ROS2 rgb camera plugin #24

yuokamoto · 2021-12-12T13:46:04Z

No description provided.

Source/RapyutaSimulationPlugins/Private/Sensors/ROS2Camera.cpp

Source/RapyutaSimulationPlugins/Private/Sensors/ROS2CameraActor.cpp

Source/RapyutaSimulationPlugins/Private/Sensors/ROS2CameraComponent.cpp

Tadinu · 2021-12-28T03:27:18Z

Source/RapyutaSimulationPlugins/Private/Sensors/ROS2CameraComponent.cpp

+FROSImage UROS2CameraComponent::GetData()
+{
+    SceneCaptureComponent->CaptureScene();
+    CaptureNonBlocking();
+    if(!RenderRequestQueue.IsEmpty()){
+        // Peek the next RenderRequest from queue
+        FRenderRequest* nextRenderRequest = nullptr;
+        RenderRequestQueue.Peek(nextRenderRequest);
+
+        if(nextRenderRequest){ //nullptr check
+            if(nextRenderRequest->RenderFence.IsFenceComplete()){ // Check if rendering is done, indicated by RenderFence
+                for (int I = 0; I < nextRenderRequest->Image.Num(); I++)
+                {
+                    Data.data[I * 3 + 0] = nextRenderRequest->Image[I].R;
+                    Data.data[I * 3 + 1] = nextRenderRequest->Image[I].G;
+                    Data.data[I * 3 + 2] = nextRenderRequest->Image[I].B;
+                    // UE_LOG(LogTemp, Warning, TEXT("AsyncTaskDone"));
+                }
+
+                // Delete the first element from RenderQueue
+                RenderRequestQueue.Pop();
+                delete nextRenderRequest;
+            }
+        }
+    }    


Since here we do CaptureNonBlocking(), which means the GameThread is non-blocking, and the capturing (pixel reading from RHI) could possibly take some time before RHI returns the data.
Thus, if(nextRenderRequest->RenderFence.IsFenceComplete()) might not be instantly satisfied.

But more essentially, an instant data fetching function named GetData() could be inevitably blocking, if users want to invoke a non-blocking version, they might want to fetch data at a later time, when it is available from RHI, after invoking CaptureNonBlocking()?

References:

TimmHess actually runs the fence check in a tick then do async image saving.

Another one of async fetching style by off-loading the fence check to a thread.

Source/RapyutaSimulationPlugins/Private/Sensors/ROS2CameraActor.cpp

Source/RapyutaSimulationPlugins/Public/Sensors/ROS2CameraComponent.h

* add rgb camera plugin

yuokamoto mentioned this pull request Dec 12, 2021

Camera sensor support? #23

Closed

yuokamoto marked this pull request as ready for review December 13, 2021 03:41

Tadinu reviewed Dec 13, 2021

View reviewed changes

Source/RapyutaSimulationPlugins/Private/Sensors/ROS2Camera.cpp Outdated Show resolved Hide resolved

yuokamoto force-pushed the camaera_plugin branch 2 times, most recently from be4d238 to a65d276 Compare December 26, 2021 05:57

yuokamoto requested review from Tadinu and tetorea December 26, 2021 06:00

tetorea approved these changes Dec 27, 2021

View reviewed changes