[AE-49] Colour mode mismatch between CameraCaptureRawBytes.ino and CameraRawBytesVisualizer.pde

[aliphys]

Problem

😞 This is what happens when useGreyScale is set to true for the Nicla Vision:

😄 This is what is observed, when userGreyScale is set to false for the Nicla Vision

This creates three problems:

• The header comment and code in CameraRawBytesVisualizer conflict in what the default colour mode is
• From a UX perspective (cc @sebromero ) the User can face additional hurdles in testing out a basic function (e.g this forum post)
• There is no clear pathway for users to realise the problem, upon seeing the corrupt image.

Details

The CameraCaptureRawBytes sketch is used to send a video stream over Serial to the computer. Which can then be displayed with the CameraRawBytesVisualizer Processing sketch.

A series of pre-directives, configures the IMAGE_MODE to be either CAMERA_RGB (Nicla Vision and GIGA boards) or CAMERA-GREYSCALE (Portenta H7 family).

ArduinoCore-mbed/libraries/Camera/examples/CameraCaptureRawBytes/CameraCaptureRawBytes.ino

Lines 3 to 19 in 0504d34

	#ifdef ARDUINO_NICLA_VISION
	#include "gc2145.h"
	GC2145 galaxyCore;
	Camera cam(galaxyCore);
	#define IMAGE_MODE CAMERA_RGB565
	#elif defined(ARDUINO_PORTENTA_H7_M7)
	#include "hm0360.h"
	HM0360 himax;
	Camera cam(himax);
	#define IMAGE_MODE CAMERA_GRAYSCALE
	#elif defined(ARDUINO_GIGA)
	#include "ov767x.h"
	// uncomment the correct camera in use
	OV7670 ov767x;
	// OV7675 ov767x;
	Camera cam(ov767x);
	#define IMAGE_MODE CAMERA_RGB565

In the default configuration, the accompanying CameraRawBytesVisualizer sketch is configured to be compatible with CAMERA_GREYSCLE, even though the header comment states that the Processing sketch is for RGB.

ArduinoCore-mbed/libraries/Camera/extras/CameraRawBytesVisualizer/CameraRawBytesVisualizer.pde

Lines 1 to 7 in 0504d34

	/*
	This sketch reads a raw Stream of RGB565 pixels
	from the Serial port and displays the frame on
	the window.
	Use with the Examples -> CameraCaptureRawBytes Arduino sketch.
	This example code is in the public domain.
	*/

ArduinoCore-mbed/libraries/Camera/extras/CameraRawBytesVisualizer/CameraRawBytesVisualizer.pde

Lines 19 to 20 in 0504d34

	// Must match the image mode in the Arduino sketch
	final boolean useGrayScale = true;

---

Thanks @marqdevx , @Hannes7eicher and @jacobhylen for letting me pick your brains 🧠

[aliphys]

Replicated issue on Arduino GIGA R1 WiFi with OV7675 cc @karlsoderby

useGrayScale set to false

useGreyScale set to true

[aliphys]

Possible solution: Use keyPressed() method in Processing to switch the boolean value for useGrayScale.

Something like:

void keyPressed() {
  if (key == ' ') {
    useGrayScale = !useGrayScale;
  }
}

This does not require the user to change the useGrayScale variable manually, and can be switched without restarting the Processing sketch. -> Smoother user experience and less room for error

[rneurink]

See case CAMERA_GRAYSCALE:, it is commented out and currently GRAYSCALE sets to camera to use a bayer output. This is why the grayscale image looks really weird. If you uncomment these lines you can get it to work.

ArduinoCore-mbed/libraries/GC2145/gc2145.cpp

Lines 892 to 926 in 9795c3c

	int GC2145::setPixelFormat(int32_t pixformat)
	{
	int ret = 0;
	uint8_t reg;
	// P0 regs
	ret |= regWrite(GC2145_I2C_ADDR, 0xFE, 0x00);
	// Read current output format reg
	reg = regRead(GC2145_I2C_ADDR, REG_OUTPUT_FMT);
	switch (pixformat) {
	case CAMERA_RGB565:
	ret |= regWrite(GC2145_I2C_ADDR,
	REG_OUTPUT_FMT, REG_OUTPUT_SET_FMT(reg, REG_OUTPUT_FMT_RGB565));
	break;
	case CAMERA_GRAYSCALE:
	// TODO: There's no support for extracting GS from YUV so we use Bayer for 1BPP for now.
	//ret |= regWrite(GC2145_I2C_ADDR,
	// REG_OUTPUT_FMT, REG_OUTPUT_SET_FMT(reg, REG_OUTPUT_FMT_YCBYCR));
	//break;
	case CAMERA_BAYER:
	// There's no BAYER support so it will just look off.
	// Make sure odd/even row are switched to work with our bayer conversion.
	ret |= regWrite(GC2145_I2C_ADDR,
	REG_SYNC_MODE, REG_SYNC_MODE_DEF | REG_SYNC_MODE_ROW_SWITCH);
	ret |= regWrite(GC2145_I2C_ADDR,
	REG_OUTPUT_FMT, REG_OUTPUT_SET_FMT(reg, REG_OUTPUT_FMT_BAYER));
	break;
	default:
	return -1;
	}
	return ret;
	}