Crash when using Mat.atNum traverse pixel for a jpeg image

[cuishijie1991]

Hi,
I use opencv_dart to traverse the image pixels, use the Floyd-Steinberg algorithm to process the value of each pixel, and reassign the value to use the dithering effect.
Everything was fine at the beginning of the test, but when I tested a jpeg image, I found that the Mat.atNum method started to throw an exception when it executed a certain pixel point.
I tested iOS and Android platforms, and this problem occurred for this image. I will provide the test code and images. If possible, please help me find the reason, thank you!

1. image file
   

2. test sample

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  File? _image;
  Uint8List? _bytes;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: _image == null
            ? Text("no image")
            : (_bytes == null
                ? Image.file(
                    _image!,
                    width: 300,
                  )
                : Image.memory(
                    _bytes!,
                    width: 300,
                  )),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: () {
          ImagePicker.platform
              .getImageFromSource(source: ImageSource.gallery)
              .then((f) {
            if (f != null) {
              setState(() {
                _bytes = null;
                _image = File(f.path);
              });
              Future.delayed(
                  Duration(seconds: 2),
                  () => cvDitherImage1(_image!.readAsBytesSync()).then((b) {
                        setState(() {
                          _bytes = b;
                        });
                      }));
            }
          });
        },
        tooltip: 'Increment',
        child: const Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

///利用Floyd-Steinberg算法处理图片
Future<Uint8List?> cvDitherImage1(Uint8List src, {int threshold = 127}) async {
  cv.Mat im = cv.imdecode(src, cv.IMREAD_GRAYSCALE);
  try {
    if (im.isEmpty) {
      return null;
    }
    int w = im.width;
    int h = im.height;
    List<int> pixelCache = []; //时间主要耗费在通过ffi读写图片像素上，做一层内存缓存，保证从图片读写只用一次
    //把像素读入缓存
    for (var y = 0; y < h; y++) {
      for (var x = 0; x < w; x++) {
        pixelCache.add(im.atNum(x, y) as int);
      }
    }
    setPixelWithError(int x, int y, double error) {
      if (x < 0 || x >= w || y < 0 || y >= h) {
        return;
      }
      pixelCache[y * w + x] =
          (pixelCache[y * w + x] + error).clamp(0, 255).toInt();
    }

    //执行算法，修改缓存中的像素
    for (var y = 0; y < h; y++) {
      for (var x = 0; x < w; x++) {
        int oldPixel = pixelCache[y * w + x];
        int newPixel = oldPixel > threshold ? 255 : 0;
        pixelCache[y * w + x] = newPixel;
        var error = oldPixel - newPixel;
        // Floyd-Steinberg Dithering 误差扩散权重
        setPixelWithError(x + 1, y, error * 7 / 16); // 右边
        setPixelWithError(x - 1, y + 1, error * 3 / 16); // 左下
        setPixelWithError(x, y + 1, error * 5 / 16); // 正下
        setPixelWithError(x + 1, y + 1, error * 1 / 16); // 右下
      }
    }
    //缓存中的像素，写回图片
    for (var y = 0; y < h; y++) {
      for (var x = 0; x < w; x++) {
        im.setNum(x, y, pixelCache[y * w + x]);
      }
    }
    var (_, bytes) = cv.imencode(".jpg", im);
    return bytes;
  } finally {
    im.dispose();
  }
}

3. android crash messages

---

Build fingerprint: 'Xiaomi/fuxi/fuxi:15/AQ3A.240912.001/OS2.0.3.0.VMCCNXM:user/release-keys'
Revision: '0'
ABI: 'arm64'
Timestamp: 2025-01-18 14:38:10.513210568+0800
Process uptime: 17s
Cmdline: com.example.test
pid: 24553, tid: 24712, name: 1.ui >>> com.example.test <<<
uid: 10417
tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE)
pac_enabled_keys: 000000000000000f (PR_PAC_APIAKEY, PR_PAC_APIBKEY, PR_PAC_APDAKEY, PR_PAC_APDBKEY)
signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0xb40000706ee7f000
x0 0000006f006db4c1 x1 0000006f06e586a9 x2 0000000000000000 x3 b40000706ee7f000
x4 0000006f0000a431 x5 0000006f06dde861 x6 0000006f06dde8b0 x7 0000000000001682
x8 0000000000000007 x9 0000000000000047 x10 000000709ac3bbb8 x11 0000000000000001
x12 000000000003c020 x13 0000000000000000 x14 0000000000000001 x15 00000070a38ce618
x16 00000070a36eb000 x17 0000006f06dde8c1 x18 0000007091474000 x19 000000706ee90e20
x20 b400007181334c60 x21 b40000721138f990 x22 0000006f00008081 x23 0000006f0043d779
x24 0000006f06da7391 x25 00000070a36ea000 x26 b40000721138f990 x27 0000006f00009c70
x28 000000080000006f x29 00000070a38ce628
lr 000000706ee910a8 sp 00000070a36ea000 pc 000000706ee877b0 pst 0000000000001000
1 total frames
backtrace:
#00 pc 00000000000077b0 [anon:dart-code]

4. iOS crash message

• thread Remove the dependency of Flutter #9, name = 'io.flutter.1.ui', stop reason = EXC_BAD_ACCESS (code=1, address=0x131224010)
  frame #0: 0x0000000138116524
  -> 0x138116524: ldrb w1, [x0]
  0x138116528: mov x0, x1
  0x13811652c: lsl x1, x0, Support macOS apple silicon #1
  0x138116530: mov x0, x1
  Target 0: (Runner) stopped.

[rainyl]

@cuishijie1991 Sorry for the late reply.

I have just tested your code and there are some errors.

First of all, always use Mat.at (including Mat.atNum etc.) and Mat.set carefully!!!

For better performance, these methods do not have bound check, and will operate native pointer directly, which means, if you do not check the bound manually, there will be a risk of pointer invalid access issue.

Now, for your app, check

for (var y = 0; y < h; y++) {
      for (var x = 0; x < w; x++) {
        pixelCache.add(im.atNum(x, y) as int); // here, you should use atNum(y, x)
      }
    }

and

for (var y = 0; y < h; y++) {
      for (var x = 0; x < w; x++) {
        im.setNum(x, y, pixelCache[y * w + x]); // you should use setNum(y, x)
      }
    }

Now, it works,

However, your code could be improved.

1. This project supports asynchronous now, you can just use cv.imdecodeAsync, cv.imencodeAsync, etc.
2. The most cost when operating Mat pixels is the property getting, i.e., getting the total rows/cols/depth every time at or set is called, that's why Mat.forEachPixel was added, you can operate pixels via Mat.forEachPixel, maybe in the future I will add more detailed functions like atU8, atF32, etc.
3. If your app is performance sensitive, you can also ust ptrAt<> to get the native pointers and set the values directly.
4. If you need to cache the pixels, the most convenient way is copying the Mat.data.

Hope it helps.

[cuishijie1991]

@rainyl
Thank you for your time and patience.
This is indeed my own problem. I mistakenly passed the row and col parameters in reverse but didn't notice it. At the beginning, I tested many pictures with equal width and height, so I mistakenly pointed the problem to the library. I will closed this issue.
Your suggestions are also very helpful to me.
Because the algorithm needs to modify the pixel value of a certain point many times, I cache it in memory for processing. Now I copy Mat.data directly, which takes 40% of the time of copying point by point before, much faster than before !