VFSFileImpl.exists() always returns true (IDFGH-1106)

[judge2005]

Environment

• Development Kit: ESP32-PICO-Kit|
• Kit version: v1
• Module or chip used: ESP32-PICO-D4
• IDF version: Arduino 1.0.2 (so v3.2)
• Build System: Make
• Compiler version: xtensa-esp32-elf-gcc.exe (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a5) 5.2.0
• Operating System: Windows
• Power Supply: USB

Problem Description

I'm using the SPIFFS file system. If a reference to the SPIFFS file system is passed to a function that expects a reference to a FS, the FS implementation of exists() is called. This always returns true, even for files that don't exist.

Expected Behavior

I should be able to use an FS& for a SPIFF& - SPIFF is a sub-class of FS. So exists() on the FS class should work for a SPIFFS filing system.

Actual Behavior

I can't!

Steps to reproduce

Build the included code and run it.

Code to reproduce this issue

#include "dirent.h"
#include "Arduino.h"
#include "SPIFFS.h"

void testSPIFFS(FS &fs, String path) {
	// Should print out "fs.exists(/somerandomfile) : 0". It doesn't
	Serial.printf("fs.exists(%s) : %d\n", path.c_str(), fs.exists(path));
}

void setup() {
	Serial.begin(115200);
	SPIFFS.begin();
	String path = "/somerandomfile";

	Serial.printf("SPIFFS.exists(%s) : %d\n", path.c_str(), SPIFFS.exists(path));
	testSPIFFS(SPIFFS, path);

	// The /spiffs part is prepended by one of the classes above, which is why it is used here.
	DIR *dir = opendir("/spiffs/somerandomfile2");
	
	// Should print out "opendir(/spiffs/somerandomfile2) : 0", but it doesn't
	Serial.printf("opendir(%s) : %d\n", "/spiffs/somerandomfile2", (int)dir);
}

void loop() {
}

Other items if possible

In the end, FS.exists() calls VFSFileImpl.exists() which ends up calling opendir(). opendir() always returns a DIR* for a file on the SPIFFS filing system if the file does not exist. All of this could be avoided if the methods on FS were virtual...

[igrr]

@judge2005 The classes you mention are implemented in the arduino-esp32 repository, not in ESP-IDF: https://github.com/espressif/arduino-esp32/tree/master/libraries/FS. Do I understand correctly that the bug report on the ESP-IDF side is the following?

opendir() always returns a DIR* for a file on the SPIFFS filing system if the file does not exist

(Regarding the fact that FS.exists doesn't do what is expected for a SPIFFS filesystem, I agree. In fact, the issue is that in the arduino-esp32, SPIFFS is an instance of fs::SPIFFSFS, which is a subclass of fs::FS. This is not the intended usage of FS abstraction layer. SPIFFS behavior should have been injected at the FSImpl layer, not by subclassing the fs::FS. Arduino-esp8266 repository demonstrates the correct usage: https://github.com/esp8266/Arduino/tree/master/libraries/SDFS/src, https://github.com/esp8266/Arduino/blob/master/cores/esp8266/spiffs_api.cpp. However, that is an issue for the arduino-esp32 repository).

[judge2005]

OK. I had assumed that the bug was in opendir() not in how SPIFFS had been implemented on the Arduino side. Or is it both? I.e should opendir() work with a spiffs filing system?

[igrr]

I think it is both:

• The fact that SPIFFSFS class shadows a non-virtual method of FS class is a bug; it can be solved by making exists a virtual method, or (closer to the original design) remove SPIFFSFS class and inject SPIFFS-specific functionality by means of an *Impl class. So SPIFFS object would be an instance of FS class, with its pimpl pointing to a SPIFFSFSImpl instance. Which approach to take would be up to the arduino-esp32 repo developers.
• opendir returning non-NULL for a non-existent path is clearly not expected behavior. However if you start reading about SPIFFS and realize that it doesn't have directories, then it kind of makes sense... From performance perspective, checking if a directory "exists" in SPIFFS is equivalent to iterating over all files in SPIFFS, and seeing if the directory is a prefix of any of the file names. So the "expected" behavior has a clear downside from performance point of view. As such, maybe we should rather document this behavior.

[judge2005]

Personally, I think opendir should work for a SPIFFS filing system and the warning should be that it could be slow. However, I also agree that the SPIFFS class should do the right thing, so will create an issue there too. I would be grateful if you would leave this open so I can refer to it.

BTW, open(fs, "r") has the same problem. I haven't tried any other functions (or argument variations).

[igrr]

Looks like this was fixed on Arduino side in espressif/arduino-esp32@fd5a2f0, closing.