Add WeMos D1 mini pro

[ordonot]

Please add support for WeMos D1 mini pro (16 MBytes)
https://www.wemos.cc/product/d1-mini-pro.html

(Moved my wish on behalf of ivankravets from platformio/platformio-core#798 - hope I got the right place now.)

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/38047911-add-wemos-d1-mini-pro?utm_campaign=plugin&utm_content=tracker%2F38211776&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F38211776&utm_medium=issues&utm_source=github).

[ivankravets]

I don't see this board here https://github.com/esp8266/Arduino/blob/master/boards.txt

Is the only 1 difference Flash size?

[ordonot]

Yes, but seems to be the first one with SPIFFS >4MBytes so esptool needs an update too.
esp8266/Arduino#2323
esp8266/Arduino#2351

[mcauser]

Waiting on this: esp8266/Arduino#2550

[adrianmihalko]

Guys, any update on this?

[martinayotte]

Using the hack details here http://www.packom.org/esp8266/16mb/flash/eeprom/2016/10/14/esp8266-16mbyte-flash_handling.html, I was able to get SPIFFS working on 16MB, but there still issues : SDK still try to access WiFi configs like 4MB, so I had to move SPIFFS_START after the first 4MB, with 12MB remaining. Also, I still face issue with OTA, when e-boot is call, I get "need boot 1.4+", and hangs forever until doing a erase_flash with esptool.py.

[ivankravets]

Does it work in Arduino IDE?

[mark-hahn]

Bump. I just got my D1 mini pros and I need the extra file space on my product. Anything I can do to help?

[martinayotte]

As mentioned earlier, the hack provided in http://www.packom.org/esp8266/16mb/flash/eeprom/2016/10/14/esp8266-16mbyte-flash_handling.html is the way to go.

[mark-hahn]

Thanks for the reply.

I tried accessing SpiFlashChip but platformIO doesn't have the library. What library should I use? His routines would get me the ability to write and read the flash but only in a way like the EEProm which isn't very useful.

Secondly, it's kind of hard to call it a workaround when the upload tools and the file system don't work. I saw the reply that had some kludges for the FS but I didn't understand what he was suggesting.

[martinayotte]

I'm only play a bit with PlateformIO, I'm still using plain ArduinoESP framework.
But PlateformIO has a copy of this framework, so it has also the SPIFFS there too.
The changes needed should be done in Esp.cpp, where there are method EspClass::flashEraseSector()/EspClass::flashWrite()/EspClass::flashRead().
The hack mentioned above should be done in this file. Then, you will still use normal upload tools like before with 4M, but you also have to change LD script so that _SPIFFS_start is equal to 0x40600000 instead of 0x40300000 to avoid that WiFi configs settings been in the middle of SPIFFS. This hack will give 12MB available in SPIFFS.
The only pitfall is that OTA isn't working anymore, I will have to dig a bit more when I get chance :
To fix that issue, _SPIFFS_start need to be dynamically reverted to 0x40300000 when OTA is done.

[mark-hahn]

Thanks, I'm keeping this information. However it is a messy fix and also I need OTA. I can wait a while for a simple solution.

[valtersitz]

Hi everybody,

First thank you for developping such useful tools for the ESP!
I am currently working on the ESP WEMOS D1 mini pro and I have difficulties uploading my code on it (same as you I guess). I took a look at the hack solution but I need the OTA and SPIFFs to both work for my project. As any improvment been released since January? If not do you have an idea of when it will be out? Should I look for an other solution in the meantime?
I am novice with Platform.io and my project includes js, css and py files.

Thank you in advance,
Val

[martinayotte]

You can still use the Pro 16MB as a simple 4MB in the mean time we found a fix for having full usage of the 16MB, it still give you pretty much space.

[mark-hahn]

in the mean time we found a fix

Did you mean that you have a fix now, or to wait until you find a fix?

[martinayotte]

As mentioned above in January, there is a hack to use the above 12MB in SPIFFS, but it make OTA becomes broken until we found another hack to fix the OTA issue. If you use the 16MB as a simple 4GB, there is no issue other than loosing the above 12MB.

[javigf]

Would be nice to use Wemos PRO. Thanks I´ll try the hack

[martinayotte]

Ok ! I took time to investigate a bit more : to avoid the issue with OTA, this OTA still need to be done in low space location, not just before the SPIFFS_START. So, I added a check in Updater.cpp in the begin() method :

//address of the end of the space available for sketch and update
uint32_t updateEndAddress = (uint32_t)&_SPIFFS_start - 0x40200000;

• if (updateEndAddress > 0x100000) updateEndAddress = 0x100000;
  //size of the update rounded to a sector

So, it seems to work, I've now a 12MB partition, and OTA still working. Of course, I'm loosing the space between addresses 0x100000 and 0x400000, but it can still be use using raw EspClass::flashWrite/EspClass::flashRead for other purposes than SPIFFS.

[mark-hahn]

Of course, I'm loosing the space between addresses 0x100000 and 0x400000,

Does this mean code has to be smaller than 0x100000/2 (512 KB)? Normally it can be 2 MB, right?

Edit: Oh, you mean you are losing the SPIFF between 0x100000 and 0x400000. The code space is still the same, right?

[martinayotte]

Does this mean code has to be smaller than 0x100000/2 (512 KB)? Normally it can be 2 MB, right?

Not in ArduinoESP, the config 4MB/3MB SPIFFS is still 2x512KB. For the config 4MB/1MB SPIFFS, I didn't try it out, but it should be 2x 768KB.

Edit: Oh, you mean you are losing the SPIFF between 0x100000 and 0x400000. The code space is still the same, right?

Code space is still limited to 1M, whatever we try to do, it is in an Espressif limitation.
I mean I could not have two SPIFFS partitions for now, so this aera can still be use for somthing else using raw SPI flash access ...

[mark-hahn]

For the config 4MB/1MB SPIFFS, I didn't try it out, but it should be 2x 768KB.

Wouldn't that config have 3MB code total or 2 x 1536 KB? If so then your fix would only waste 1MB which is only 6%. That is really good.

Code space is still limited to 1M

Does that mean 2 x 512K or 2 x 1M? Where does 2 x 768KB come from? I'm sorry to be such a pain but I don't really know much about this stuff. I'll try to be helpful with experiments to see what I can do with your fix.

[martinayotte]

My bad : I add in mind 1536KB, and I divided by two while writing ;-)
But, the usable space still 1MB out of this 1532KB.

[mark-hahn]

Let me check if I understand correctly. We should be able to use ...

• 1 MB of running code
• 1 MB of buffer for OTA
• 2 MB only usable for reading/writing flash
• 12 MB for SPIFFS

Is this correct? It would be nice if this setup was a choice in platformio.ini build flags.

[martinayotte]

Right ! except "2 MB only usable for reading/writing flash minus the WiFi configs and EEPROM"
I'm a bit lazy, and still using Arduino IDE...
But it is only requires to do patching mentioned above and tweak SPIFFS defines as well as LD script.

[cumbaya]

@martinayotte May you please explain all steps to use 12MByte SPIFFS and OTA in ArduinoIDE?
What files have to be edited? updater.cpp, boards.txt, *.ld ?
What changes do I have to apply?
Thanks in advance

[martinayotte]

For board.txt, I've created a 16MB(12MB) section in Wemos-D1Mini by copy/pasting another definition and provide spiffs_start at 0x300000 and spiffs_end at 0x17FB00.
Then, I've changed Esp.cpp to use the first hack mentioned above with code like the following in every flash functions (here is shown for flashWrite, but the same is needed to flashRead and flashEraseSector) :

bool EspClass::flashWrite(uint32_t offset, uint32_t *data, size_t size) {
    ets_isr_mask(FLASH_INT_MASK);
+    int32_t flash_size_sdk = flashchip->chip_size;
+    flashchip->chip_size = getFlashChipRealSize();
    int rc = spi_flash_write(offset, (uint32_t*) data, size);
+    flashchip->chip_size = flash_size_sdk;
    ets_isr_unmask(FLASH_INT_MASK);
    return rc == 0;
}

And for the OTA, to prevent getting located just under the SPIFFS, I forced it to stay at 0x100000 in Updater.cpp :

    //address of the end of the space available for sketch and update
    uint32_t updateEndAddress = (uint32_t)&_SPIFFS_start - 0x40200000;
+    if (updateEndAddress > 0x100000) updateEndAddress = 0x100000;
    //size of the update rounded to a sector

Finally, yes, the LD script also need to be tweaked with spiffs_start at 0x300000 too.

[cumbaya]

Thanks a lot, @martinayotte
I modified boards.txt an used the generic-section:

generic.menu.FlashSize.16M=16M (12M SPIFFS) generic.menu.FlashSize.16M.build.flash_size=1M generic.menu.FlashSize.16M.build.flash_ld=eagle.flash.16m.ld generic.menu.FlashSize.16M.build.spiffs_start=0x300000 generic.menu.FlashSize.16M.build.spiffs_end=0x17FB00 generic.menu.FlashSize.16M.build.spiffs_blocksize=8192

So, I have to change eagle.flash.16m.ld, right?
This file looks like that:

/* Flash Split for 16M chips /
/ sketch 1019KB /
/ eeprom 20KB /
/ spiffs 15MB */

MEMORY
{
dport0_0_seg : org = 0x3FF00000, len = 0x10
dram0_0_seg : org = 0x3FFE8000, len = 0x14000
iram1_0_seg : org = 0x40100000, len = 0x8000
irom0_0_seg : org = 0x40201010, len = 0xfeff0
}

PROVIDE ( _SPIFFS_start = 0x40300000 );
PROVIDE ( _SPIFFS_end = 0x41200000 );
PROVIDE ( _SPIFFS_page = 0x100 );
PROVIDE ( _SPIFFS_block = 0x2000 );

INCLUDE "../ld/eagle.app.v6.common.ld"

Do I have to change _SPIFFS_start to 0x300000 and _SPIFFS_end to 0x17FB00?

[capedra]

@martinayotte, @cumbaya

PROVIDE ( _SPIFFS_start = 0x40300000 );
PROVIDE ( _SPIFFS_end = 0x411FB000 );

maybe??
How may I use the library "FS.h" to read/write/format files/dirs on Wemos Pro? I can't save any files using SPIFFS from "FS.h" after following your procedures.

[capedra]

@martinayotte :
I already tried every configuration but nothing seems to work with "FS.h" functions like open for writing and even write/println functions. I changed "_SPIFFS_start" to every possible memory address but still nothing. Please help.

[martinayotte]

In my setup, I moved the start above the 4M boundary, because ROM still access WiFi configs at the end of this first 4M.
PROVIDE ( _SPIFFS_start = 0x40600000 );
@capedra , did you modified the EspClass in Esp.cpp with the hack mentioned above ?

[capedra]

@martinayotte Yes, I already modified all the necessary files but still nothing. SPIFFS from "FS.h" keeps returning false for simple functions like format(). I already tried every _SPIFFS_start and _SPIFFS_end but the only configuration that works is the one from 4MB Wemos. Please help.
Edit 1:
The following start does not work at all for the functions declared on "FS.h" file:
PROVIDE ( _SPIFFS_start = 0x40600000 );

[martinayotte]

Are you calling SPIFFS.begin() before any other calls.
If No, it is pretty normal that functions such SPIFFS.format() are failing.
Other than that, I can't see why it is not working for you, since it does for me.

[capedra]

@martinayotte it's also returning false whenever I try to mount the flash file system using the call SPIFFS.begin().
I've already tried the following "_SPIFFS_end" but these issues persist:

PROVIDE ( _SPIFFS_end = 0x411FB000 );
PROVIDE ( _SPIFFS_end = 0x41200000 );
PROVIDE ( _SPIFFS_end = 0x4117FB00 );

[valtersitz]

Hi guys,
The following question is not 100% topic related but I am pretty sure it could help us all understand the memory layout management.
I was wondering if the memory mapping was fully customizable with the hacks @martinayotte provided?
If that so it would be great for both smaller (2MB) and larger (16MB) flash memories: we could have our code ( limited to 1MB), OTA space and some SPIFF space, everything at the right amount. And, if I am not wrong we could use the remaining space to read/write ROM, maybe even enlarging EEPROM space (as this is emulated I guess we could change the size of it).
I am willing to help but as I am not an expert I would like your feedback on this idea (is it even possible?) before starting! What are your thoughts?