Is there any way of increasing application flash memory via custom partition map? (ie. Kill OTA partitions, etc)

[SoftwareGuy]

Hardware:

Board: Generic ESP32 Dev Module
Core Installation/update date: Latest
IDE name: PlatformIO
Flash Frequency: 40Mhz (default?)
Upload Speed: 115200

Description:

For some projects, 1.25MB of Flash memory to store your program code is not enough. Like the title suggests, is there any way to increase the amount of flash memory available? Could we kill off the OTA and OTA update partitions if we're not using it? Personally, I don't know what OTA is nor do I want to use it, so to have a dedicated partition just wasting space is kinda silly in my eyes.

I did see on the ESP32 forums a way to increase the 1.25MB partition to 2MB but I'm not sure if that was applicable for ESP32 or ESP8266 only.

Sketch:

N/A

Debug Messages:

N/A

[beegee-tokyo]

With latest version of PlatformIO very easy.
Create your own partition table definition (e.g. custompart.csv) within the project folder (same level as platformio.ini)
Add board_build.partitions = $PROJECT_DIR/custompart.csv to platformio.ini
Enjoy your new partitions
See issue and solution

Just make sure you understand which partitions are necessary and how to write a partition table description.
Here is an example of mine

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x10000, 0x1E0000,
app1,     app,  ota_1,   0x1F0000,0x1E0000,
spiffs,   data, spiffs,  0x3F0000,0x10000,
#eeprom,   data, 0x99,    0x3F0000,0x1000,
#spiffs,   data, spiffs,  0x3F1000,0xF000,

I removed EEPROM partition, made SPIFFS very small and increased space for application.

Get more information how to write custom partition tables from the docs

[SoftwareGuy]

Oh thank you so much! It'll be great to have more space to be able for my project (I'm using the ESP32 in a game project).

What is the EEPROM partition? Is it an emulated EEPROM or does it have some internal use?

[beegee-tokyo]

EEPROM is emulated in Flash. It is not used by FreeRTOS, only by your app if you decide to use it. I think it is there only for compatibility with Arduino. I use Prefernces library instead. Much easier.

[SoftwareGuy]

Thanks for confirming. I defined my ESP32 partitions like so:

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x10000, 0x372000,
eeprom,   data, 0x99,    0x382000,0x1000,
spiffs,   data, spiffs,  0x383000,0x7D000,

So I have a 512KB SPIFFS partition, around 3.5MB partition for app code, and the rest is system use. Do you see any issues here?

[beegee-tokyo]

Should work. As far as I know, with only app0 OTA will not work. And if you do not use eeprom nor spiffs then you can even extend app0 to the max size

[SoftwareGuy]

Forgot to comment but can confirm the custom flash memory partition map is working! PlatformIO now says 3.45MB flash than just 1.25MB flash, which is great.

I'll close this for now. Thank you again for your help.

[ASL07]

Hi,

Forgive for the newbie question but, how do you calculate partition size from in bytes from this?

**# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x372000,
eeprom, data, 0x99, 0x382000,0x1000,
spiffs, data, spiffs, 0x383000,0x7D000,
**

I am trying to figure out how to change the size of the partitions for my project and I couldn't find how to make the conversion anywhere.

[prabbit237]

Hi,

Forgive for the newbie question but, how do you calculate partition size from in bytes from this?

**# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x372000,
eeprom, data, 0x99, 0x382000,0x1000,
spiffs, data, spiffs, 0x383000,0x7D000,
**

I am trying to figure out how to change the size of the partitions for my project and I couldn't find how to make the conversion anywhere.

You need to figure out what sizes you want, first. The next to the last column is the Offset (how far from start does it start at?) The last column is the size. And remember, certain things have to start at a specific spot, some memory is reserved and it won't let you start on odd numbers (not sure if they only have to be even or if they have to be in multiples of 16 or 256, etc.)

Memory from 0 to 0x8fff is reserved
nvs, data, nvs, 0x9000, 0x5000, This one has to stay. Don't change it at all.
Notice this section starts at 0x9000 and is
0x5000 in size, thus right up to the next one

otadata, data, ota, 0xe000, 0x2000, Ditto...well, they MAY be changeable but I wouldn't try it.
0xe000+0x2000=0x100000 So we've used up all this flash for the system

app0, app, ota_0, 0x10000, 0x372000, The app0 MUST start at 0x10000. No way to move that,
starting point either down or (probably) up. So it starts at 0x10000 and is 0x372000 in size. It's 0x372000 in size, or roughly 3.6Mb.

eeprom, data, 0x99, 0x382000,0x1000,
spiffs, data, spiffs, 0x383000,0x7D000,
You could leave these last two out and gain another close to 1/2Mb (just change the size on the App0.

Also worth noting, the next to the last column (size) seems totally unneeded. If left out, it'll start it at the end of the previous section.

To use this, it depends on your compiler. Arduino needs the boards file changed. PlatformIO looks for a .csv file created and mentioned in the .ini file

So one project is using:

Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, , 1024K, (the size can be in decimal and with size qualifiers.)
app1, app, ota_1, , 0K, (probably not even needed)
spiffs, data, spiffs, , 0x2f0000,

Then in platform.io, I have:

[env:heltec_wifi_kit_32]
platform = espressif32
board = heltec_wifi_kit_32
framework = arduino
upload_speed = 115200
board_build.partitions = coffee-partition.csv
lib_deps =
MAX6675_Thermocouple
u8g2
elapsedMillis
NTPClient

(This is eventually going to be a coffee timer and also over-heat alert. They burn up WAY too many pots at work by letting them run dry.)

[khoat0]

Hi,
Forgive for the newbie question but, how do you calculate partition size from in bytes from this?
**# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x372000,
eeprom, data, 0x99, 0x382000,0x1000,
spiffs, data, spiffs, 0x383000,0x7D000,
**
I am trying to figure out how to change the size of the partitions for my project and I couldn't find how to make the conversion anywhere.

You need to figure out what sizes you want, first. The next to the last column is the Offset (how far from start does it start at?) The last column is the size. And remember, certain things have to start at a specific spot, some memory is reserved and it won't let you start on odd numbers (not sure if they only have to be even or if they have to be in multiples of 16 or 256, etc.)

Memory from 0 to 0x8fff is reserved
nvs, data, nvs, 0x9000, 0x5000, This one has to stay. Don't change it at all.
Notice this section starts at 0x9000 and is
0x5000 in size, thus right up to the next one

otadata, data, ota, 0xe000, 0x2000, Ditto...well, they MAY be changeable but I wouldn't try it.
0xe000+0x2000=0x100000 So we've used up all this flash for the system

app0, app, ota_0, 0x10000, 0x372000, The app0 MUST start at 0x10000. No way to move that,
starting point either down or (probably) up. So it starts at 0x10000 and is 0x372000 in size. It's 0x372000 in size, or roughly 3.6Mb.

eeprom, data, 0x99, 0x382000,0x1000,
spiffs, data, spiffs, 0x383000,0x7D000,
You could leave these last two out and gain another close to 1/2Mb (just change the size on the App0.

Also worth noting, the next to the last column (size) seems totally unneeded. If left out, it'll start it at the end of the previous section.

To use this, it depends on your compiler. Arduino needs the boards file changed. PlatformIO looks for a .csv file created and mentioned in the .ini file

So one project is using:

Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, , 1024K, (the size can be in decimal and with size qualifiers.)
app1, app, ota_1, , 0K, (probably not even needed)
spiffs, data, spiffs, , 0x2f0000,

Then in platform.io, I have:

[env:heltec_wifi_kit_32]
platform = espressif32
board = heltec_wifi_kit_32
framework = arduino
upload_speed = 115200
board_build.partitions = coffee-partition.csv
lib_deps =
MAX6675_Thermocouple
u8g2
elapsedMillis
NTPClient

(This is eventually going to be a coffee timer and also over-heat alert. They burn up WAY too many pots at work by letting them run dry.)

Thank you so much. I have success with your help.

[mahboud]

Here's another way to increase the application partition size:

#339 (comment)