ESP-WROOM-32 MicroPython Memory Management

[danman009]

Hey all,

I am working on a hobby MicroPython project on my ESP-WROOM-32 board and trying to debug a memory management issue.

The project involves:

BLE (aioble) for communication with a mobile app
MQTT (umqtt.simple) for cloud communication
Both BLE and MQTT work fine at startup, and my MQTT client successfully runs every minute. However, I’ve run into a consistent memory issue when I interact with a BLE characteristic that triggers a process.

What Happens?

When I connect to a BLE characteristic and it returns a result, it completes successfully.
After this, when my MQTT client tries to run client.connect(), it either hangs or fails with an error code related to memory.
Using gc.mem_free(), I’ve verified that 40KB of memory is required for client.connect(), and this memory is available before the call.
I also noticed that the garbage collector runs after the BLE process finishes, even when I explicitly try to disable or prevent it.
The one time that GC did not run, MQTT connected without issue.
What I Suspect Is Happening:
I believe the issue isn't total memory available, but rather contiguous memory fragmentation.

What I’ve Tried So Far:

Preallocating a bytearray(40000) at class instantiation, then freeing it before client.connect() → Did not work
Disabling the garbage collector (gc.disable()) → Still ran unexpectedly
Setting gc.threshold(0) to reduce automatic GC behavior → Did not solve the issue
What I’m Looking For:

How does MicroPython handle large memory allocations internally?
Is there a reliable way to ensure that MQTT’s TLS handshake (client.connect()) always gets a contiguous 40KB block?
Are there any best practices for preventing memory fragmentation in MicroPython?
My memory usage is close to the limit, but I’m nearly finished with my design, so I’d really like to make it work in MicroPython instead of switching languages.

Any guidance would be greatly appreciated!

[Josverl]

I think you are spot on with memory fragmentation:

• Allocate large buffers early
• do not free, but re-use
• Use memory views to access parts of the buffer

Suggested reads:

• https://docs.micropython.org/en/latest/reference/constrained.html
• Wiki page : Memory Manager
• Search discussions on "Memory fragmentation"

[projectgus]

This may not fix the problem entirely, but we did just merge a fix (for upcoming release 1.25) for a problem reported where reconnecting with umqtt didn't work even though there was enough free memory (when using TLS): #16015

This will be in the next nightly build as well as the upcoming release, it may or may not make a difference for you.

my MQTT client successfully runs every minute.

If you're starting a new MQTT TLS connection every minute then this a lot of RAM churn. Do you have an option to keep the connection open and reuse it?