+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Ouroboros Flash Reader
+Patrick Schönberger
+16.07.2025
+Introduction
+-
+
- Part of Project FIENDISH +
- Automatically extracts files from ESP-based Shelly smart home +devices +
- Can run on a Raspberry Pi or a micro controller +
+
+
The Problem
+How to access data on Shelly devices?
+-
+
- Extract the flash memory and read the data +
- Do it read-only and document the process +
-
+
- Most Shelly devices are based on ESP micro controllers +
- Existing solutions (esptool, mos) can modify memory and are not +fully automated +
- They are very complex, and thus hard to modify +
- How about a custom tool? +
Talking to the ESP
+-
+
- What is the bare minimum needed to talk to an ESP? +
- Two modes: boot and run +
- Serial pins are exposed on some Shelly devices and accessible on +others +
- In boot mode they can be used to communicate using a custom +protocol +
+
+
+
+
+
+
-
+
- What can we do with the protocol?
+
-
+
- Synchronize +
- Write RAM, flash and registers +
- Configure memory etc. +
- On the ESP32 it can read flash, but not on the ESP8266, ESP32C3 or +ESP32C6 +
+
+
+
https://docs.espressif.com/projects/esptool/en/latest/esp32/advanced-topics/serial-protocol.html
+ +Reading Flash Memory
+-
+
- We cannot directly read flash, but we can write RAM +
- Write a program, load it into RAM and run it on the ESP +
- This program is called flash loader or flasher stub +
- Esptool uses two versions:
+
-
+
- Legacy C version +
- New Rust version +
+
$ cloc esp-flasher-stub/
+-----------------------------------------------------------
+Language files blank comment code
+-----------------------------------------------------------
+Rust 12 327 78 1863
+Logos 14 32 0 249
+YAML 3 34 12 214
+Markdown 1 34 0 89
+TOML 4 8 2 61
+-----------------------------------------------------------
+SUM: 34 435 92 2476
+-----------------------------------------------------------$ cloc esp-hal
+-----------------------------------------------------------
+Language files blank comment code
+-----------------------------------------------------------
+Rust 492 18739 26120 115809
+Linker Script 51 499 1404 11315
+Markdown 66 2014 12 5262
+TOML 43 505 412 4481
+Logos 44 229 15 1105
+YAML 13 151 67 1054
+Jinja Template 3 52 0 255
+JSON 2 0 0 48
+CSV 3 0 0 21
+SVG 1 0 0 4
+-----------------------------------------------------------
+SUM: 718 22189 28030 139354
+-----------------------------------------------------------$ cloc esptool-legacy-flasher-stub/
+-----------------------------------------------------------
+Language files blank comment code
+-----------------------------------------------------------
+Linker Script 32 787 1188 18751
+C 6 1284 636 7689
+C/C++ Header 8 463 557 1687
+make 1 36 33 130
+YAML 4 19 0 114
+Python 2 25 21 80
+Markdown 3 58 0 76
+Bourne Shell 2 9 8 23
+TOML 1 1 0 20
+Jinja Template 1 4 2 14
+-----------------------------------------------------------
+SUM: 60 2686 2445 28584
+------------------------------------------------------------
+
- C version is deprecated but it is much simpler and supports +ESP8266 +
- Customize this version by removing write and erase commands +
- Technically there is write access until the flash loader is +activated +
- The extraction tool is small and contains no write or erase +commands +
Serial Protocol
+-
+
- Bytes are grouped using SLIP frames +
- Host sends requests containing commands +
- Target sends a matching reply +
https://docs.espressif.com/projects/esptool/en/latest/esp32/advanced-topics/serial-protocol.html
+Steps to Read Flash
+-
+
- Sync +
- Identify Chip +
- (Read MAC) +
- (Change baud rate) +
- Upload Stub +
- Read Flash +
Modifying the Flash Loader
+-
+
Consists mostly of six C files:
+
+
Uploading the Flash Loader
+-
+
- Compile using specific toolchains +
- Extract text and data sections from resulting ELF file +
- Generate header file that is compiled with the extraction tool +
- This includes sectionâs sizes, addresses and the entry point +
- Upload the corresponding loader after identifying the chip +
const unsigned char *elf_esp32_text_buffer =
+ (unsigned char[]){0x08,0x00,0xf4,...};
+const unsigned long elf_esp32_text_size = 2100;
+const unsigned long elf_esp32_text_addr = 1074520064;
+
+const unsigned char *elf_esp32_data_buffer =
+ (unsigned char[]){0x9b,0xe6,0x0b,...};
+const unsigned long elf_esp32_data_size = 60;
+const unsigned long elf_esp32_data_addr = 1073561756;
+
+const unsigned long elf_esp32_entry = 1074521000;Different ESP Versions
+-
+
- ESP32-C3 and later are identified using
+
GET_SECURITY_INFOcommand
+ - Earlier versions have a register with a magic value +
- Try security info first and reset on failure +
- Different registers containing MAC address +
Boot Mode
+-
+
- GPIO0 low at start +
- Wire pins to RST and GPIO0 +
- Pull both low +
- Then pull RST high first and GPIO0 after +
- Supported when using Raspberry Pi or ESP as host +
Summary
+-
+
- Connect to serial and set to boot mode +
- Use serial protocol to identify the chip +
- Upload modified flash loader +
- Extract +
Extracting File Systems
+-
+
- SPIFFS/Littlefs +
- Gen 1 devices have known layout +
- On newer devices file system is found using metadata +
- Files can be extracted using specialized tools +
Shelly Plus 1 File System
+api_math.js
+ca.pem // signing CA
+conf0.json // empty configuration
+conf3.json // initial configuration
+conf9.json // current configuration
+index.html.gz // web frontend
+init.js
+rpc_acl_auth.json // list of rpc endpoints
+rpc_acl_no_auth.json
+shelly_cloud.pem // Shelly Cloud certificate
+shelly_plugin_api.js
+storage.json // user data
+tzinfo
+updater.datconf9.json
+
+
+shelly_cloud.pem
+Subject: O = Allterco
+Not Before: Aug 4 12:03:41 2020 GMT
+Not After : Aug 2 12:03:41 2030 GMT
+-----BEGIN CERTIFICATE-----
+MIICrTCCAZUCFCuIEEAQJOFLZuEtr/CWkvxi9YPAMA0GCSqGSIb3DQEBCwUAMBMx
+ETAPBgNVBAoMCEFsbHRlcmNvMB4XDTIwMDgwNDEyMDM0MVoXDTMwMDgwMjEyMDM0
+MVowEzERMA8GA1UECgwIQWxsdGVyY28wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
+ggEKAoIBAQDHGDBDHpPUbtC9QAAjX3bi487AVY5JgYB2gyp6R9cjdsNGMbYnWdxn
+BmsIUKJPg7B5NcQObiMe6djUvwo0c2Xl9L+P9LOskP2WNDdpquX3XJu580hXHHVB
+mwOgJ0fi+5U9mOFHhc1gYGLmhO9oqsE80SgpmsPQHloMIqmcaolLzgC9PWGu8nSD
+ToJq+dXyNFHzLVyBEugHQpeIR8Fq0do4dtlsfTWvv9U+fpGPegjdkPenSxGrOVwd
+syFzNahxQGKmpZE/1fsq5QSh9+ZgwpdDChVNpkj9TBC1ApDTUasNco/6Meb/0Xur
+pxpWPNfkIpZ7ebtGHVd/ZkGTPUnL7FXHAgMBAAEwDQYJKoZIhvcNAQELBQADggEB
+ADwbvD7Mf7SOinV8JkOue8D/tvp+OiYTYLHYppzCLcBK3D1kQ7aqla2T8ebEFbLh
+hpau7MxJcizVWZs4vJvFYxEwBTQldobhmG5lHnoKQcOSxis1kWr5xdKhji/QYo6T
+wS/cird9hAcuc+RmLCMdpEDyia/vX+vvvRdyKsmB7A6Vkdu8s2B2jlhQNkitYzvq
+UDKogJrWe6fQUTpTThMyGbqhp9cQ64M4DJG1cwSBZ/hiUAMKO/y5WVNWFbXIb/Om
+xwkXgof2RXN1AjjqMcBh3GNVK4ZV5XM9WCocZjOCi2yZxaxaRWyuGR7EqAQZ+wdr
+P5XrzYspeVl1WtBzwGFssPc=
+-----END CERTIFICATE-----