Ouroboros Flash Reader

Patrick Schönberger

16.07.2025

Introduction

  • Part of Project FIENDISH
  • Automatically extracts files from ESP-based Shelly smart home devices
  • Can run on a Raspberry Pi or a micro controller

The Problem

How to access data on Shelly devices?

  • Extract the flash memory and read the data
  • Do it read-only and document the process

  • Most Shelly devices are based on ESP micro controllers
  • Existing solutions (esptool, mos) can modify memory and are not fully automated
  • They are very complex, and thus hard to modify
  • How about a custom tool?

Talking to the ESP

  • What is the bare minimum needed to talk to an ESP?
  • Two modes: boot and run
  • Serial pins are exposed on some Shelly devices and accessible on others
  • In boot mode they can be used to communicate using a custom protocol

  • What can we do with the protocol?
    • Synchronize
    • Write RAM, flash and registers
    • Configure memory etc.
    • On the ESP32 it can read flash, but not on the ESP8266, ESP32C3 or ESP32C6

https://docs.espressif.com/projects/esptool/en/latest/esp32/advanced-topics/serial-protocol.html

Reading Flash Memory

  • We cannot directly read flash, but we can write RAM
  • Write a program, load it into RAM and run it on the ESP
  • This program is called flash loader or flasher stub
  • Esptool uses two versions:
    • Legacy C version
    • New Rust version

$ cloc esp-flasher-stub/
-----------------------------------------------------------
Language          files       blank     comment        code
-----------------------------------------------------------
Rust                 12         327          78        1863
Logos                14          32           0         249
YAML                  3          34          12         214
Markdown              1          34           0          89
TOML                  4           8           2          61
-----------------------------------------------------------
SUM:                 34         435          92        2476
-----------------------------------------------------------

$ cloc esp-hal
-----------------------------------------------------------
Language          files       blank     comment        code
-----------------------------------------------------------
Rust                492       18739       26120      115809
Linker Script        51         499        1404       11315
Markdown             66        2014          12        5262
TOML                 43         505         412        4481
Logos                44         229          15        1105
YAML                 13         151          67        1054
Jinja Template        3          52           0         255
JSON                  2           0           0          48
CSV                   3           0           0          21
SVG                   1           0           0           4
-----------------------------------------------------------
SUM:                718       22189       28030      139354
-----------------------------------------------------------

$ cloc esptool-legacy-flasher-stub/
-----------------------------------------------------------
Language          files       blank     comment        code
-----------------------------------------------------------
Linker Script        32         787        1188       18751
C                     6        1284         636        7689
C/C++ Header          8         463         557        1687
make                  1          36          33         130
YAML                  4          19           0         114
Python                2          25          21          80
Markdown              3          58           0          76
Bourne Shell          2           9           8          23
TOML                  1           1           0          20
Jinja Template        1           4           2          14
-----------------------------------------------------------
SUM:                 60        2686        2445       28584
-----------------------------------------------------------

  • C version is deprecated but it is much simpler and supports ESP8266
  • Customize this version by removing write and erase commands
  • Technically there is write access until the flash loader is activated
  • The extraction tool is small and contains no write or erase commands

Serial Protocol

  • Bytes are grouped using SLIP frames
  • Host sends requests containing commands
  • Target sends a matching reply

https://docs.espressif.com/projects/esptool/en/latest/esp32/advanced-topics/serial-protocol.html

Steps to Read Flash

  • Sync
  • Identify Chip
  • (Read MAC)
  • (Change baud rate)
  • Upload Stub
  • Read Flash

Modifying the Flash Loader

  • Consists mostly of six C files:

    miniz.c             // compression
slip.c              // slip
stub_commands.c     // handle commands (modify)
stub_flasher.c      // main program
stub_io.c           // serial communication
stub_write_flash.c  // write flash (remove)

Uploading the Flash Loader

  • Compile using specific toolchains
  • Extract text and data sections from resulting ELF file
  • Generate header file that is compiled with the extraction tool
  • This includes section’s sizes, addresses and the entry point
  • Upload the corresponding loader after identifying the chip

const unsigned char *elf_esp32_text_buffer =
                    (unsigned char[]){0x08,0x00,0xf4,...};
const unsigned long  elf_esp32_text_size = 2100;
const unsigned long  elf_esp32_text_addr = 1074520064;

const unsigned char *elf_esp32_data_buffer =
                    (unsigned char[]){0x9b,0xe6,0x0b,...};
const unsigned long  elf_esp32_data_size = 60;
const unsigned long  elf_esp32_data_addr = 1073561756;

const unsigned long  elf_esp32_entry = 1074521000;

Different ESP Versions

  • ESP32-C3 and later are identified using GET_SECURITY_INFO command
  • Earlier versions have a register with a magic value
  • Try security info first and reset on failure
  • Different registers containing MAC address

Boot Mode

  • GPIO0 low at start
  • Wire pins to RST and GPIO0
  • Pull both low
  • Then pull RST high first and GPIO0 after
  • Supported when using Raspberry Pi or ESP as host

Summary

  • Connect to serial and set to boot mode
  • Use serial protocol to identify the chip
  • Upload modified flash loader
  • Extract

Extracting File Systems

  • SPIFFS/Littlefs
  • Gen 1 devices have known layout
  • On newer devices file system is found using metadata
  • Files can be extracted using specialized tools

Shelly Plus 1 File System

api_math.js
ca.pem                // signing CA
conf0.json            // empty configuration
conf3.json            // initial configuration
conf9.json            // current configuration
index.html.gz         // web frontend
init.js
rpc_acl_auth.json     // list of rpc endpoints
rpc_acl_no_auth.json
shelly_cloud.pem      // Shelly Cloud certificate
shelly_plugin_api.js
storage.json          // user data
tzinfo
updater.dat

conf9.json

{
  "wifi": {
    "sta": {
      "enable": true,
      "ssid": "SSID",
      "pass": "PASSWORD"
      "last_bssid": "aa:17:5d:15:ae:03",
    },
  },
  "shelly": {
    "cloud": {
      "enable": true,
      "server": "192.168.112.231:6022/jrpc",
      "token": "eyJhbGciOiJIUzI1NiIsInR..."
    },
  },
  ...
}

shelly_cloud.pem

Subject: O = Allterco
Not Before: Aug  4 12:03:41 2020 GMT
Not After : Aug  2 12:03:41 2030 GMT
-----BEGIN CERTIFICATE-----
MIICrTCCAZUCFCuIEEAQJOFLZuEtr/CWkvxi9YPAMA0GCSqGSIb3DQEBCwUAMBMx
ETAPBgNVBAoMCEFsbHRlcmNvMB4XDTIwMDgwNDEyMDM0MVoXDTMwMDgwMjEyMDM0
MVowEzERMA8GA1UECgwIQWxsdGVyY28wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQDHGDBDHpPUbtC9QAAjX3bi487AVY5JgYB2gyp6R9cjdsNGMbYnWdxn
BmsIUKJPg7B5NcQObiMe6djUvwo0c2Xl9L+P9LOskP2WNDdpquX3XJu580hXHHVB
mwOgJ0fi+5U9mOFHhc1gYGLmhO9oqsE80SgpmsPQHloMIqmcaolLzgC9PWGu8nSD
ToJq+dXyNFHzLVyBEugHQpeIR8Fq0do4dtlsfTWvv9U+fpGPegjdkPenSxGrOVwd
syFzNahxQGKmpZE/1fsq5QSh9+ZgwpdDChVNpkj9TBC1ApDTUasNco/6Meb/0Xur
pxpWPNfkIpZ7ebtGHVd/ZkGTPUnL7FXHAgMBAAEwDQYJKoZIhvcNAQELBQADggEB
ADwbvD7Mf7SOinV8JkOue8D/tvp+OiYTYLHYppzCLcBK3D1kQ7aqla2T8ebEFbLh
hpau7MxJcizVWZs4vJvFYxEwBTQldobhmG5lHnoKQcOSxis1kWr5xdKhji/QYo6T
wS/cird9hAcuc+RmLCMdpEDyia/vX+vvvRdyKsmB7A6Vkdu8s2B2jlhQNkitYzvq
UDKogJrWe6fQUTpTThMyGbqhp9cQ64M4DJG1cwSBZ/hiUAMKO/y5WVNWFbXIb/Om
xwkXgof2RXN1AjjqMcBh3GNVK4ZV5XM9WCocZjOCi2yZxaxaRWyuGR7EqAQZ+wdr
P5XrzYspeVl1WtBzwGFssPc=
-----END CERTIFICATE-----

Thank You