---
-title: Ourosboros Flash Reader
+title: Ouroboros Flash Reader
author: Patrick Schönberger
date: 16.07.2025
transition: none
revealjs-url: https://cdn.jsdelivr.net/npm/reveal.js@5.2.1
slideNumber: true
+navigationMode: linear
---
## Structure
- Start with the problem
- Work our way to the solution step by step
+::: notes
+
+This is my note.
+
+- It can contain Markdown
+- like this listt
+
+:::
+
<!-- access smart home hw on crime scenes -->
## The Problem
-- How to access data on Shelly devices?
-- Most Shelly devices are based on ESPs
-- They don't contain usable data like logs
-- They do contain user-identifiable data
+How to access data on Shelly devices?
##
- Extract the flash memory and read the data
- Do it read-only and document the process
+
+##
+
+- Most Shelly devices are based on ESP micro controllers
- Existing solutions (esptool, mos) can modify memory
- They are very complex, and thus hard to modify
- What about a custom tool?
- We cannot directly read flash, but we can write RAM
- Write a program for the ESP, load it into RAM and run
-- This is called the flash loader or stub loader
+- This is called flash loader or flasher stub
- Esptool uses two versions:
- Legacy C version
- New Rust version
##
-- The C version is getting replaced but it is much simpler and it supports ESP8266
-- We customize this version by removing write and erase commands
-- This way we technically have write access until the flash loader is activated
+- C version is deprecated but it is much simpler and supports ESP8266
+- Customize this version by removing write and erase commands
+- Technically there is write access until the flash loader is activated
- The extraction tool is small and contains no write or erase commands
+##
+
+```txt
+$ cloc esp-flasher-stub/
+-----------------------------------------------------------
+Language files blank comment code
+-----------------------------------------------------------
+Rust 12 327 78 1863
+Logos 14 32 0 249
+YAML 3 34 12 214
+Markdown 1 34 0 89
+TOML 4 8 2 61
+-----------------------------------------------------------
+SUM: 34 435 92 2476
+-----------------------------------------------------------
+```
+
+##
+
+```txt
+$ cloc esp-hal
+-----------------------------------------------------------
+Language files blank comment code
+-----------------------------------------------------------
+Rust 492 18739 26120 115809
+Linker Script 51 499 1404 11315
+Markdown 66 2014 12 5262
+TOML 43 505 412 4481
+Logos 44 229 15 1105
+YAML 13 151 67 1054
+Jinja Template 3 52 0 255
+JSON 2 0 0 48
+CSV 3 0 0 21
+SVG 1 0 0 4
+-----------------------------------------------------------
+SUM: 718 22189 28030 139354
+-----------------------------------------------------------
+```
+
+##
+
+```txt
+$ cloc esptool-legacy-flasher-stub/
+-----------------------------------------------------------
+Language files blank comment code
+-----------------------------------------------------------
+Linker Script 32 787 1188 18751
+C 6 1284 636 7689
+C/C++ Header 8 463 557 1687
+make 1 36 33 130
+YAML 4 19 0 114
+Python 2 25 21 80
+Markdown 3 58 0 76
+Bourne Shell 2 9 8 23
+TOML 1 1 0 20
+Jinja Template 1 4 2 14
+-----------------------------------------------------------
+SUM: 60 2686 2445 28584
+-----------------------------------------------------------
+```
+
<!-- how does the serial protocol work? -->
## Serial Protocol
-- SLIP frames...
+- Bytes are grouped using SLIP frames
+- Host sends requests containing commands
+- Target sends a matching reply
+
+::: notes
+Exceptions:
+
+- after loading flash loader it sends a slip frame on its own
+- after sync request 8 replies are sent
+- when reading flash using 0xd2 replies are not slip/request frames
+:::
+
+## Steps to Read Flash
+
+- Sync
+- Identify Chip
+- (Read MAC)
+- (Change baud rate)
+- Upload Stub
+- Read Flash
+
+::: notes
+Starting from boot mode
+sync always at 115200 but esp detects other baud rates from known payload
+:::
<!-- - stub_write_flash.c // write flash -->
<!-- - so we remove stub_write_flash.c and modify stub_commands.c -->
<!-- - additionally simplify the makefile -->
+
+## Modifying the Flash Loader
+
+- Consists mostly of six C files:
+ ```txt
+ miniz.c // compression
+ slip.c // slip
+ stub_commands.c // handle commands (modify)
+ stub_flasher.c // main program
+ stub_io.c // serial communication
+ stub_write_flash.c // write flash (remove)
+ ```
+
+
+
<!-- compiling and uploading the flash loader -->
<!-- - download toolchains -->
<!-- - compile the stub using specific toolchains -->
<!-- - this header gets compiled with the extraction tool (host) -->
<!-- - at runtime, after the chip is identified, upload .text and .data using MEM_ ram commands -->
<!-- - addresses for the sections and for the entry point are in elf file and get written to header alongside the elf sections -->
+
+## Uploading the Flash Loader
+
+- Compile using specific toolchains
+- Extract text and data sections from resulting ELF file
+- Generate header file that is compiled with the extraction tool
+- This includes section's sizes, addresses and the entry point
+- Upload the corresponding loader after identifying the chip
+
+
+
<!-- and how do we make the esp enter bootmode? -->
<!-- - wire two gpio pins to RST and GPIO0 -->
<!-- - pull both low -->
<!-- - GPIO0 has to be low when it is turned back on -->
<!-- - pull RST high to turn it on -->
<!-- - pull GPIO0 high after the esp has started -->
+
+## Boot Mode
+
+- GPIO0 low at start
+- Wire pins to RST and GPIO0
+- Pull both low
+- Then pull RST high first and GPIO0 after
+- Supported when using Raspberry Pi or ESP as host
+
+::: notes
+For development also possible via USB on linux hosts
+:::
+
+
+
<!-- overview: -->
<!-- - bootmode/serial -->
<!-- - serial protocol -->
<!-- - flash loader -->
+
+## Summary
+
+- Connect to serial and set to boot mode
+- Use serial protocol to identify the chip
+- Upload modified flash loader
+- Extract
+
+::: notes
+what is left:
+
+- differences between esp versions
+- how to extract files
+- what is on the files?
+:::
+
+
+
<!-- differences between esp versions -->
<!-- - identification: -->
<!-- - ESP32-C3 and later use GET_SECURITY_INFO which contains a chip_id -->
<!-- - esp8266 mac has to be calculated -->
<!-- - different flash loader versions -->
<!-- - esp8266 has no data section -->
-<!-- different hosts: -->
-<!-- - linux (usb) -->
-<!-- - rpi (gpio) -->
-<!-- - esp (gpio) -->
+
+## Different ESP Versions
+
+- ESP32-C3 and later are identified using `GET_SECURITY_INFO` command
+- Earlier versions have a register with a magic value
+- Try security info first and reset on failure
+- Different registers containing MAC address
+
+::: notes
+- sec info first bc it fails with unknown command
+- read register is a known command
+- reset -> bootmode -> sync again
+- on ESP8266 mac has to be calculated
+- esp8266 has no data section when compiling flash loader
+:::
+
+
+
<!-- extracting the file system -->
<!-- - esp8266 -->
<!-- - esp32 -->
-<!-- interesting files -->
-<!-- - wifi credentials -->
-<!-- - certificates -->
-<!-- - jwt token -->
+## Extracting File Systems
-## Demo
+- SPIFFS/Littlefs
+- Gen 1 devices have known layout
+- On newer devices file system is found using metadata
+- Files can be extracted using specialized tools
-## cloc
-##
+<!-- interesting files -->
+<!-- - wifi credentials -->
+<!-- - certificates -->
+<!-- - jwt token -->
-```sh
-$ cloc esp-flasher-stub/
------------------------------------------------------------
-Language files blank comment code
------------------------------------------------------------
-Rust 12 327 78 1863
-Logos 14 32 0 249
-YAML 3 34 12 214
-Markdown 1 34 0 89
-TOML 4 8 2 61
------------------------------------------------------------
-SUM: 34 435 92 2476
------------------------------------------------------------
+#### Shelly Plus 1 File System
+
+```txt
+api_math.js
+ca.pem // signing CA
+conf0.json // empty configuration
+conf3.json // initial configuration
+conf9.json // current configuration
+index.html.gz // web frontend
+init.js
+rpc_acl_auth.json // list of rpc endpoints
+rpc_acl_no_auth.json
+shelly_cloud.pem // Shelly Cloud certificate
+shelly_plugin_api.js
+storage.json // user data
+tzinfo
+updater.dat
```
-##
-
-```sh
-$ cloc esp-hal
------------------------------------------------------------
-Language files blank comment code
------------------------------------------------------------
-Rust 492 18739 26120 115809
-Linker Script 51 499 1404 11315
-Markdown 66 2014 12 5262
-TOML 43 505 412 4481
-Logos 44 229 15 1105
-YAML 13 151 67 1054
-Jinja Template 3 52 0 255
-JSON 2 0 0 48
-CSV 3 0 0 21
-SVG 1 0 0 4
------------------------------------------------------------
-SUM: 718 22189 28030 139354
------------------------------------------------------------
+#### `conf9.json`
+
+```txt
+{
+ "wifi": {
+ "sta": {
+ "enable": true,
+ "ssid": "SSID",
+ "pass": "PASSWORD"
+ "last_bssid": "aa:17:5d:15:ae:03",
+ },
+ },
+ "shelly": {
+ "cloud": {
+ "enable": true,
+ "server": "192.168.112.231:6022/jrpc",
+ "token": "eyJhbGciOiJIUzI1NiIsInR..."
+ },
+ },
+ ...
+}
```
-##
-
-```sh
-$ cloc esptool-legacy-flasher-stub/
------------------------------------------------------------
-Language files blank comment code
------------------------------------------------------------
-Linker Script 32 787 1188 18751
-C 6 1284 636 7689
-C/C++ Header 8 463 557 1687
-make 1 36 33 130
-YAML 4 19 0 114
-Python 2 25 21 80
-Markdown 3 58 0 76
-Bourne Shell 2 9 8 23
-TOML 1 1 0 20
-Jinja Template 1 4 2 14
------------------------------------------------------------
-SUM: 60 2686 2445 28584
------------------------------------------------------------
+#### `shelly_cloud.pem`
+
+```txt
+Subject: O = Allterco
+Not Before: Aug 4 12:03:41 2020 GMT
+Not After : Aug 2 12:03:41 2030 GMT
+-----BEGIN CERTIFICATE-----
+MIICrTCCAZUCFCuIEEAQJOFLZuEtr/CWkvxi9YPAMA0GCSqGSIb3DQEBCwUAMBMx
+ETAPBgNVBAoMCEFsbHRlcmNvMB4XDTIwMDgwNDEyMDM0MVoXDTMwMDgwMjEyMDM0
+MVowEzERMA8GA1UECgwIQWxsdGVyY28wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
+ggEKAoIBAQDHGDBDHpPUbtC9QAAjX3bi487AVY5JgYB2gyp6R9cjdsNGMbYnWdxn
+BmsIUKJPg7B5NcQObiMe6djUvwo0c2Xl9L+P9LOskP2WNDdpquX3XJu580hXHHVB
+mwOgJ0fi+5U9mOFHhc1gYGLmhO9oqsE80SgpmsPQHloMIqmcaolLzgC9PWGu8nSD
+ToJq+dXyNFHzLVyBEugHQpeIR8Fq0do4dtlsfTWvv9U+fpGPegjdkPenSxGrOVwd
+syFzNahxQGKmpZE/1fsq5QSh9+ZgwpdDChVNpkj9TBC1ApDTUasNco/6Meb/0Xur
+pxpWPNfkIpZ7ebtGHVd/ZkGTPUnL7FXHAgMBAAEwDQYJKoZIhvcNAQELBQADggEB
+ADwbvD7Mf7SOinV8JkOue8D/tvp+OiYTYLHYppzCLcBK3D1kQ7aqla2T8ebEFbLh
+hpau7MxJcizVWZs4vJvFYxEwBTQldobhmG5lHnoKQcOSxis1kWr5xdKhji/QYo6T
+wS/cird9hAcuc+RmLCMdpEDyia/vX+vvvRdyKsmB7A6Vkdu8s2B2jlhQNkitYzvq
+UDKogJrWe6fQUTpTThMyGbqhp9cQ64M4DJG1cwSBZ/hiUAMKO/y5WVNWFbXIb/Om
+xwkXgof2RXN1AjjqMcBh3GNVK4ZV5XM9WCocZjOCi2yZxaxaRWyuGR7EqAQZ+wdr
+P5XrzYspeVl1WtBzwGFssPc=
+-----END CERTIFICATE-----
```
+
+## Thank You
projects / ouroboros-slides / blobdiff