From 0ab17eeaa85251be7bac87403c48b954eb01cef6 Mon Sep 17 00:00:00 2001 From: patrick-scho Date: Fri, 25 Jul 2025 13:01:30 +0200 Subject: [PATCH] add generated html to repo --- pres.html | 1147 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 1147 insertions(+) create mode 100644 pres.html diff --git a/pres.html b/pres.html new file mode 100644 index 0000000..97d861c --- /dev/null +++ b/pres.html @@ -0,0 +1,1147 @@ + + + + + + + Ouroboros Flash Reader + + + + + + + + + +
+
+ +
+

Ouroboros Flash Reader

+

Patrick Schönberger

+

16.07.2025

+
+ +
+

Introduction

+
    +
  • Part of Project FIENDISH
    • +
  • Automatically extracts files from ESP-based Shelly smart home +devices
    • +
  • Can run on a Raspberry Pi or a micro controller
    • +
+ +
+ +
+

+

+ +

+
+ +
+

The Problem

+

How to access data on Shelly devices?

+
+ +
+

+
    +
  • Extract the flash memory and read the data
    • +
  • Do it read-only and document the process
    • +
+ +
+ +
+

+
    +
  • Most Shelly devices are based on ESP micro controllers
    • +
  • Existing solutions (esptool, mos) can modify memory and are not +fully automated
    • +
  • They are very complex, and thus hard to modify
    • +
  • How about a custom tool?
    • +
+ +
+ +
+

Talking to the ESP

+
    +
  • What is the bare minimum needed to talk to an ESP?
    • +
  • Two modes: boot and run
    • +
  • Serial pins are exposed on some Shelly devices and accessible on +others
    • +
  • In boot mode they can be used to communicate using a custom +protocol
    • +
+ +
+ +
+

+

+ +

+ +
+ +
+

+

+ +

+
+ +
+

+

+ +

+
+ +
+

+
    +
  • What can we do with the protocol? +
      +
    • Synchronize
      • +
    • Write RAM, flash and registers
      • +
    • Configure memory etc.
      • +
    • On the ESP32 it can read flash, but not on the ESP8266, ESP32C3 or +ESP32C6
      • +
    • +
+ +
+ +
+

+

+ +

+

https://docs.espressif.com/projects/esptool/en/latest/esp32/advanced-topics/serial-protocol.html

+ +
+ +
+

Reading Flash Memory

+
    +
  • We cannot directly read flash, but we can write RAM
    • +
  • Write a program, load it into RAM and run it on the ESP
    • +
  • This program is called flash loader or flasher stub
    • +
  • Esptool uses two versions: +
      +
    • Legacy C version
      • +
    • New Rust version
      • +
    • +
+ +
+ +
+

+
$ cloc esp-flasher-stub/
+-----------------------------------------------------------
+Language          files       blank     comment        code
+-----------------------------------------------------------
+Rust                 12         327          78        1863
+Logos                14          32           0         249
+YAML                  3          34          12         214
+Markdown              1          34           0          89
+TOML                  4           8           2          61
+-----------------------------------------------------------
+SUM:                 34         435          92        2476
+-----------------------------------------------------------
+
+ +
+

+
$ cloc esp-hal
+-----------------------------------------------------------
+Language          files       blank     comment        code
+-----------------------------------------------------------
+Rust                492       18739       26120      115809
+Linker Script        51         499        1404       11315
+Markdown             66        2014          12        5262
+TOML                 43         505         412        4481
+Logos                44         229          15        1105
+YAML                 13         151          67        1054
+Jinja Template        3          52           0         255
+JSON                  2           0           0          48
+CSV                   3           0           0          21
+SVG                   1           0           0           4
+-----------------------------------------------------------
+SUM:                718       22189       28030      139354
+-----------------------------------------------------------
+
+ +
+

+
$ cloc esptool-legacy-flasher-stub/
+-----------------------------------------------------------
+Language          files       blank     comment        code
+-----------------------------------------------------------
+Linker Script        32         787        1188       18751
+C                     6        1284         636        7689
+C/C++ Header          8         463         557        1687
+make                  1          36          33         130
+YAML                  4          19           0         114
+Python                2          25          21          80
+Markdown              3          58           0          76
+Bourne Shell          2           9           8          23
+TOML                  1           1           0          20
+Jinja Template        1           4           2          14
+-----------------------------------------------------------
+SUM:                 60        2686        2445       28584
+-----------------------------------------------------------
+ +
+ +
+

+
    +
  • C version is deprecated but it is much simpler and supports +ESP8266
    • +
  • Customize this version by removing write and erase commands
    • +
  • Technically there is write access until the flash loader is +activated
    • +
  • The extraction tool is small and contains no write or erase +commands
    • +
+ +
+ +
+

Serial Protocol

+
    +
  • Bytes are grouped using SLIP frames
    • +
  • Host sends requests containing commands
    • +
  • Target sends a matching reply
    • +
+ +
+ +
+

+

+
+ +
+

+

+

https://docs.espressif.com/projects/esptool/en/latest/esp32/advanced-topics/serial-protocol.html

+
+ +
+

Steps to Read Flash

+
    +
  • Sync
    • +
  • Identify Chip
    • +
  • (Read MAC)
    • +
  • (Change baud rate)
    • +
  • Upload Stub
    • +
  • Read Flash
    • +
+ +
+ +
+

Modifying the Flash Loader

+
    +

  • Consists mostly of six C files:

    +
    miniz.c             // compression
+slip.c              // slip
+stub_commands.c     // handle commands (modify)
+stub_flasher.c      // main program
+stub_io.c           // serial communication
+stub_write_flash.c  // write flash (remove)
    • +
+ +
+ +
+

Uploading the Flash Loader

+
    +
  • Compile using specific toolchains
    • +
  • Extract text and data sections from resulting ELF file
    • +
  • Generate header file that is compiled with the extraction tool
    • +
  • This includes section’s sizes, addresses and the entry point
    • +
  • Upload the corresponding loader after identifying the chip
    • +
+ +
+ +
+

+
const unsigned char *elf_esp32_text_buffer =
+                    (unsigned char[]){0x08,0x00,0xf4,...};
+const unsigned long  elf_esp32_text_size = 2100;
+const unsigned long  elf_esp32_text_addr = 1074520064;
+
+const unsigned char *elf_esp32_data_buffer =
+                    (unsigned char[]){0x9b,0xe6,0x0b,...};
+const unsigned long  elf_esp32_data_size = 60;
+const unsigned long  elf_esp32_data_addr = 1073561756;
+
+const unsigned long  elf_esp32_entry = 1074521000;
+
+ +
+

Different ESP Versions

+
    +
  • ESP32-C3 and later are identified using +GET_SECURITY_INFO command
    • +
  • Earlier versions have a register with a magic value
    • +
  • Try security info first and reset on failure
    • +
  • Different registers containing MAC address
    • +
+ +
+ +
+

Boot Mode

+
    +
  • GPIO0 low at start
    • +
  • Wire pins to RST and GPIO0
    • +
  • Pull both low
    • +
  • Then pull RST high first and GPIO0 after
    • +
  • Supported when using Raspberry Pi or ESP as host
    • +
+ +
+ +
+

Summary

+
    +
  • Connect to serial and set to boot mode
    • +
  • Use serial protocol to identify the chip
    • +
  • Upload modified flash loader
    • +
  • Extract
    • +
+ +
+ +
+
+

Extracting File Systems

+
    +
  • SPIFFS/Littlefs
    • +
  • Gen 1 devices have known layout
    • +
  • On newer devices file system is found using metadata
    • +
  • Files can be extracted using specialized tools
    • +
+ +
+
+

Shelly Plus 1 File System

+
api_math.js
+ca.pem                // signing CA
+conf0.json            // empty configuration
+conf3.json            // initial configuration
+conf9.json            // current configuration
+index.html.gz         // web frontend
+init.js
+rpc_acl_auth.json     // list of rpc endpoints
+rpc_acl_no_auth.json
+shelly_cloud.pem      // Shelly Cloud certificate
+shelly_plugin_api.js
+storage.json          // user data
+tzinfo
+updater.dat
+
+
+

conf9.json

+
{
+  "wifi": {
+    "sta": {
+      "enable": true,
+      "ssid": "SSID",
+      "pass": "PASSWORD"
+      "last_bssid": "aa:17:5d:15:ae:03",
+    },
+  },
+  "shelly": {
+    "cloud": {
+      "enable": true,
+      "server": "192.168.112.231:6022/jrpc",
+      "token": "eyJhbGciOiJIUzI1NiIsInR..."
+    },
+  },
+  ...
+}
+ +
+
+

shelly_cloud.pem

+
Subject: O = Allterco
+Not Before: Aug  4 12:03:41 2020 GMT
+Not After : Aug  2 12:03:41 2030 GMT
+-----BEGIN CERTIFICATE-----
+MIICrTCCAZUCFCuIEEAQJOFLZuEtr/CWkvxi9YPAMA0GCSqGSIb3DQEBCwUAMBMx
+ETAPBgNVBAoMCEFsbHRlcmNvMB4XDTIwMDgwNDEyMDM0MVoXDTMwMDgwMjEyMDM0
+MVowEzERMA8GA1UECgwIQWxsdGVyY28wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
+ggEKAoIBAQDHGDBDHpPUbtC9QAAjX3bi487AVY5JgYB2gyp6R9cjdsNGMbYnWdxn
+BmsIUKJPg7B5NcQObiMe6djUvwo0c2Xl9L+P9LOskP2WNDdpquX3XJu580hXHHVB
+mwOgJ0fi+5U9mOFHhc1gYGLmhO9oqsE80SgpmsPQHloMIqmcaolLzgC9PWGu8nSD
+ToJq+dXyNFHzLVyBEugHQpeIR8Fq0do4dtlsfTWvv9U+fpGPegjdkPenSxGrOVwd
+syFzNahxQGKmpZE/1fsq5QSh9+ZgwpdDChVNpkj9TBC1ApDTUasNco/6Meb/0Xur
+pxpWPNfkIpZ7ebtGHVd/ZkGTPUnL7FXHAgMBAAEwDQYJKoZIhvcNAQELBQADggEB
+ADwbvD7Mf7SOinV8JkOue8D/tvp+OiYTYLHYppzCLcBK3D1kQ7aqla2T8ebEFbLh
+hpau7MxJcizVWZs4vJvFYxEwBTQldobhmG5lHnoKQcOSxis1kWr5xdKhji/QYo6T
+wS/cird9hAcuc+RmLCMdpEDyia/vX+vvvRdyKsmB7A6Vkdu8s2B2jlhQNkitYzvq
+UDKogJrWe6fQUTpTThMyGbqhp9cQ64M4DJG1cwSBZ/hiUAMKO/y5WVNWFbXIb/Om
+xwkXgof2RXN1AjjqMcBh3GNVK4ZV5XM9WCocZjOCi2yZxaxaRWyuGR7EqAQZ+wdr
+P5XrzYspeVl1WtBzwGFssPc=
+-----END CERTIFICATE-----
+ +
+
+

Thank You

+ +
+
+
+ + + + + + + + + + + -- 2.50.1