From 758f808544b19cfeee8fba9da045d06bd3357ea7 Mon Sep 17 00:00:00 2001 From: patrick-scho Date: Sun, 13 Jul 2025 13:10:03 +0200 Subject: [PATCH] replace adoc with md, add Makefile --- Makefile | 7 ++ pres.adoc | 185 ----------------------------------------- pres.md | 241 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 248 insertions(+), 185 deletions(-) create mode 100644 Makefile delete mode 100644 pres.adoc create mode 100644 pres.md diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..9157f44 --- /dev/null +++ b/Makefile @@ -0,0 +1,7 @@ +all: + pandoc -t revealjs --standalone pres.md -o pres_md.html + +embed: + pandoc -t revealjs --standalone --embed-resources pres.md -o pres_md.html + +.PHONS: all embed diff --git a/pres.adoc b/pres.adoc deleted file mode 100644 index 14a6a0c..0000000 --- a/pres.adoc +++ /dev/null @@ -1,185 +0,0 @@ -= Ourosboros Flash Reader -:revealjsdir: ./reveal.js-5.2.1/ - -Patrick Schönberger - -16.07.2025 - -== Structure - -- Problem -- Solution -- Implementation - -== Problem - -== Solution - -== Implementation - -structure: start with a problem and reproduce the work leading to the working solution - -access smart home hw on crime scenes -many shelly devices are based on esp32/esp8266 -they dont contain any usable data but contain identifiable user data -this can be used to inquire about the user account -so we need to extract the content of the esp's flash memory -we then also need to extract filesystems from the memory -do it read-only, verifiably -existing solutions (esptool, mos) can also write and erase memory -they are also complex, making understanding and changing the code time consuming -what about writing a custom extraction tool? -what is the bare minimum needed to talk to the esp? -- two modes: bootmode and runmode -- decided by GPIO0 at start -- the esp as well as the shelly devices expose uart pins (tx/rx) -- in runmode they output logging information -- in bootmode they listen to a custom serial protocol -so we need a serial connection and the ability to enter boot mode! -what can the serial protocol do? -- sync -- write ram/flash/registers -- configuration etc. -- on the esp32 it can read flash, but not on esp8266, esp32c3, esp32c6 -how do other tools read flash? -- we cant directly read flash, but we can write ram -- write a program, load it into ram, run it and then talk to it instead -- flash loader/stub -- esptool uses two different variants, c based and rust based -- the c based one is older and getting replaced, but it is also dramatically simpler and also supports the esp8266 -- so we use the c based one and customize it (remove write and erase flash commands) -technically this means we do have write access until the flash loader is activated -the extraction tool is also small and runs a fixed number of commands --> as sure as we can be -how does the serial protocol work? -- data is encoded using SLIP frames -- the host sends a request and the target (esp) sends a response -- steps to read flash: - - sync - - identify chip - - read mac - - (change baud) - - upload stub - - read flash -modifying the flash loader -- stub consists of 6 .c files: - - miniz.c // compression - - slip.c // slip - - stub_commands.c // handle commands - - stub_flasher.c // main program - - stub_io.c // serial communication - - stub_write_flash.c // write flash -- so we remove stub_write_flash.c and modify stub_commands.c -- additionally simplify the makefile -compiling and uploading the flash loader -- download toolchains -- compile the stub using specific toolchains -- this gives us an elf file -- use a python script to extract the .text and .data sections from the elf -- generate a header file and write the raw bytes to a `const unsigned char[]` -- this header gets compiled with the extraction tool (host) -- at runtime, after the chip is identified, upload .text and .data using MEM_ ram commands -- addresses for the sections and for the entry point are in elf file and get written to header alongside the elf sections -and how do we make the esp enter bootmode? -- wire two gpio pins to RST and GPIO0 -- pull both low - - RST low turns the esp off - - GPIO0 has to be low when it is turned back on -- pull RST high to turn it on -- pull GPIO0 high after the esp has started -overview: -- bootmode/serial -- serial protocol -- flash loader -differences between esp versions -- identification: - - ESP32-C3 and later use GET_SECURITY_INFO which contains a chip_id - - previous models have a register with a magic value identifying the chip -- mac address: - - different registers - - esp8266 mac has to be calculated -- different flash loader versions - - esp8266 has no data section -different hosts: -- linux (usb) -- rpi (gpio) -- esp (gpio) -extracting the file system -- esp8266 -- esp32 -interesting files -- wifi credentials -- certificates -- jwt token - - -DEMO! - - -== cloc - -```sh -$ cloc esp-flasher-stub/ - 38 text files. - 34 unique files. - 5 files ignored. - -github.com/AlDanial/cloc v 2.04 T=0.02 s (2259.9 files/s, 199599.0 lines/s) ------------------------------------------------------------ -Language files blank comment code ------------------------------------------------------------ -Rust 12 327 78 1863 -Logos 14 32 0 249 -YAML 3 34 12 214 -Markdown 1 34 0 89 -TOML 4 8 2 61 ------------------------------------------------------------ -SUM: 34 435 92 2476 ------------------------------------------------------------ - -$ cloc esp-hal - 742 text files. - 718 unique files. - 35 files ignored. - -github.com/AlDanial/cloc v 2.04 T=0.36 s (1978.1 files/s, 522278.6 lines/s) ------------------------------------------------------------ -Language files blank comment code ------------------------------------------------------------ -Rust 492 18739 26120 115809 -Linker Script 51 499 1404 11315 -Markdown 66 2014 12 5262 -TOML 43 505 412 4481 -Logos 44 229 15 1105 -YAML 13 151 67 1054 -Jinja Template 3 52 0 255 -JSON 2 0 0 48 -CSV 3 0 0 21 -SVG 1 0 0 4 ------------------------------------------------------------ -SUM: 718 22189 28030 139354 ------------------------------------------------------------ - -$ cloc esptool-legacy-flasher-stub/ - 63 text files. - 60 unique files. - 4 files ignored. - -github.com/AlDanial/cloc v 2.04 T=0.06 s (952.9 files/s, 535446.8 lines/s) ------------------------------------------------------------ -Language files blank comment code ------------------------------------------------------------ -Linker Script 32 787 1188 18751 -C 6 1284 636 7689 -C/C++ Header 8 463 557 1687 -make 1 36 33 130 -YAML 4 19 0 114 -Python 2 25 21 80 -Markdown 3 58 0 76 -Bourne Shell 2 9 8 23 -TOML 1 1 0 20 -Jinja Template 1 4 2 14 ------------------------------------------------------------ -SUM: 60 2686 2445 28584 ------------------------------------------------------------ -``` diff --git a/pres.md b/pres.md new file mode 100644 index 0000000..20c971e --- /dev/null +++ b/pres.md @@ -0,0 +1,241 @@ +--- +title: Ourosboros Flash Reader +author: Patrick Schönberger +date: 16.07.2025 +transition: none +revealjs-url: https://cdn.jsdelivr.net/npm/reveal.js@5.2.1 +slideNumber: true +--- + +## Structure + + + +- Start with the problem +- Work our way to the solution step by step + + + + + + + + + + + + + + +## The Problem + +- How to access data on Shelly devices? +- Most Shelly devices are based on ESPs +- They don't contain usable data like logs +- They do contain user-identifiable data + +## + +- Extract the flash memory and read the data +- Do it read-only and document the process +- Existing solutions (esptool, mos) can modify memory +- They are very complex, and thus hard to modify +- What about a custom tool? + + + + + + + + + + + + + + + + +## Talking to the ESP + +- What is the bare minimum needed to talk to an ESP? +- Two modes: boot and run +- Serial pins are exposed on some Shelly devices and accessible on others +- In boot mode they can be used to communicate using a custom protocol + + +## + +- What can we do with the protocol? + - Synchronize + - Write RAM, flash and registers + - Configure memory etc. + - On the ESP32 it can read flash, but not on the ESP8266, ESP32C3 or ESP32C6 +- How do other tools read flash? + + + + + + + + + + + + + + +## Reading Flash Memory + +- We cannot directly read flash, but we can write RAM +- Write a program for the ESP, load it into RAM and run +- This is called the flash loader or stub loader +- Esptool uses two versions: + - Legacy C version + - New Rust version + +## + +- The C version is getting replaced but it is much simpler and it supports ESP8266 +- We customize this version by removing write and erase commands +- This way we technically have write access until the flash loader is activated +- The extraction tool is small and contains no write or erase commands + + + + + + + + + + + + + + +## Serial Protocol + +- SLIP frames... + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +## Demo + + +## cloc + +## + +```sh +$ cloc esp-flasher-stub/ +----------------------------------------------------------- +Language files blank comment code +----------------------------------------------------------- +Rust 12 327 78 1863 +Logos 14 32 0 249 +YAML 3 34 12 214 +Markdown 1 34 0 89 +TOML 4 8 2 61 +----------------------------------------------------------- +SUM: 34 435 92 2476 +----------------------------------------------------------- +``` + +## + +```sh +$ cloc esp-hal +----------------------------------------------------------- +Language files blank comment code +----------------------------------------------------------- +Rust 492 18739 26120 115809 +Linker Script 51 499 1404 11315 +Markdown 66 2014 12 5262 +TOML 43 505 412 4481 +Logos 44 229 15 1105 +YAML 13 151 67 1054 +Jinja Template 3 52 0 255 +JSON 2 0 0 48 +CSV 3 0 0 21 +SVG 1 0 0 4 +----------------------------------------------------------- +SUM: 718 22189 28030 139354 +----------------------------------------------------------- +``` + +## + +```sh +$ cloc esptool-legacy-flasher-stub/ +----------------------------------------------------------- +Language files blank comment code +----------------------------------------------------------- +Linker Script 32 787 1188 18751 +C 6 1284 636 7689 +C/C++ Header 8 463 557 1687 +make 1 36 33 130 +YAML 4 19 0 114 +Python 2 25 21 80 +Markdown 3 58 0 76 +Bourne Shell 2 9 8 23 +TOML 1 1 0 20 +Jinja Template 1 4 2 14 +----------------------------------------------------------- +SUM: 60 2686 2445 28584 +----------------------------------------------------------- +``` -- 2.50.1