X-Git-Url: https://gitweb.ps.run/ouroboros-slides/blobdiff_plain/f4ad10e4d30e611cefd06474173adb20263c191e..758f808544b19cfeee8fba9da045d06bd3357ea7:/pres.md

diff --git a/pres.md b/pres.md
new file mode 100644
index 0000000..20c971e
--- /dev/null
+++ b/pres.md
@@ -0,0 +1,241 @@
+---
+title: Ourosboros Flash Reader
+author: Patrick Schönberger
+date: 16.07.2025
+transition: none
+revealjs-url: https://cdn.jsdelivr.net/npm/reveal.js@5.2.1
+slideNumber: true
+---
+
+## Structure
+<!-- structure: start with a problem and reproduce the work leading to the working solution -->
+
+
+- Start with the problem
+- Work our way to the solution step by step
+
+
+
+<!-- access smart home hw on crime scenes -->
+<!-- many shelly devices are based on esp32/esp8266 -->
+<!-- they dont contain any usable data but contain identifiable user data -->
+<!-- this can be used to inquire about the user account -->
+<!-- so we need to extract the content of the esp's flash memory -->
+<!-- we then also need to extract filesystems from the memory -->
+<!-- do it read-only, verifiably -->
+<!-- existing solutions (esptool, mos) can also write and erase memory -->
+<!-- they are also complex, making understanding and changing the code time consuming -->
+<!-- what about writing a custom extraction tool? -->
+
+## The Problem
+
+- How to access data on Shelly devices?
+- Most Shelly devices are based on ESPs
+- They don't contain usable data like logs
+- They do contain user-identifiable data
+
+## 
+
+- Extract the flash memory and read the data
+- Do it read-only and document the process
+- Existing solutions (esptool, mos) can modify memory
+- They are very complex, and thus hard to modify
+- What about a custom tool?
+
+
+
+<!-- what is the bare minimum needed to talk to the esp? -->
+<!-- - two modes: bootmode and runmode -->
+<!-- - decided by GPIO0 at start -->
+<!-- - the esp as well as the shelly devices expose uart pins (tx/rx) -->
+<!-- - in runmode they output logging information -->
+<!-- - in bootmode they listen to a custom serial protocol -->
+<!-- so we need a serial connection and the ability to enter boot mode! -->
+<!-- what can the serial protocol do? -->
+<!-- - sync -->
+<!-- - write ram/flash/registers -->
+<!-- - configuration etc. -->
+<!-- - on the esp32 it can read flash, but not on esp8266, esp32c3, esp32c6 -->
+
+## Talking to the ESP
+
+- What is the bare minimum needed to talk to an ESP?
+- Two modes: boot and run
+- Serial pins are exposed on some Shelly devices and accessible on others
+- In boot mode they can be used to communicate using a custom protocol
+<!-- * So we need a serial connection and the ability to enter boot mode -->
+
+## 
+
+- What can we do with the protocol?
+  - Synchronize
+  - Write RAM, flash and registers
+  - Configure memory etc.
+  - On the ESP32 it can read flash, but not on the ESP8266, ESP32C3 or ESP32C6
+- How do other tools read flash?
+
+
+
+<!-- how do other tools read flash? -->
+<!-- - we cant directly read flash, but we can write ram -->
+<!-- - write a program, load it into ram, run it and then talk to it instead -->
+<!-- - flash loader/stub -->
+<!-- - esptool uses two different variants, c based and rust based -->
+<!-- - the c based one is older and getting replaced, but it is also dramatically simpler and also supports the esp8266 -->
+<!-- - so we use the c based one and customize it (remove write and erase flash commands) -->
+<!-- technically this means we do have write access until the flash loader is activated -->
+<!-- the extraction tool is also small and runs a fixed number of commands -->
+<!-- -> as sure as we can be -->
+
+## Reading Flash Memory
+
+- We cannot directly read flash, but we can write RAM
+- Write a program for the ESP, load it into RAM and run
+- This is called the flash loader or stub loader
+- Esptool uses two versions:
+  - Legacy C version
+  - New Rust version
+
+## 
+
+- The C version is getting replaced but it is much simpler and it supports ESP8266
+- We customize this version by removing write and erase commands
+- This way we technically have write access until the flash loader is activated
+- The extraction tool is small and contains no write or erase commands
+
+
+
+<!-- how does the serial protocol work? -->
+<!-- - data is encoded using SLIP frames -->
+<!-- - the host sends a request and the target (esp) sends a response -->
+<!-- - steps to read flash: -->
+<!--   - sync -->
+<!--   - identify chip -->
+<!--   - read mac -->
+<!--   - (change baud) -->
+<!--   - upload stub -->
+<!--   - read flash -->
+
+## Serial Protocol
+
+- SLIP frames...
+
+
+
+<!-- modifying the flash loader -->
+<!-- - stub consists of 6 .c files: -->
+<!--   - miniz.c         // compression -->
+<!--   - slip.c          // slip -->
+<!--   - stub_commands.c // handle commands -->
+<!--   - stub_flasher.c  // main program -->
+<!--   - stub_io.c       // serial communication -->
+<!--   - stub_write_flash.c // write flash -->
+<!-- - so we remove stub_write_flash.c and modify stub_commands.c -->
+<!-- - additionally simplify the makefile -->
+<!-- compiling and uploading the flash loader -->
+<!-- - download toolchains -->
+<!-- - compile the stub using specific toolchains -->
+<!-- - this gives us an elf file -->
+<!-- - use a python script to extract the .text and .data sections from the elf -->
+<!-- - generate a header file and write the raw bytes to a `const unsigned char[]` -->
+<!-- - this header gets compiled with the extraction tool (host) -->
+<!-- - at runtime, after the chip is identified, upload .text and .data using MEM_ ram commands -->
+<!-- - addresses for the sections and for the entry point are in elf file and get written to header alongside the elf sections -->
+<!-- and how do we make the esp enter bootmode? -->
+<!-- - wire two gpio pins to RST and GPIO0 -->
+<!-- - pull both low -->
+<!--   - RST low turns the esp off -->
+<!--   - GPIO0 has to be low when it is turned back on -->
+<!-- - pull RST high to turn it on -->
+<!-- - pull GPIO0 high after the esp has started -->
+<!-- overview: -->
+<!-- - bootmode/serial -->
+<!-- - serial protocol -->
+<!-- - flash loader -->
+<!-- differences between esp versions -->
+<!-- - identification: -->
+<!--   - ESP32-C3 and later use GET_SECURITY_INFO which contains a chip_id -->
+<!--   - previous models have a register with a magic value identifying the chip -->
+<!-- - mac address: -->
+<!--   - different registers -->
+<!--   - esp8266 mac has to be calculated -->
+<!-- - different flash loader versions -->
+<!--   - esp8266 has no data section -->
+<!-- different hosts: -->
+<!-- - linux (usb) -->
+<!-- - rpi (gpio) -->
+<!-- - esp (gpio) -->
+<!-- extracting the file system -->
+<!-- - esp8266 -->
+<!-- - esp32 -->
+<!-- interesting files -->
+<!-- - wifi credentials -->
+<!-- - certificates -->
+<!-- - jwt token -->
+
+
+## Demo
+
+
+## cloc
+
+## 
+
+```sh
+$ cloc esp-flasher-stub/
+-----------------------------------------------------------
+Language          files       blank     comment        code
+-----------------------------------------------------------
+Rust                 12         327          78        1863
+Logos                14          32           0         249
+YAML                  3          34          12         214
+Markdown              1          34           0          89
+TOML                  4           8           2          61
+-----------------------------------------------------------
+SUM:                 34         435          92        2476
+-----------------------------------------------------------
+```
+
+## 
+
+```sh
+$ cloc esp-hal
+-----------------------------------------------------------
+Language          files       blank     comment        code
+-----------------------------------------------------------
+Rust                492       18739       26120      115809
+Linker Script        51         499        1404       11315
+Markdown             66        2014          12        5262
+TOML                 43         505         412        4481
+Logos                44         229          15        1105
+YAML                 13         151          67        1054
+Jinja Template        3          52           0         255
+JSON                  2           0           0          48
+CSV                   3           0           0          21
+SVG                   1           0           0           4
+-----------------------------------------------------------
+SUM:                718       22189       28030      139354
+-----------------------------------------------------------
+```
+
+## 
+
+```sh
+$ cloc esptool-legacy-flasher-stub/
+-----------------------------------------------------------
+Language          files       blank     comment        code
+-----------------------------------------------------------
+Linker Script        32         787        1188       18751
+C                     6        1284         636        7689
+C/C++ Header          8         463         557        1687
+make                  1          36          33         130
+YAML                  4          19           0         114
+Python                2          25          21          80
+Markdown              3          58           0          76
+Bourne Shell          2           9           8          23
+TOML                  1           1           0          20
+Jinja Template        1           4           2          14
+-----------------------------------------------------------
+SUM:                 60        2686        2445       28584
+-----------------------------------------------------------
+```