Update presentation

diff --git a/pres.md b/pres.md
index 20c971e0bde164960a3fb7042dfb757556ac9875..d761b73b2c11570f8e7ecb0b17fcf30c4d3a5116 100644 (file)
--- a/pres.md
+++ b/pres.md
@@ -1,10 +1,11 @@
 ---
 ---

-title: Ourosboros Flash Reader
+title: Ouroboros Flash Reader

 author: Patrick Schönberger
 date: 16.07.2025
 transition: none
 revealjs-url: https://cdn.jsdelivr.net/npm/reveal.js@5.2.1
 slideNumber: true
 author: Patrick Schönberger
 date: 16.07.2025
 transition: none
 revealjs-url: https://cdn.jsdelivr.net/npm/reveal.js@5.2.1
 slideNumber: true

+navigationMode: linear

 ---
 
 ## Structure
 ---
 
 ## Structure


@@ -14,6 +15,15 @@ slideNumber: true
 - Start with the problem
 - Work our way to the solution step by step
 
 - Start with the problem
 - Work our way to the solution step by step
 

+::: notes
+
+This is my note.
+
+- It can contain Markdown
+- like this listt
+
+:::
+

 
 
 <!-- access smart home hw on crime scenes -->
 
 
 <!-- access smart home hw on crime scenes -->


@@ -29,15 +39,16 @@ slideNumber: true
 
 ## The Problem
 
 
 ## The Problem
 

-- How to access data on Shelly devices?
-- Most Shelly devices are based on ESPs
-- They don't contain usable data like logs
-- They do contain user-identifiable data
+How to access data on Shelly devices?

 
 ## 
 
 - Extract the flash memory and read the data
 - Do it read-only and document the process
 
 ## 
 
 - Extract the flash memory and read the data
 - Do it read-only and document the process

+
+## 
+
+- Most Shelly devices are based on ESP micro controllers

 - Existing solutions (esptool, mos) can modify memory
 - They are very complex, and thus hard to modify
 - What about a custom tool?
 - Existing solutions (esptool, mos) can modify memory
 - They are very complex, and thus hard to modify
 - What about a custom tool?


@@ -91,18 +102,79 @@ slideNumber: true
 
 - We cannot directly read flash, but we can write RAM
 - Write a program for the ESP, load it into RAM and run
 
 - We cannot directly read flash, but we can write RAM
 - Write a program for the ESP, load it into RAM and run

-- This is called the flash loader or stub loader
+- This is called flash loader or flasher stub

 - Esptool uses two versions:
   - Legacy C version
   - New Rust version
 
 ## 
 
 - Esptool uses two versions:
   - Legacy C version
   - New Rust version
 
 ## 
 

-- The C version is getting replaced but it is much simpler and it supports ESP8266
-- We customize this version by removing write and erase commands
-- This way we technically have write access until the flash loader is activated
+- C version is deprecated but it is much simpler and supports ESP8266
+- Customize this version by removing write and erase commands
+- Technically there is write access until the flash loader is activated

 - The extraction tool is small and contains no write or erase commands
 
 - The extraction tool is small and contains no write or erase commands
 

+## 
+
+```txt
+$ cloc esp-flasher-stub/
+-----------------------------------------------------------
+Language          files       blank     comment        code
+-----------------------------------------------------------
+Rust                 12         327          78        1863
+Logos                14          32           0         249
+YAML                  3          34          12         214
+Markdown              1          34           0          89
+TOML                  4           8           2          61
+-----------------------------------------------------------
+SUM:                 34         435          92        2476
+-----------------------------------------------------------
+```
+
+## 
+
+```txt
+$ cloc esp-hal
+-----------------------------------------------------------
+Language          files       blank     comment        code
+-----------------------------------------------------------
+Rust                492       18739       26120      115809
+Linker Script        51         499        1404       11315
+Markdown             66        2014          12        5262
+TOML                 43         505         412        4481
+Logos                44         229          15        1105
+YAML                 13         151          67        1054
+Jinja Template        3          52           0         255
+JSON                  2           0           0          48
+CSV                   3           0           0          21
+SVG                   1           0           0           4
+-----------------------------------------------------------
+SUM:                718       22189       28030      139354
+-----------------------------------------------------------
+```
+
+## 
+
+```txt
+$ cloc esptool-legacy-flasher-stub/
+-----------------------------------------------------------
+Language          files       blank     comment        code
+-----------------------------------------------------------
+Linker Script        32         787        1188       18751
+C                     6        1284         636        7689
+C/C++ Header          8         463         557        1687
+make                  1          36          33         130
+YAML                  4          19           0         114
+Python                2          25          21          80
+Markdown              3          58           0          76
+Bourne Shell          2           9           8          23
+TOML                  1           1           0          20
+Jinja Template        1           4           2          14
+-----------------------------------------------------------
+SUM:                 60        2686        2445       28584
+-----------------------------------------------------------
+```
+

 
 
 <!-- how does the serial protocol work? -->
 
 
 <!-- how does the serial protocol work? -->


@@ -118,7 +190,31 @@ slideNumber: true
 
 ## Serial Protocol
 
 
 ## Serial Protocol
 

-- SLIP frames...
+- Bytes are grouped using SLIP frames
+- Host sends requests containing commands
+- Target sends a matching reply
+
+::: notes
+Exceptions:
+
+- after loading flash loader it sends a slip frame on its own
+- after sync request 8 replies are sent
+- when reading flash using 0xd2 replies are not slip/request frames
+:::
+
+## Steps to Read Flash
+
+- Sync
+- Identify Chip
+- (Read MAC)
+- (Change baud rate)
+- Upload Stub
+- Read Flash
+
+::: notes
+Starting from boot mode
+sync always at 115200 but esp detects other baud rates from known payload
+:::

 
 
 
 
 
 


@@ -132,6 +228,21 @@ slideNumber: true
 <!--   - stub_write_flash.c // write flash -->
 <!-- - so we remove stub_write_flash.c and modify stub_commands.c -->
 <!-- - additionally simplify the makefile -->
 <!--   - stub_write_flash.c // write flash -->
 <!-- - so we remove stub_write_flash.c and modify stub_commands.c -->
 <!-- - additionally simplify the makefile -->

+
+## Modifying the Flash Loader
+
+- Consists mostly of six C files:
+  ```txt
+  miniz.c             // compression
+  slip.c              // slip
+  stub_commands.c     // handle commands (modify)
+  stub_flasher.c      // main program
+  stub_io.c           // serial communication
+  stub_write_flash.c  // write flash (remove)
+  ```
+
+
+

 <!-- compiling and uploading the flash loader -->
 <!-- - download toolchains -->
 <!-- - compile the stub using specific toolchains -->
 <!-- compiling and uploading the flash loader -->
 <!-- - download toolchains -->
 <!-- - compile the stub using specific toolchains -->


@@ -141,6 +252,17 @@ slideNumber: true
 <!-- - this header gets compiled with the extraction tool (host) -->
 <!-- - at runtime, after the chip is identified, upload .text and .data using MEM_ ram commands -->
 <!-- - addresses for the sections and for the entry point are in elf file and get written to header alongside the elf sections -->
 <!-- - this header gets compiled with the extraction tool (host) -->
 <!-- - at runtime, after the chip is identified, upload .text and .data using MEM_ ram commands -->
 <!-- - addresses for the sections and for the entry point are in elf file and get written to header alongside the elf sections -->

+
+## Uploading the Flash Loader
+
+- Compile using specific toolchains
+- Extract text and data sections from resulting ELF file
+- Generate header file that is compiled with the extraction tool
+- This includes section's sizes, addresses and the entry point
+- Upload the corresponding loader after identifying the chip
+
+
+

 <!-- and how do we make the esp enter bootmode? -->
 <!-- - wire two gpio pins to RST and GPIO0 -->
 <!-- - pull both low -->
 <!-- and how do we make the esp enter bootmode? -->
 <!-- - wire two gpio pins to RST and GPIO0 -->
 <!-- - pull both low -->


@@ -148,10 +270,43 @@ slideNumber: true
 <!--   - GPIO0 has to be low when it is turned back on -->
 <!-- - pull RST high to turn it on -->
 <!-- - pull GPIO0 high after the esp has started -->
 <!--   - GPIO0 has to be low when it is turned back on -->
 <!-- - pull RST high to turn it on -->
 <!-- - pull GPIO0 high after the esp has started -->

+
+## Boot Mode
+
+- GPIO0 low at start
+- Wire pins to RST and GPIO0
+- Pull both low
+- Then pull RST high first and GPIO0 after
+- Supported when using Raspberry Pi or ESP as host
+
+::: notes
+For development also possible via USB on linux hosts
+:::
+
+
+

 <!-- overview: -->
 <!-- - bootmode/serial -->
 <!-- - serial protocol -->
 <!-- - flash loader -->
 <!-- overview: -->
 <!-- - bootmode/serial -->
 <!-- - serial protocol -->
 <!-- - flash loader -->

+
+## Summary
+
+- Connect to serial and set to boot mode
+- Use serial protocol to identify the chip
+- Upload modified flash loader
+- Extract
+
+::: notes
+what is left:
+
+- differences between esp versions
+- how to extract files
+- what is on the files?
+:::
+
+
+

 <!-- differences between esp versions -->
 <!-- - identification: -->
 <!--   - ESP32-C3 and later use GET_SECURITY_INFO which contains a chip_id -->
 <!-- differences between esp versions -->
 <!-- - identification: -->
 <!--   - ESP32-C3 and later use GET_SECURITY_INFO which contains a chip_id -->


@@ -161,81 +316,107 @@ slideNumber: true
 <!--   - esp8266 mac has to be calculated -->
 <!-- - different flash loader versions -->
 <!--   - esp8266 has no data section -->
 <!--   - esp8266 mac has to be calculated -->
 <!-- - different flash loader versions -->
 <!--   - esp8266 has no data section -->

-<!-- different hosts: -->
-<!-- - linux (usb) -->
-<!-- - rpi (gpio) -->
-<!-- - esp (gpio) -->
+
+## Different ESP Versions
+
+- ESP32-C3 and later are identified using `GET_SECURITY_INFO` command
+- Earlier versions have a register with a magic value
+- Try security info first and reset on failure
+- Different registers containing MAC address
+
+::: notes
+- sec info first bc it fails with unknown command
+- read register is a known command
+- reset -> bootmode -> sync again
+- on ESP8266 mac has to be calculated
+- esp8266 has no data section when compiling flash loader
+:::
+
+
+

 <!-- extracting the file system -->
 <!-- - esp8266 -->
 <!-- - esp32 -->
 <!-- extracting the file system -->
 <!-- - esp8266 -->
 <!-- - esp32 -->

-<!-- interesting files -->
-<!-- - wifi credentials -->
-<!-- - certificates -->
-<!-- - jwt token -->

 
 

+## Extracting File Systems

 
 

-## Demo
+- SPIFFS/Littlefs
+- Gen 1 devices have known layout
+- On newer devices file system is found using metadata
+- Files can be extracted using specialized tools

 
 
 
 

-## cloc

 
 

-## 
+<!-- interesting files -->
+<!-- - wifi credentials -->
+<!-- - certificates -->
+<!-- - jwt token -->

 
 

-```sh
-$ cloc esp-flasher-stub/
------------------------------------------------------------
-Language          files       blank     comment        code
------------------------------------------------------------
-Rust                 12         327          78        1863
-Logos                14          32           0         249
-YAML                  3          34          12         214
-Markdown              1          34           0          89
-TOML                  4           8           2          61
------------------------------------------------------------
-SUM:                 34         435          92        2476
------------------------------------------------------------
+#### Shelly Plus 1 File System
+
+```txt
+api_math.js
+ca.pem                // signing CA
+conf0.json            // empty configuration
+conf3.json            // initial configuration
+conf9.json            // current configuration
+index.html.gz         // web frontend
+init.js
+rpc_acl_auth.json     // list of rpc endpoints
+rpc_acl_no_auth.json
+shelly_cloud.pem      // Shelly Cloud certificate
+shelly_plugin_api.js
+storage.json          // user data
+tzinfo
+updater.dat

 ```
 
 ```
 

-## 
-
-```sh
-$ cloc esp-hal
------------------------------------------------------------
-Language          files       blank     comment        code
------------------------------------------------------------
-Rust                492       18739       26120      115809
-Linker Script        51         499        1404       11315
-Markdown             66        2014          12        5262
-TOML                 43         505         412        4481
-Logos                44         229          15        1105
-YAML                 13         151          67        1054
-Jinja Template        3          52           0         255
-JSON                  2           0           0          48
-CSV                   3           0           0          21
-SVG                   1           0           0           4
------------------------------------------------------------
-SUM:                718       22189       28030      139354
------------------------------------------------------------
+#### `conf9.json`
+
+```txt
+{
+  "wifi": {
+    "sta": {
+      "enable": true,
+      "ssid": "SSID",
+      "pass": "PASSWORD"
+      "last_bssid": "aa:17:5d:15:ae:03",
+    },
+  },
+  "shelly": {
+    "cloud": {
+      "enable": true,
+      "server": "192.168.112.231:6022/jrpc",
+      "token": "eyJhbGciOiJIUzI1NiIsInR..."
+    },
+  },
+  ...
+}

 ```
 
 ```
 

-## 
-
-```sh
-$ cloc esptool-legacy-flasher-stub/
------------------------------------------------------------
-Language          files       blank     comment        code
------------------------------------------------------------
-Linker Script        32         787        1188       18751
-C                     6        1284         636        7689
-C/C++ Header          8         463         557        1687
-make                  1          36          33         130
-YAML                  4          19           0         114
-Python                2          25          21          80
-Markdown              3          58           0          76
-Bourne Shell          2           9           8          23
-TOML                  1           1           0          20
-Jinja Template        1           4           2          14
------------------------------------------------------------
-SUM:                 60        2686        2445       28584
------------------------------------------------------------
+#### `shelly_cloud.pem`
+
+```txt
+Subject: O = Allterco
+Not Before: Aug  4 12:03:41 2020 GMT
+Not After : Aug  2 12:03:41 2030 GMT
+-----BEGIN CERTIFICATE-----
+MIICrTCCAZUCFCuIEEAQJOFLZuEtr/CWkvxi9YPAMA0GCSqGSIb3DQEBCwUAMBMx
+ETAPBgNVBAoMCEFsbHRlcmNvMB4XDTIwMDgwNDEyMDM0MVoXDTMwMDgwMjEyMDM0
+MVowEzERMA8GA1UECgwIQWxsdGVyY28wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
+ggEKAoIBAQDHGDBDHpPUbtC9QAAjX3bi487AVY5JgYB2gyp6R9cjdsNGMbYnWdxn
+BmsIUKJPg7B5NcQObiMe6djUvwo0c2Xl9L+P9LOskP2WNDdpquX3XJu580hXHHVB
+mwOgJ0fi+5U9mOFHhc1gYGLmhO9oqsE80SgpmsPQHloMIqmcaolLzgC9PWGu8nSD
+ToJq+dXyNFHzLVyBEugHQpeIR8Fq0do4dtlsfTWvv9U+fpGPegjdkPenSxGrOVwd
+syFzNahxQGKmpZE/1fsq5QSh9+ZgwpdDChVNpkj9TBC1ApDTUasNco/6Meb/0Xur
+pxpWPNfkIpZ7ebtGHVd/ZkGTPUnL7FXHAgMBAAEwDQYJKoZIhvcNAQELBQADggEB
+ADwbvD7Mf7SOinV8JkOue8D/tvp+OiYTYLHYppzCLcBK3D1kQ7aqla2T8ebEFbLh
+hpau7MxJcizVWZs4vJvFYxEwBTQldobhmG5lHnoKQcOSxis1kWr5xdKhji/QYo6T
+wS/cird9hAcuc+RmLCMdpEDyia/vX+vvvRdyKsmB7A6Vkdu8s2B2jlhQNkitYzvq
+UDKogJrWe6fQUTpTThMyGbqhp9cQ64M4DJG1cwSBZ/hiUAMKO/y5WVNWFbXIb/Om
+xwkXgof2RXN1AjjqMcBh3GNVK4ZV5XM9WCocZjOCi2yZxaxaRWyuGR7EqAQZ+wdr
+P5XrzYspeVl1WtBzwGFssPc=
+-----END CERTIFICATE-----

 ```
 ```

+
+## Thank You