Overview of Esp32 + lvgl + MicroPython + SeedSigner

Overview of each dev layer and their purpose.

Esp32 C/C++ resources

Include and compile hardware drivers for the platform.

Basic esp-idf components

Core project dependencies are declared in idf_component.yml.

e.g.

dependencies:
  idf: ">=5.1"
  lvgl/lvgl: "~8.4.0"
  espressif/esp_lcd_axs15231b: "^1.0.0"
  espressif/esp_io_expander_tca9554: "^2.0.0"
  espressif/esp_codec_dev: "^1.3.4"
  espressif/button: "^4.1.0"

The library of available components are listed in the ESP Component Registry. For us python devs, this is somewhat akin to the requirements.txt and PyPi ecosystem.

The IDF compiler will pull these resources and write them to the managed_components/ subdir. We should not edit anything in that subdir.

In this example, esp_lcd_axs15231b is the hardware driver for that specific screen type. By including it in our managed components, we can simply #include "esp_lcd_axs15231b.h" and use the provided functions to initialize and drive the display.

It's quite convenient that Espressif also includes stock lvgl in its component registry.

Specify hardware details

Things like gpio pins, specific screen type, etc have to be specified somewhere.

In our project space, we would have something like a hardware/ subdir with a displays.c file that takes compiler params or #define constants to insantiate and manage a specific screen with specific gpio pins.

All ESP display drivers implement the same esp_lcd_panel_interface.h etc interfaces so once a driver is instantiated, the rest of the code doesn't need to know which is actually running under the hood.

Should be the same for touch input (likely built into the corresponding display driver), camera drivers, etc.

lvgl

Within the esp-bsp is: esp_lvgl_port. It can be included in idf_component.yml so it'll be automatically added to managed_components/.

The "port" part of the name is misleading. This is really a convenient utility layer that handles the ESP-specific details that are necessary to wire up and use standard lvgl.

But part of that wiring requires the hardware-specific display and touch drivers.

So esp_lvgl_port exposes a lv_port.h interface.

You pass in those required specific details and then esp_lvgl_port does the rest to initialize and run lvgl.

Note: There is also a component called esp_lvgl_adapter. It's not clear yet how it differs from esp_lvgl_port, though esp_lvgl_adapter might be newer and perhaps has more features/convenience...?

Simple main.c

With all these pieces in place, we can build a simple main.c that gets all of this support structure running. And then all that's left is to define the actual app (build a UX in lvgl and the logic to actually do something).

Here's a simple example that instantiates display and touch drivers and passes them into esp_lvgl_port and then can begin making lvgl calls: esp_lvgl_port/examples/touchscreen

Additional layers of abstraction

Cool, it all came together in main.c. But now we'll make some further separations:

hardware module

• All of the hardware-specific setup and lvgl wiring.
• Any ESP-specific drivers, interfaces, etc are contained here.
• It will instantiate the hardware drivers and wire them up to lvgl.
• We'll define a generic .h interface so external code doesn't see these details.
  • This is how future support for non-ESP hardware can be plugged in.

service modules

• Our custom lvgl display code (i.e. how to draw Screen X, Y, or Z) lives in a rendering module.
• It can assume that lvgl has already been set up by the hardware module.
• We'll provide a .h interface for:
  • init() to do all the setup.
  • render_screen_*() for each Screen X, Y, and Z.

App logic

• When to draw Screen X, Y, or Z, what info goes on Screen X, Y, or Z.
• Calls the render module to do the drawing for it.

---

MicroPython

MicroPython firmware is custom compiled from its C/C++ source code for specific esp32 BOARD and VARIANT definitions. It can also be built for a wide variety of other microcontrollers.

Lessons learned

The history of this R&D effort has been an absolute disaster trying to pull everything into MicroPython and have it be the central coordination layer. It's just too much complexity, it's been plagued by errors trying to compile hardware resources into MicroPython, and even when it does work, it's all horribly inefficient.

As far as hardware interactions are concerned: keep the complexity AWAY from MicroPython.

Smarter approach

MicroPython will be the app logic layer described above.

In order to call those init() and render_screen_*() functions, the rendering module will need to provide MicroPython bindings. Bindings are .c files that expose those functions in a way that can be compiled into MicroPython as a custom component. They can then be accessed as if they were built-in MicroPython modules:

from myproject.renderer import (render_screen_x, render_screen_y, render_screen_z)

# ...retrieve data, do calcs, etc
some_var = foo
other_var = some_var * bar

# Ready to render to the screen
render_screen_x(title="Some Screen", some_important_data=some_var, additional_data=other_var)

This allows us to run stock MicroPython that won't even be aware that lvgl is running.

The most complex hardware interactions will only be done with MicroPython being a dumb activation layer:

from qr_decoder_service import start_scanning, stop_scanning

# Tell the decoder to start the camera, feed live preview frames
# to the display, and decode any QR frames it detects.
start_scanning(results_buffer)
while True:
  if results_buffer.has_data:
    ...
  
  # Got all the QR data read in!
  stop_scanning()
  break

The above example is the most challenging as it's the one place where we will need lots of pieces coming together into a central coordinator, but that will all happen entirely in the C/C++ domain. So while it'll be complex, it's all a fairly normal use case for the C/C++ world.

However, it remains to be seen what kinds of resource management struggles we might run into with MicroPython and these C/C++ modules running in parallel: who gets to allocate what memory, who runs on which core at what priority, can filesystem resources (fonts, images, translation files) be accessed from either side, etc.

---

SeedSigner integration

In the current python-on-Raspi code, we use ButtonListScreen a lot.

In the new MicroPython + renderer module world, we'll have a pre-defined renderer function like: render_button_list_screen.

Same idea, just different implementation:

ButtonListScreen	render_button_list_screen
python	C/C++
PIL	lvgl

The dev iterations will be slower to build in C/C++, compile, then test calling from MicroPython. But we don't have that many screens that require custom implementations. And most of the custom Screen classes that we have can be further generalized so that they can be rendered with a single more flexible C/C++ equivalent).

Truly unique screens that require custom C/C++ (e.g. the Transaction Overview animated flowchart) will take some effort.

---

Expanding beyond the esp32 ecosystem

With all the layers of separation described above, we should be pretty platform-agnostic up to a certain level:

Platform-agnostic:

• MicroPython app-level logic
• Our custom C/C++ service modules
  • e.g. the renderer module only makes standard lvgl calls
• The .h interface to talk to platform-specific hardware modules

Platform-specific

• The implementation of our hardware modules
• The specific hardware drivers used
  • e.g. Espressif's built-in ST7789 display driver
• The compiler stack, dependencies, etc
  • e.g. idf_component.yml is only relevant to esp32

So when we're ready for "esp32 + MCU_X + MCU_Y + MCU_Z" cross-ecosystem support, we'll probably have to structure each as:

[platform-specific]
+ [hardware modules]
+ [SeedSigner MicroPython submodule]
+ [SeedSigner service modules submodule]