summarize-gp2040ce --- print information about firmware

can be read from a file or over USB

Signed-off-by: Brian S. Stephan <bss@incorporeal.org>

CHANGELOG.md

@@ -3,6 +3,26 @@
 Included is a summary of changes to the project. For full details, especially on behind-the-scenes code changes and
 development tools, see the commit history.
 
+## v0.7.0
+
+### Features
+
+* New configurations can be saved as .bin/.uf2 files via "Save As..." in the TUI editor. This allows for making files of
+  different configurations that can be applied on top of one another simply by dragging the tiny UF2 onto the device.
+  This is useful for backup purposes and might also be a handy way to apply different configurations in a networkless
+  environment.
+
+### Improvements
+
+* The GP2040-CE configuration structure has been updated to v0.7.8-RC2.
+* There's a small About screen now in the TUI, but I didn't get around to adding online help yet.
+* TUI tweaks, none of which are earthshattering.
+
+### Miscellaneous
+
+* The license has been changed to GPLv3 (or later).
+* Library/dependency version bumps, a couple times.
+
 ## v0.6.0
 
 ### Added

README.md

@@ -19,7 +19,7 @@ latest Protobuf files if you can.
 
 An example of this invocation is:
 
-`visualize-storage -P ~/proj/GP2040-CE/proto -P ~/proj/GP2040-CE/lib/nanopb/generator/proto --filename memory.bin`
+`visualize-config -P ~/proj/GP2040-CE/proto -P ~/proj/GP2040-CE/lib/nanopb/generator/proto --filename memory.bin`
 
 ## Installation
 
@@ -86,7 +86,7 @@ The `--...-board-config-filename` flags allow for shipping a default configurati
 the need for generating these board configurations at compile time. This allows for more custom builds and less
 dependency on the build jobs, and is a feature in progress in the core firmware.
 
-The produced firmware + config(s) can be written to a file with `--new-binary-filename FILENAME` or straight to a RP2040
+The produced firmware + config(s) can be written to a file with `--new-filename FILENAME` or straight to a RP2040
 in BOOTSEL mode with `--usb`. The output file is a direct binary representation by default, but if `FILENAME` ends in
 ".uf2", it will be written in the UF2 format, which is generally more convenient to the end user.
 
@@ -94,7 +94,7 @@ Sample usage:
 
 ```
 % concatenate build/GP2040-CE_foo_bar.bin --binary-user-config-filename storage-dump.bin \
-    --new-binary-filename new-firmware-with-config.bin
+    --new-filename new-firmware-with-config.bin
 ```
 
 ### dump-config
@@ -119,9 +119,27 @@ Sample usage:
 % dump-gp2040ce `date +%Y%m%d`-backup.bin
 ```
 
-### visualize-storage
+### summarize-gp2040ce
 
-`visualize-storage` reads a GP2040-CE board's configuration, either over USB or from a dump of the board's flash
+`summarize-gp2040ce` prints information regarding the provided USB device or file. It attempts to detect the firmware
+and/or board config and/or user config version, which might be useful for confirming files are built properly, or to
+determine the lineage of something.
+
+Sample usage:
+
+```
+% summarize-gp2040ce --usb
+USB device:
+
+GP2040-CE Information
+  detected GP2040-CE version:     v0.7.8-9-g59e2d19b-dirty
+  detected board config version:  v0.7.8-board-test
+  detected user config version:   v0.7.8-RC2-1-g882235b3
+```
+
+### visualize-config
+
+`visualize-config` reads a GP2040-CE board's configuration, either over USB or from a dump of the board's flash
 storage section, and prints it out for visual inspection or diffing with other tools. It can also find the storage
 section from a GP2040-CE whole board dump, if you have that instead. Usage is simple; just connect your board in BOOTSEL
 mode or pass the tool your binary file to analyze along with the path to the Protobuf files.
@@ -129,7 +147,7 @@ mode or pass the tool your binary file to analyze along with the path to the Pro
 Sample output:
 
 ```
-% visualize-storage --usb
+% visualize-config --usb
 boardVersion: "v0.7.2"
 gamepadOptions {
   inputMode: INPUT_MODE_HID
@@ -209,7 +227,7 @@ a huge effort to be backwards compatible, so instead, refer to this:
 
 #### Config Structures
 
-The latest update of the configuration snapshot is from **v0.7.8-RC2**.
+The latest update of the configuration snapshot is from **v0.7.8**.
 
 ### Dumping the GP2040-CE board with picotool
 

gp2040ce_bintools/builder.py

@@ -6,15 +6,14 @@ SPDX-License-Identifier: GPL-3.0-or-later
 import argparse
 import copy
 import logging
+import re
 from typing import Optional
 
 from google.protobuf.message import Message
 
+import gp2040ce_bintools.storage as storage
 from gp2040ce_bintools import core_parser
 from gp2040ce_bintools.rp2040 import get_bootsel_endpoints, read, write
-from gp2040ce_bintools.storage import (BOARD_CONFIG_BINARY_LOCATION, STORAGE_SIZE, USER_CONFIG_BINARY_LOCATION,
-                                       USER_CONFIG_BOOTSEL_ADDRESS, convert_binary_to_uf2, get_config_from_json,
-                                       pad_config_to_storage_size, serialize_config_with_footer)
 
 logger = logging.getLogger(__name__)
 
@@ -49,10 +48,10 @@ def combine_firmware_and_config(firmware_binary: bytearray, board_config_binary:
     combined = copy.copy(firmware_binary)
     if board_config_binary:
         combined = (pad_binary_up_to_board_config(combined, or_truncate=replace_extra) +
-                    pad_config_to_storage_size(board_config_binary))
+                    storage.pad_config_to_storage_size(board_config_binary))
     if user_config_binary:
         combined = (pad_binary_up_to_user_config(combined, or_truncate=replace_extra) +
-                    pad_config_to_storage_size(user_config_binary))
+                    storage.pad_config_to_storage_size(user_config_binary))
     return combined
 
 
@@ -79,36 +78,72 @@ def concatenate_firmware_and_storage_files(firmware_filename: str,
     user_config_binary = bytearray([])
 
     if binary_board_config_filename:
-        with open(binary_board_config_filename, 'rb') as storage:
-            board_config_binary = bytearray(storage.read())
+        with open(binary_board_config_filename, 'rb') as binary_file:
+            board_config_binary = bytearray(binary_file.read())
     elif json_board_config_filename:
         with open(json_board_config_filename, 'r') as json_file:
-            config = get_config_from_json(json_file.read())
-            board_config_binary = serialize_config_with_footer(config)
+            config = storage.get_config_from_json(json_file.read())
+            board_config_binary = storage.serialize_config_with_footer(config)
 
     if binary_user_config_filename:
-        with open(binary_user_config_filename, 'rb') as storage:
-            user_config_binary = bytearray(storage.read())
+        with open(binary_user_config_filename, 'rb') as binary_file:
+            user_config_binary = bytearray(binary_file.read())
     elif json_user_config_filename:
         with open(json_user_config_filename, 'r') as json_file:
-            config = get_config_from_json(json_file.read())
-            user_config_binary = serialize_config_with_footer(config)
+            config = storage.get_config_from_json(json_file.read())
+            user_config_binary = storage.serialize_config_with_footer(config)
 
     with open(firmware_filename, 'rb') as firmware:
-        new_binary = combine_firmware_and_config(bytearray(firmware.read()), board_config_binary, user_config_binary,
+        firmware_binary = bytearray(firmware.read())
+
+    # create a sequential binary for .bin and USB uses, or index it for .uf2
+    if usb or combined_filename[-4:] != '.uf2':
+        new_binary = combine_firmware_and_config(firmware_binary, board_config_binary, user_config_binary,
                                                  replace_extra=replace_extra)
+    else:
+        # this was kind of fine, but combining multiple calls of convert_binary_to_uf2 produced
+        # incorrect total block counts in the file, which picotool handled with some squirrely
+        # double-output behavior that has me worried it'd cause a real issue, so doing the
+        # crude padding + write of empty blocks, for now...
+        #
+        # new_binary = convert_binary_to_uf2(firmware_binary)
+        # if board_config_binary:
+        #     new_binary += convert_binary_to_uf2(pad_config_to_storage_size(board_config_binary),
+        #                                         start=BOARD_CONFIG_BINARY_LOCATION)
+        # if user_config_binary:
+        #     new_binary += convert_binary_to_uf2(pad_config_to_storage_size(user_config_binary),
+        #                                         start=USER_CONFIG_BINARY_LOCATION)
+        #
+        # the correct way to do the above would be to pass a list of {offset,binary_data} to convert...,
+        # and have it calculate the total block size before starting to write, and then iterating over
+        # the three lists. doable, just not on the top of my mind right now
+        new_binary = storage.convert_binary_to_uf2(combine_firmware_and_config(firmware_binary, board_config_binary,
+                                                                               user_config_binary,
+                                                                               replace_extra=replace_extra))
 
     if combined_filename:
         with open(combined_filename, 'wb') as combined:
-            if combined_filename[-4:] == '.uf2':
-                combined.write(convert_binary_to_uf2(new_binary))
-            else:
-                combined.write(new_binary)
+            combined.write(new_binary)
+
     if usb:
         endpoint_out, endpoint_in = get_bootsel_endpoints()
         write(endpoint_out, endpoint_in, GP2040CE_START_ADDRESS, bytes(new_binary))
 
 
+def find_version_string_in_binary(binary: bytes) -> str:
+    """Search for a git describe style version string in a binary file.
+
+    Args:
+        binary: the binary to search
+    Returns:
+        the first found string, or None
+    """
+    match = re.search(b'v[0-9]+.[0-9]+.[0-9]+[A-Za-z0-9-+.]*', binary)
+    if match:
+        return match.group(0).decode(encoding='ascii')
+    return 'NONE'
+
+
 def get_gp2040ce_from_usb() -> tuple[bytes, object, object]:
     """Read the firmware + config sections from a USB device.
 
@@ -157,7 +192,7 @@ def pad_binary_up_to_board_config(firmware: bytes, or_truncate: bool = False) ->
     Raises:
         FirmwareLengthError: if the firmware is larger than the storage location
     """
-    return pad_binary_up_to_address(firmware, BOARD_CONFIG_BINARY_LOCATION, or_truncate)
+    return pad_binary_up_to_address(firmware, storage.BOARD_CONFIG_BINARY_LOCATION, or_truncate)
 
 
 def pad_binary_up_to_user_config(firmware: bytes, or_truncate: bool = False) -> bytearray:
@@ -171,7 +206,7 @@ def pad_binary_up_to_user_config(firmware: bytes, or_truncate: bool = False) ->
     Raises:
         FirmwareLengthError: if the firmware is larger than the storage location
     """
-    return pad_binary_up_to_address(firmware, USER_CONFIG_BINARY_LOCATION, or_truncate)
+    return pad_binary_up_to_address(firmware, storage.USER_CONFIG_BINARY_LOCATION, or_truncate)
 
 
 def replace_config_in_binary(board_binary: bytearray, config_binary: bytearray) -> bytearray:
@@ -187,13 +222,14 @@ def replace_config_in_binary(board_binary: bytearray, config_binary: bytearray)
     Returns:
         the resulting correctly-offset binary suitable for a GP2040-CE board
     """
-    if len(board_binary) < USER_CONFIG_BINARY_LOCATION + STORAGE_SIZE:
+    if len(board_binary) < storage.USER_CONFIG_BINARY_LOCATION + storage.STORAGE_SIZE:
         # this is functionally the same, since this doesn't sanity check the firmware
         return combine_firmware_and_config(board_binary, bytearray([]), config_binary)
     else:
         new_binary = bytearray(copy.copy(board_binary))
-        new_config = pad_config_to_storage_size(config_binary)
-        new_binary[USER_CONFIG_BINARY_LOCATION:(USER_CONFIG_BINARY_LOCATION + STORAGE_SIZE)] = new_config
+        new_config = storage.pad_config_to_storage_size(config_binary)
+        location_end = storage.USER_CONFIG_BINARY_LOCATION + storage.STORAGE_SIZE
+        new_binary[storage.USER_CONFIG_BINARY_LOCATION:location_end] = new_config
         return new_binary
 
 
@@ -210,18 +246,18 @@ def write_new_config_to_filename(config: Message, filename: str, inject: bool =
                 the whole file is replaced
     """
     if inject:
-        config_binary = serialize_config_with_footer(config)
+        config_binary = storage.serialize_config_with_footer(config)
         with open(filename, 'rb') as file:
             existing_binary = file.read()
         binary = replace_config_in_binary(bytearray(existing_binary), config_binary)
         with open(filename, 'wb') as file:
             file.write(binary)
     else:
-        binary = serialize_config_with_footer(config)
+        binary = storage.serialize_config_with_footer(config)
         with open(filename, 'wb') as file:
             if filename[-4:] == '.uf2':
-                file.write(convert_binary_to_uf2(pad_config_to_storage_size(binary),
-                                                 start=USER_CONFIG_BINARY_LOCATION))
+                file.write(storage.convert_binary_to_uf2(storage.pad_config_to_storage_size(binary),
+                                                         start=storage.USER_CONFIG_BINARY_LOCATION))
             else:
                 file.write(binary)
 
@@ -234,7 +270,7 @@ def write_new_config_to_usb(config: Message, endpoint_out: object, endpoint_in:
         endpoint_out: the USB endpoint to write to
         endpoint_in: the USB endpoint to read from
     """
-    serialized = serialize_config_with_footer(config)
+    serialized = storage.serialize_config_with_footer(config)
     # we don't write the whole area, just the minimum from the end of the storage section
     # nevertheless, the USB device needs writes to start at 256 byte boundaries
     logger.debug("serialized: %s", serialized)
@@ -245,7 +281,8 @@ def write_new_config_to_usb(config: Message, endpoint_out: object, endpoint_in:
     logger.debug("length: %s with %s bytes of padding", len(serialized), padding)
     binary = bytearray(b'\x00' * padding) + serialized
     logger.debug("binary for writing: %s", binary)
-    write(endpoint_out, endpoint_in, USER_CONFIG_BOOTSEL_ADDRESS + (STORAGE_SIZE - len(binary)), bytes(binary))
+    write(endpoint_out, endpoint_in, storage.USER_CONFIG_BOOTSEL_ADDRESS + (storage.STORAGE_SIZE - len(binary)),
+          bytes(binary))
 
 
 ############
@@ -298,3 +335,40 @@ def dump_gp2040ce():
     content, _, _ = get_gp2040ce_from_usb()
     with open(args.binary_filename, 'wb') as out_file:
         out_file.write(content)
+
+
+def summarize_gp2040ce():
+    """Provide information on a dump or USB device."""
+    parser = argparse.ArgumentParser(
+        description="Read a file or USB device to determine what GP2040-CE parts are present.",
+        parents=[core_parser],
+    )
+    input_group = parser.add_mutually_exclusive_group(required=True)
+    input_group.add_argument('--usb', action='store_true', help="inspect the RP2040 device over USB")
+    input_group.add_argument('--filename', help="input .bin or .uf2 file to inspect")
+
+    args, _ = parser.parse_known_args()
+    if args.usb:
+        content, endpoint, _ = get_gp2040ce_from_usb()
+        print(f"USB device {hex(endpoint.device.idVendor)}:{hex(endpoint.device.idProduct)}:\n")
+    else:
+        with open(args.filename, 'rb') as file_:
+            content = file_.read()
+        print(f"File {args.filename}:\n")
+
+    gp2040ce_version = find_version_string_in_binary(content)
+    try:
+        board_config = storage.get_config(storage.get_board_storage_section(bytes(content)))
+        board_config_version = board_config.boardVersion if board_config.boardVersion else "NOT SPECIFIED"
+    except storage.ConfigReadError:
+        board_config_version = "NONE"
+    try:
+        user_config = storage.get_config(storage.get_user_storage_section(bytes(content)))
+        user_config_version = user_config.boardVersion if user_config.boardVersion else "NOT FOUND"
+    except storage.ConfigReadError:
+        user_config_version = "NONE"
+
+    print("GP2040-CE Information")
+    print(f"  detected GP2040-CE version:     {gp2040ce_version}")
+    print(f"  detected board config version:  {board_config_version}")
+    print(f"  detected user config version:   {user_config_version}")

gp2040ce_bintools/gui.py

@@ -54,9 +54,7 @@ class EditScreen(ModalScreen):
             self.input_field = Input(value=repr(self.field_value), validators=[Number()], id='field-input')
         elif self.field_descriptor.type == descriptor.FieldDescriptor.TYPE_STRING:
             self.input_field = Input(value=self.field_value, id='field-input')
-        else:
-            # we don't handle whatever these are yet
-            self.input_field = Label(repr(self.field_value), id='field-input')
+
         yield Grid(
             Container(Label(self.field_descriptor.full_name, id='field-name'), id='field-name-container'),
             Container(self.input_field, id='input-field-container'),
@@ -91,17 +89,16 @@ class EditScreen(ModalScreen):
 
     def _save(self):
         """Save the field value to the retained config item."""
-        if not isinstance(self.input_field, Label):
-            if self.field_descriptor.type in (descriptor.FieldDescriptor.TYPE_INT32,
-                                              descriptor.FieldDescriptor.TYPE_INT64,
-                                              descriptor.FieldDescriptor.TYPE_UINT32,
-                                              descriptor.FieldDescriptor.TYPE_UINT64):
-                field_value = int(self.input_field.value)
-            else:
-                field_value = self.input_field.value
-            setattr(self.parent_config, self.field_descriptor.name, field_value)
-            logger.debug("parent config post-change: %s", self.parent_config)
-            self.node.set_label(pb_field_to_node_label(self.field_descriptor, field_value))
+        if self.field_descriptor.type in (descriptor.FieldDescriptor.TYPE_INT32,
+                                          descriptor.FieldDescriptor.TYPE_INT64,
+                                          descriptor.FieldDescriptor.TYPE_UINT32,
+                                          descriptor.FieldDescriptor.TYPE_UINT64):
+            field_value = int(self.input_field.value)
+        else:
+            field_value = self.input_field.value
+        setattr(self.parent_config, self.field_descriptor.name, field_value)
+        logger.debug("parent config post-change: %s", self.parent_config)
+        self.node.set_label(pb_field_to_node_label(self.field_descriptor, field_value))
 
 
 class MessageScreen(ModalScreen):

gp2040ce_bintools/storage.py

@@ -86,6 +86,44 @@ def convert_binary_to_uf2(binary: bytearray, start: int = 0) -> bytearray:
     return uf2
 
 
+def convert_uf2_to_binary(uf2: bytearray) -> bytearray:
+    """Convert a Microsoft's UF2 payload to a raw binary.
+
+    https://github.com/microsoft/uf2/tree/master#overview
+
+    Args:
+        uf2: bytearray content to convert from a UF2 payload
+    Returns:
+        the content in sequential binary format
+    """
+    if len(uf2) % 512 != 0:
+        raise ValueError(f"provided binary is length {len(uf2)}, which isn't fully divisible by 512!")
+
+    binary = bytearray()
+    old_uf2_addr = None
+
+    for index in range(0, len(uf2), 512):
+        chunk = uf2[index:index+512]
+        _, _, _, uf2_addr, bytes_, block_num, block_count, _ = struct.unpack('<LLLLLLLL', chunk[0:32])
+        content = chunk[32:508]
+        if block_num != index // 512:
+            raise ValueError(f"inconsistent block number in reading UF2, got {block_num}, expected {index // 512}!")
+        if block_count != len(uf2) // 512:
+            raise ValueError(f"inconsistent block count in reading UF2, got {block_count}, expected {len(uf2) // 512}!")
+
+        # the UF2 is indexed, which we could convert to binary with padding 0s, but we don't yet
+        if old_uf2_addr and (uf2_addr != old_uf2_addr + 256):
+            raise ValueError("segmented UF2 files are not yet supported!")
+
+        binary += content[0:bytes_]
+        old_uf2_addr = uf2_addr
+
+    # when this is all done we should have counted the expected number of blocks
+    if block_count != block_num + 1:
+        raise ValueError(f"not all expected blocks ({block_count}) were found, only got {block_num + 1}!")
+    return binary
+
+
 def get_config(content: bytes) -> Message:
     """Read the config from a GP2040-CE storage section.
 
@@ -178,7 +216,10 @@ def get_config_from_file(filename: str, whole_board: bool = False, allow_no_file
     """
     try:
         with open(filename, 'rb') as dump:
-            content = dump.read()
+            if filename[-4:] == '.uf2':
+                content = bytes(convert_uf2_to_binary(bytearray(dump.read())))
+            else:
+                content = dump.read()
     except FileNotFoundError:
         if not allow_no_file:
             raise
@@ -324,10 +365,15 @@ def dump_config():
         description="Read the configuration section from a USB device and save it to a binary file.",
         parents=[core_parser],
     )
+    parser.add_argument('--board-config', action='store_true', default=False,
+                        help="dump the board config rather than the user config")
     parser.add_argument('filename', help="file to save the GP2040-CE board's config section to --- if the "
                                          "suffix is .uf2, it is saved in UF2 format, else it is a raw binary")
     args, _ = parser.parse_known_args()
-    config, _, _ = get_user_config_from_usb()
+    if args.board_config:
+        config, _, _ = get_board_config_from_usb()
+    else:
+        config, _, _ = get_user_config_from_usb()
     binary_config = serialize_config_with_footer(config)
     with open(args.filename, 'wb') as out_file:
         if args.filename[-4:] == '.uf2':

pyproject.toml

@@ -38,7 +38,8 @@ concatenate = "gp2040ce_bintools.builder:concatenate"
 dump-config = "gp2040ce_bintools.storage:dump_config"
 dump-gp2040ce = "gp2040ce_bintools.builder:dump_gp2040ce"
 edit-config = "gp2040ce_bintools.gui:edit_config"
-visualize-storage = "gp2040ce_bintools.storage:visualize"
+summarize-gp2040ce = "gp2040ce_bintools.builder:summarize_gp2040ce"
+visualize-config = "gp2040ce_bintools.storage:visualize"
 
 [tool.flake8]
 enable-extensions = "G,M"

tests/test_builder.py

@@ -10,11 +10,8 @@ import unittest.mock as mock
 import pytest
 from decorator import decorator
 
+import gp2040ce_bintools.builder as builder
 from gp2040ce_bintools import get_config_pb2
-from gp2040ce_bintools.builder import (FirmwareLengthError, combine_firmware_and_config,
-                                       concatenate_firmware_and_storage_files, get_gp2040ce_from_usb,
-                                       pad_binary_up_to_board_config, pad_binary_up_to_user_config,
-                                       replace_config_in_binary, write_new_config_to_filename, write_new_config_to_usb)
 from gp2040ce_bintools.storage import (STORAGE_SIZE, get_board_storage_section, get_config, get_config_footer,
                                        get_user_storage_section, serialize_config_with_footer)
 
@@ -38,8 +35,8 @@ def test_concatenate_to_file(tmp_path):
     tmp_file = os.path.join(tmp_path, 'concat.bin')
     firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
     config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
-    concatenate_firmware_and_storage_files(firmware_file, binary_user_config_filename=config_file,
-                                           combined_filename=tmp_file)
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_user_config_filename=config_file,
+                                                   combined_filename=tmp_file)
     with open(tmp_file, 'rb') as file:
         content = file.read()
     assert len(content) == 2 * 1024 * 1024
@@ -50,8 +47,8 @@ def test_concatenate_board_config_to_file(tmp_path):
     tmp_file = os.path.join(tmp_path, 'concat.bin')
     firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
     config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
-    concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
-                                           combined_filename=tmp_file)
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+                                                   combined_filename=tmp_file)
     with open(tmp_file, 'rb') as file:
         content = file.read()
     assert len(content) == (2 * 1024 * 1024) - (16 * 1024)
@@ -62,8 +59,8 @@ def test_concatenate_both_configs_to_file(tmp_path):
     tmp_file = os.path.join(tmp_path, 'concat.bin')
     firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
     config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
-    concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
-                                           binary_user_config_filename=config_file, combined_filename=tmp_file)
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+                                                   binary_user_config_filename=config_file, combined_filename=tmp_file)
     with open(tmp_file, 'rb') as file:
         content = file.read()
     assert len(content) == 2 * 1024 * 1024
@@ -81,8 +78,8 @@ def test_concatenate_user_json_to_file(tmp_path):
     tmp_file = os.path.join(tmp_path, 'concat.bin')
     firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
     config_file = os.path.join(HERE, 'test-files', 'test-config.json')
-    concatenate_firmware_and_storage_files(firmware_file, json_user_config_filename=config_file,
-                                           combined_filename=tmp_file)
+    builder.concatenate_firmware_and_storage_files(firmware_file, json_user_config_filename=config_file,
+                                                   combined_filename=tmp_file)
     with open(tmp_file, 'rb') as file:
         content = file.read()
     assert len(content) == 2 * 1024 * 1024
@@ -93,7 +90,7 @@ def test_concatenate_to_file_incomplete_args_is_error(tmp_path):
     tmp_file = os.path.join(tmp_path, 'concat.bin')
     firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
     with pytest.raises(ValueError):
-        concatenate_firmware_and_storage_files(firmware_file, combined_filename=tmp_file)
+        builder.concatenate_firmware_and_storage_files(firmware_file, combined_filename=tmp_file)
 
 
 def test_concatenate_to_usb(tmp_path):
@@ -103,34 +100,71 @@ def test_concatenate_to_usb(tmp_path):
     end_out, end_in = mock.MagicMock(), mock.MagicMock()
     with mock.patch('gp2040ce_bintools.builder.get_bootsel_endpoints', return_value=(end_out, end_in)):
         with mock.patch('gp2040ce_bintools.builder.write') as mock_write:
-            concatenate_firmware_and_storage_files(firmware_file, binary_user_config_filename=config_file,
-                                                   usb=True)
+            builder.concatenate_firmware_and_storage_files(firmware_file, binary_user_config_filename=config_file,
+                                                           usb=True)
 
     assert mock_write.call_args.args[2] == 0x10000000
     assert len(mock_write.call_args.args[3]) == 2 * 1024 * 1024
 
 
+def test_concatenate_to_uf2(tmp_path, firmware_binary, config_binary):
+    """Test that we write a UF2 file as expected."""
+    tmp_file = os.path.join(tmp_path, 'concat.uf2')
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+                                                   binary_user_config_filename=config_file,
+                                                   combined_filename=tmp_file)
+    with open(tmp_file, 'rb') as file:
+        content = file.read()
+    # size of the file should be 2x the binary version, and the binary is 2 MB
+    assert len(content) == 2 * 2 * 1024 * 1024
+
+
+def test_concatenate_to_uf2_board_only(tmp_path, firmware_binary, config_binary):
+    """Test that we write a UF2 file as expected."""
+    tmp_file = os.path.join(tmp_path, 'concat.uf2')
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+                                                   combined_filename=tmp_file)
+    with open(tmp_file, 'rb') as file:
+        content = file.read()
+    # size of the file should be 2x the binary version (minus user config space), and the binary is 2 MB - 16KB
+    assert len(content) == 2 * (2 * 1024 * 1024 - 16384)
+
+
+def test_find_version_string(firmware_binary):
+    """Test that we can find a version string in a binary."""
+    assert builder.find_version_string_in_binary(firmware_binary) == 'v0.7.5'
+
+
+def test_dont_always_find_version_string(firmware_binary):
+    """Test that we can find a version string in a binary."""
+    assert builder.find_version_string_in_binary(b'\x00') == 'NONE'
+
+
 def test_padding_firmware(firmware_binary):
     """Test that firmware is padded to the expected size."""
-    padded = pad_binary_up_to_user_config(firmware_binary)
+    padded = builder.pad_binary_up_to_user_config(firmware_binary)
     assert len(padded) == 2080768
 
 
 def test_padding_firmware_can_truncate():
     """Test that firmware is padded to the expected size."""
-    padded = pad_binary_up_to_user_config(bytearray(b'\x00' * 4 * 1024 * 1024), or_truncate=True)
+    padded = builder.pad_binary_up_to_user_config(bytearray(b'\x00' * 4 * 1024 * 1024), or_truncate=True)
     assert len(padded) == 2080768
 
 
 def test_padding_firmware_to_board(firmware_binary):
     """Test that firmware is padded to the expected size."""
-    padded = pad_binary_up_to_board_config(firmware_binary)
+    padded = builder.pad_binary_up_to_board_config(firmware_binary)
     assert len(padded) == 2080768 - (16 * 1024)
 
 
 def test_firmware_plus_storage_section(firmware_binary, storage_dump):
     """Test that combining firmware and storage produces a valid combined binary."""
-    whole_board = combine_firmware_and_config(firmware_binary, None, storage_dump)
+    whole_board = builder.combine_firmware_and_config(firmware_binary, None, storage_dump)
     # if this is valid, we should be able to find the storage and footer again
     storage = get_user_storage_section(whole_board)
     footer_size, _, _ = get_config_footer(storage)
@@ -139,7 +173,7 @@ def test_firmware_plus_storage_section(firmware_binary, storage_dump):
 
 def test_firmware_plus_user_config_binary(firmware_binary, config_binary):
     """Test that combining firmware and user config produces a valid combined binary."""
-    whole_board = combine_firmware_and_config(firmware_binary, None, config_binary)
+    whole_board = builder.combine_firmware_and_config(firmware_binary, None, config_binary)
     # if this is valid, we should be able to find the storage and footer again
     storage = get_user_storage_section(whole_board)
     footer_size, _, _ = get_config_footer(storage)
@@ -148,8 +182,8 @@ def test_firmware_plus_user_config_binary(firmware_binary, config_binary):
 
 def test_chunky_firmware_plus_user_config_binary(config_binary):
     """Test that combining giant firmware and storage produces a valid combined binary."""
-    whole_board = combine_firmware_and_config(bytearray(b'\x00' * 4 * 1024 * 1024), None, config_binary,
-                                              replace_extra=True)
+    whole_board = builder.combine_firmware_and_config(bytearray(b'\x00' * 4 * 1024 * 1024), None, config_binary,
+                                                      replace_extra=True)
     # if this is valid, we should be able to find the storage and footer again
     storage = get_user_storage_section(whole_board)
     footer_size, _, _ = get_config_footer(storage)
@@ -158,7 +192,7 @@ def test_chunky_firmware_plus_user_config_binary(config_binary):
 
 def test_firmware_plus_board_config_binary(firmware_binary, config_binary):
     """Test that combining firmware and board config produces a valid combined binary."""
-    almost_whole_board = combine_firmware_and_config(firmware_binary, config_binary, None)
+    almost_whole_board = builder.combine_firmware_and_config(firmware_binary, config_binary, None)
     assert len(almost_whole_board) == (2 * 1024 * 1024) - (16 * 1024)
     # if this is valid, we should be able to find the storage and footer again
     storage = get_board_storage_section(almost_whole_board)
@@ -168,7 +202,7 @@ def test_firmware_plus_board_config_binary(firmware_binary, config_binary):
 
 def test_firmware_plus_board_and_user_config_binary(firmware_binary, config_binary):
     """Test that combining firmware and both board and user configs produces a valid combined binary."""
-    whole_board = combine_firmware_and_config(firmware_binary, config_binary, config_binary)
+    whole_board = builder.combine_firmware_and_config(firmware_binary, config_binary, config_binary)
     assert len(whole_board) == 2 * 1024 * 1024
     # if this is valid, we should be able to find the storage and footer again
     storage = get_board_storage_section(whole_board)
@@ -182,12 +216,12 @@ def test_firmware_plus_board_and_user_config_binary(firmware_binary, config_bina
 def test_combine_must_get_at_least_one_config(firmware_binary):
     """Test that we error if we are asked to combine with nothing to combine."""
     with pytest.raises(ValueError):
-        combine_firmware_and_config(firmware_binary, None, None)
+        builder.combine_firmware_and_config(firmware_binary, None, None)
 
 
 def test_replace_config_in_binary(config_binary):
     """Test that a config binary is placed in the storage location of a source binary to overwrite."""
-    whole_board = replace_config_in_binary(bytearray(b'\x00' * 3 * 1024 * 1024), config_binary)
+    whole_board = builder.replace_config_in_binary(bytearray(b'\x00' * 3 * 1024 * 1024), config_binary)
     assert len(whole_board) == 3 * 1024 * 1024
     # if this is valid, we should be able to find the storage and footer again
     storage = get_user_storage_section(whole_board)
@@ -197,7 +231,7 @@ def test_replace_config_in_binary(config_binary):
 
 def test_replace_config_in_binary_not_big_enough(config_binary):
     """Test that a config binary is placed in the storage location of a source binary to pad."""
-    whole_board = replace_config_in_binary(bytearray(b'\x00' * 1 * 1024 * 1024), config_binary)
+    whole_board = builder.replace_config_in_binary(bytearray(b'\x00' * 1 * 1024 * 1024), config_binary)
     assert len(whole_board) == 2 * 1024 * 1024
     # if this is valid, we should be able to find the storage and footer again
     storage = get_user_storage_section(whole_board)
@@ -207,8 +241,8 @@ def test_replace_config_in_binary_not_big_enough(config_binary):
 
 def test_padding_firmware_too_big(firmware_binary):
     """Test that firmware is padded to the expected size."""
-    with pytest.raises(FirmwareLengthError):
-        _ = pad_binary_up_to_user_config(firmware_binary + firmware_binary + firmware_binary)
+    with pytest.raises(builder.FirmwareLengthError):
+        _ = builder.pad_binary_up_to_user_config(firmware_binary + firmware_binary + firmware_binary)
 
 
 @with_pb2s
@@ -224,7 +258,7 @@ def test_write_new_config_to_whole_board(whole_board_dump, tmp_path):
     config = get_config(get_user_storage_section(board_dump))
     assert config.boardVersion == 'v0.7.5'
     config.boardVersion = 'v0.7.5-COOL'
-    write_new_config_to_filename(config, tmp_file, inject=True)
+    builder.write_new_config_to_filename(config, tmp_file, inject=True)
 
     # read new file
     with open(tmp_file, 'rb') as file:
@@ -244,7 +278,7 @@ def test_write_new_config_to_firmware(firmware_binary, tmp_path):
     config_pb2 = get_config_pb2()
     config = config_pb2.Config()
     config.boardVersion = 'v0.7.5-COOL'
-    write_new_config_to_filename(config, tmp_file, inject=True)
+    builder.write_new_config_to_filename(config, tmp_file, inject=True)
 
     # read new file
     with open(tmp_file, 'rb') as file:
@@ -261,7 +295,7 @@ def test_write_new_config_to_config_bin(firmware_binary, tmp_path):
     config_pb2 = get_config_pb2()
     config = config_pb2.Config()
     config.boardVersion = 'v0.7.5-COOL'
-    write_new_config_to_filename(config, tmp_file)
+    builder.write_new_config_to_filename(config, tmp_file)
 
     # read new file
     with open(tmp_file, 'rb') as file:
@@ -279,7 +313,7 @@ def test_write_new_config_to_config_uf2(firmware_binary, tmp_path):
     config_pb2 = get_config_pb2()
     config = config_pb2.Config()
     config.boardVersion = 'v0.7.5-COOL'
-    write_new_config_to_filename(config, tmp_file)
+    builder.write_new_config_to_filename(config, tmp_file)
 
     # read new file
     with open(tmp_file, 'rb') as file:
@@ -296,7 +330,7 @@ def test_write_new_config_to_usb(config_binary):
     serialized = serialize_config_with_footer(config)
     end_out, end_in = mock.MagicMock(), mock.MagicMock()
     with mock.patch('gp2040ce_bintools.builder.write') as mock_write:
-        write_new_config_to_usb(config, end_out, end_in)
+        builder.write_new_config_to_usb(config, end_out, end_in)
 
     # check that it got padded
     assert len(serialized) == 3321
@@ -315,7 +349,7 @@ def test_get_gp2040ce_from_usb():
     mock_in = mock.MagicMock()
     with mock.patch('gp2040ce_bintools.builder.get_bootsel_endpoints', return_value=(mock_out, mock_in)) as mock_get:
         with mock.patch('gp2040ce_bintools.builder.read') as mock_read:
-            config, _, _ = get_gp2040ce_from_usb()
+            config, _, _ = builder.get_gp2040ce_from_usb()
 
     mock_get.assert_called_once()
     mock_read.assert_called_with(mock_out, mock_in, 0x10000000, 2 * 1024 * 1024)

tests/test_commands.py

@@ -29,14 +29,14 @@ def with_pb2s(test, *args, **kwargs):
 
 def test_version_flag():
     """Test that tools report the version."""
-    result = run(['visualize-storage', '-v'], capture_output=True, encoding='utf8')
+    result = run(['visualize-config', '-v'], capture_output=True, encoding='utf8')
     assert __version__ in result.stdout
 
 
 def test_help_flag():
     """Test that tools report the usage information."""
-    result = run(['visualize-storage', '-h'], capture_output=True, encoding='utf8')
-    assert 'usage: visualize-storage' in result.stdout
+    result = run(['visualize-config', '-h'], capture_output=True, encoding='utf8')
+    assert 'usage: visualize-config' in result.stdout
     assert 'Read the configuration section from a dump of a GP2040-CE board' in result.stdout
 
 
@@ -64,9 +64,16 @@ def test_concatenate_invocation_json(tmpdir):
     assert out[2093382:2097152] == storage
 
 
+def test_summarize_invocation(tmpdir):
+    """Test that we can get some summary information."""
+    result = run(['summarize-gp2040ce', '--filename', 'tests/test-files/test-firmware.bin'],
+                 capture_output=True, encoding='utf8')
+    assert 'detected GP2040-CE version:     v0.7.5' in result.stdout
+
+
 def test_storage_dump_invocation():
     """Test that a normal invocation against a dump works."""
-    result = run(['visualize-storage', '-P', 'tests/test-files/proto-files',
+    result = run(['visualize-config', '-P', 'tests/test-files/proto-files',
                   '--filename', 'tests/test-files/test-storage-area.bin'],
                  capture_output=True, encoding='utf8')
     assert 'boardVersion: "v0.7.5"' in result.stdout
@@ -74,7 +81,7 @@ def test_storage_dump_invocation():
 
 def test_debug_storage_dump_invocation():
     """Test that a normal invocation against a dump works."""
-    result = run(['visualize-storage', '-d', '-P', 'tests/test-files/proto-files',
+    result = run(['visualize-config', '-d', '-P', 'tests/test-files/proto-files',
                   '--filename', 'tests/test-files/test-storage-area.bin'],
                  capture_output=True, encoding='utf8')
     assert 'boardVersion: "v0.7.5"' in result.stdout
@@ -83,7 +90,7 @@ def test_debug_storage_dump_invocation():
 
 def test_storage_dump_json_invocation():
     """Test that a normal invocation against a dump works."""
-    result = run(['visualize-storage', '-P', 'tests/test-files/proto-files', '--json',
+    result = run(['visualize-config', '-P', 'tests/test-files/proto-files', '--json',
                   '--filename', 'tests/test-files/test-storage-area.bin'],
                  capture_output=True, encoding='utf8')
     to_dict = json.loads(result.stdout)

tests/test_gui.py

@@ -125,6 +125,41 @@ async def test_simple_edit_via_input_field():
         assert pilot.app.config.displayOptions.deprecatedI2cSpeed == 5
 
 
+@pytest.mark.asyncio
+@with_pb2s
+async def test_cancel_simple_edit_via_input_field():
+    """Test that we can cancel out of saving an int via UI and see it reflected in the config."""
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
+    async with app.run_test() as pilot:
+        tree = pilot.app.query_one(Tree)
+        display_node = tree.root.children[5]
+        i2cspeed_node = display_node.children[4]
+        assert pilot.app.config.displayOptions.deprecatedI2cSpeed == 400000
+
+        tree.root.expand_all()
+        await pilot.wait_for_scheduled_animations()
+        tree.select_node(i2cspeed_node)
+        tree.action_select_cursor()
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Input#field-input')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.press('backspace', 'backspace', 'backspace', 'backspace', 'backspace', 'backspace', '5')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Button#cancel-button')
+        assert pilot.app.config.displayOptions.deprecatedI2cSpeed == 400000
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_about():
+    """Test that we can bring up the about box."""
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
+    async with app.run_test() as pilot:
+        await pilot.press('?')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Button#ok-button')
+
+
 @pytest.mark.asyncio
 @with_pb2s
 async def test_simple_edit_via_input_field_enum():

tests/test_storage.py

@@ -3,6 +3,7 @@
 SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
 SPDX-License-Identifier: GPL-3.0-or-later
 """
+import math
 import os
 import sys
 import unittest.mock as mock
@@ -139,6 +140,15 @@ def test_convert_binary_to_uf2(whole_board_with_board_config_dump):
     assert uf2[524:528] == bytearray(b'\x00\x01\x00\x10')   # address to write the second chunk
 
 
+def test_convert_unaligned_binary_to_uf2(firmware_binary):
+    """Do some sanity checks in the attempt to convert a binary to a UF2."""
+    uf2 = storage.convert_binary_to_uf2(firmware_binary)
+    assert len(uf2) == math.ceil(len(firmware_binary)/256) * 512    # 256 byte complete/partial chunks -> 512 b chunks
+    assert uf2[0:4] == b'\x55\x46\x32\x0a' == b'UF2\n'      # proper magic
+    assert uf2[8:12] == bytearray(b'\x00\x20\x00\x00')      # family ID set
+    assert uf2[524:528] == bytearray(b'\x00\x01\x00\x10')   # address to write the second chunk
+
+
 def test_convert_binary_to_uf2_with_offsets(whole_board_with_board_config_dump):
     """Do some sanity checks in the attempt to convert a binary to a UF2."""
     uf2 = storage.convert_binary_to_uf2(whole_board_with_board_config_dump, start=storage.USER_CONFIG_BINARY_LOCATION)
@@ -148,6 +158,35 @@ def test_convert_binary_to_uf2_with_offsets(whole_board_with_board_config_dump):
     assert uf2[524:528] == bytearray(b'\x00\xc1\x1f\x10')   # address to write the second chunk
 
 
+def test_convert_binary_to_uf2_to_binary(whole_board_with_board_config_dump):
+    """Do some sanity checks in the attempt to convert a binary to a UF2."""
+    uf2 = storage.convert_binary_to_uf2(whole_board_with_board_config_dump)
+    binary = storage.convert_uf2_to_binary(uf2)
+    assert len(binary) == 2097152
+    assert whole_board_with_board_config_dump == binary
+
+
+def test_malformed_uf2(whole_board_with_board_config_dump):
+    """Check that we expect a properly-formed UF2."""
+    uf2 = storage.convert_binary_to_uf2(whole_board_with_board_config_dump)
+
+    # truncated UF2 --- byte mismatch
+    with pytest.raises(ValueError):
+        storage.convert_uf2_to_binary(uf2[:-4])
+
+    # truncated uf2 --- counter is wrong
+    with pytest.raises(ValueError):
+        storage.convert_uf2_to_binary(uf2[512:])
+
+    # truncated uf2 --- total count is wrong
+    with pytest.raises(ValueError):
+        storage.convert_uf2_to_binary(uf2[:-512])
+
+    # malformed UF2 --- counter jumps in the middle, suggests total blocks is wrong
+    with pytest.raises(ValueError):
+        storage.convert_uf2_to_binary(uf2 + uf2)
+
+
 @with_pb2s
 def test_serialize_config_with_footer(storage_dump, config_binary):
     """Test that reserializing a read in config matches the original.