CHANGELOG.md

@@ -3,26 +3,6 @@
 Included is a summary of changes to the project. For full details, especially on behind-the-scenes code changes and
 development tools, see the commit history.
 
-## v0.7.0
-
-### Features
-
-* New configurations can be saved as .bin/.uf2 files via "Save As..." in the TUI editor. This allows for making files of
-  different configurations that can be applied on top of one another simply by dragging the tiny UF2 onto the device.
-  This is useful for backup purposes and might also be a handy way to apply different configurations in a networkless
-  environment.
-
-### Improvements
-
-* The GP2040-CE configuration structure has been updated to v0.7.8-RC2.
-* There's a small About screen now in the TUI, but I didn't get around to adding online help yet.
-* TUI tweaks, none of which are earthshattering.
-
-### Miscellaneous
-
-* The license has been changed to GPLv3 (or later).
-* Library/dependency version bumps, a couple times.
-
 ## v0.6.0
 
 ### Added

README.md

@@ -19,7 +19,7 @@ latest Protobuf files if you can.
 
 An example of this invocation is:
 
-`visualize-config -P ~/proj/GP2040-CE/proto -P ~/proj/GP2040-CE/lib/nanopb/generator/proto --filename memory.bin`
+`visualize-storage -P ~/proj/GP2040-CE/proto -P ~/proj/GP2040-CE/lib/nanopb/generator/proto --filename memory.bin`
 
 ## Installation
 
@@ -86,7 +86,7 @@ The `--...-board-config-filename` flags allow for shipping a default configurati
 the need for generating these board configurations at compile time. This allows for more custom builds and less
 dependency on the build jobs, and is a feature in progress in the core firmware.
 
-The produced firmware + config(s) can be written to a file with `--new-filename FILENAME` or straight to a RP2040
+The produced firmware + config(s) can be written to a file with `--new-binary-filename FILENAME` or straight to a RP2040
 in BOOTSEL mode with `--usb`. The output file is a direct binary representation by default, but if `FILENAME` ends in
 ".uf2", it will be written in the UF2 format, which is generally more convenient to the end user.
 
@@ -94,7 +94,7 @@ Sample usage:
 
 ```
 % concatenate build/GP2040-CE_foo_bar.bin --binary-user-config-filename storage-dump.bin \
-    --new-filename new-firmware-with-config.bin
+    --new-binary-filename new-firmware-with-config.bin
 ```
 
 ### dump-config
@@ -119,27 +119,9 @@ Sample usage:
 % dump-gp2040ce `date +%Y%m%d`-backup.bin
 ```
 
-### summarize-gp2040ce
+### visualize-storage
 
-`summarize-gp2040ce` prints information regarding the provided USB device or file. It attempts to detect the firmware
+`visualize-storage` reads a GP2040-CE board's configuration, either over USB or from a dump of the board's flash
-and/or board config and/or user config version, which might be useful for confirming files are built properly, or to
-determine the lineage of something.
-
-Sample usage:
-
-```
-% summarize-gp2040ce --usb
-USB device:
-
-GP2040-CE Information
-  detected GP2040-CE version:     v0.7.8-9-g59e2d19b-dirty
-  detected board config version:  v0.7.8-board-test
-  detected user config version:   v0.7.8-RC2-1-g882235b3
-```
-
-### visualize-config
-
-`visualize-config` reads a GP2040-CE board's configuration, either over USB or from a dump of the board's flash
 storage section, and prints it out for visual inspection or diffing with other tools. It can also find the storage
 section from a GP2040-CE whole board dump, if you have that instead. Usage is simple; just connect your board in BOOTSEL
 mode or pass the tool your binary file to analyze along with the path to the Protobuf files.
@@ -147,7 +129,7 @@ mode or pass the tool your binary file to analyze along with the path to the Pro
 Sample output:
 
 ```
-% visualize-config --usb
+% visualize-storage --usb
 boardVersion: "v0.7.2"
 gamepadOptions {
   inputMode: INPUT_MODE_HID
@@ -227,7 +209,7 @@ a huge effort to be backwards compatible, so instead, refer to this:
 
 #### Config Structures
 
-The latest update of the configuration snapshot is from **v0.7.8**.
+The latest update of the configuration snapshot is from **v0.7.8-RC2**.
 
 ### Dumping the GP2040-CE board with picotool
 

gp2040ce_bintools/builder.py

@@ -6,14 +6,15 @@ SPDX-License-Identifier: GPL-3.0-or-later
 import argparse
 import copy
 import logging
-import re
 from typing import Optional
 
 from google.protobuf.message import Message
 
-import gp2040ce_bintools.storage as storage
 from gp2040ce_bintools import core_parser
 from gp2040ce_bintools.rp2040 import get_bootsel_endpoints, read, write
+from gp2040ce_bintools.storage import (BOARD_CONFIG_BINARY_LOCATION, STORAGE_SIZE, USER_CONFIG_BINARY_LOCATION,
+                                       USER_CONFIG_BOOTSEL_ADDRESS, convert_binary_to_uf2, get_config_from_json,
+                                       pad_config_to_storage_size, serialize_config_with_footer)
 
 logger = logging.getLogger(__name__)
 
@@ -48,10 +49,10 @@ def combine_firmware_and_config(firmware_binary: bytearray, board_config_binary:
     combined = copy.copy(firmware_binary)
     if board_config_binary:
         combined = (pad_binary_up_to_board_config(combined, or_truncate=replace_extra) +
-                    storage.pad_config_to_storage_size(board_config_binary))
+                    pad_config_to_storage_size(board_config_binary))
     if user_config_binary:
         combined = (pad_binary_up_to_user_config(combined, or_truncate=replace_extra) +
-                    storage.pad_config_to_storage_size(user_config_binary))
+                    pad_config_to_storage_size(user_config_binary))
     return combined
 
 
@@ -78,72 +79,36 @@ def concatenate_firmware_and_storage_files(firmware_filename: str,
     user_config_binary = bytearray([])
 
     if binary_board_config_filename:
-        with open(binary_board_config_filename, 'rb') as binary_file:
+        with open(binary_board_config_filename, 'rb') as storage:
-            board_config_binary = bytearray(binary_file.read())
+            board_config_binary = bytearray(storage.read())
     elif json_board_config_filename:
         with open(json_board_config_filename, 'r') as json_file:
-            config = storage.get_config_from_json(json_file.read())
+            config = get_config_from_json(json_file.read())
-            board_config_binary = storage.serialize_config_with_footer(config)
+            board_config_binary = serialize_config_with_footer(config)
 
     if binary_user_config_filename:
-        with open(binary_user_config_filename, 'rb') as binary_file:
+        with open(binary_user_config_filename, 'rb') as storage:
-            user_config_binary = bytearray(binary_file.read())
+            user_config_binary = bytearray(storage.read())
     elif json_user_config_filename:
         with open(json_user_config_filename, 'r') as json_file:
-            config = storage.get_config_from_json(json_file.read())
+            config = get_config_from_json(json_file.read())
-            user_config_binary = storage.serialize_config_with_footer(config)
+            user_config_binary = serialize_config_with_footer(config)
 
     with open(firmware_filename, 'rb') as firmware:
-        firmware_binary = bytearray(firmware.read())
+        new_binary = combine_firmware_and_config(bytearray(firmware.read()), board_config_binary, user_config_binary,
-
-    # create a sequential binary for .bin and USB uses, or index it for .uf2
-    if usb or combined_filename[-4:] != '.uf2':
-        new_binary = combine_firmware_and_config(firmware_binary, board_config_binary, user_config_binary,
                                                  replace_extra=replace_extra)
-    else:
-        # this was kind of fine, but combining multiple calls of convert_binary_to_uf2 produced
-        # incorrect total block counts in the file, which picotool handled with some squirrely
-        # double-output behavior that has me worried it'd cause a real issue, so doing the
-        # crude padding + write of empty blocks, for now...
-        #
-        # new_binary = convert_binary_to_uf2(firmware_binary)
-        # if board_config_binary:
-        #     new_binary += convert_binary_to_uf2(pad_config_to_storage_size(board_config_binary),
-        #                                         start=BOARD_CONFIG_BINARY_LOCATION)
-        # if user_config_binary:
-        #     new_binary += convert_binary_to_uf2(pad_config_to_storage_size(user_config_binary),
-        #                                         start=USER_CONFIG_BINARY_LOCATION)
-        #
-        # the correct way to do the above would be to pass a list of {offset,binary_data} to convert...,
-        # and have it calculate the total block size before starting to write, and then iterating over
-        # the three lists. doable, just not on the top of my mind right now
-        new_binary = storage.convert_binary_to_uf2(combine_firmware_and_config(firmware_binary, board_config_binary,
-                                                                               user_config_binary,
-                                                                               replace_extra=replace_extra))
 
     if combined_filename:
         with open(combined_filename, 'wb') as combined:
-            combined.write(new_binary)
+            if combined_filename[-4:] == '.uf2':
-
+                combined.write(convert_binary_to_uf2(new_binary))
+            else:
+                combined.write(new_binary)
     if usb:
         endpoint_out, endpoint_in = get_bootsel_endpoints()
         write(endpoint_out, endpoint_in, GP2040CE_START_ADDRESS, bytes(new_binary))
 
 
-def find_version_string_in_binary(binary: bytes) -> str:
-    """Search for a git describe style version string in a binary file.
-
-    Args:
-        binary: the binary to search
-    Returns:
-        the first found string, or None
-    """
-    match = re.search(b'v[0-9]+.[0-9]+.[0-9]+[A-Za-z0-9-+.]*', binary)
-    if match:
-        return match.group(0).decode(encoding='ascii')
-    return 'NONE'
-
-
 def get_gp2040ce_from_usb() -> tuple[bytes, object, object]:
     """Read the firmware + config sections from a USB device.
 
@@ -192,7 +157,7 @@ def pad_binary_up_to_board_config(firmware: bytes, or_truncate: bool = False) ->
     Raises:
         FirmwareLengthError: if the firmware is larger than the storage location
     """
-    return pad_binary_up_to_address(firmware, storage.BOARD_CONFIG_BINARY_LOCATION, or_truncate)
+    return pad_binary_up_to_address(firmware, BOARD_CONFIG_BINARY_LOCATION, or_truncate)
 
 
 def pad_binary_up_to_user_config(firmware: bytes, or_truncate: bool = False) -> bytearray:
@@ -206,7 +171,7 @@ def pad_binary_up_to_user_config(firmware: bytes, or_truncate: bool = False) ->
     Raises:
         FirmwareLengthError: if the firmware is larger than the storage location
     """
-    return pad_binary_up_to_address(firmware, storage.USER_CONFIG_BINARY_LOCATION, or_truncate)
+    return pad_binary_up_to_address(firmware, USER_CONFIG_BINARY_LOCATION, or_truncate)
 
 
 def replace_config_in_binary(board_binary: bytearray, config_binary: bytearray) -> bytearray:
@@ -222,14 +187,13 @@ def replace_config_in_binary(board_binary: bytearray, config_binary: bytearray)
     Returns:
         the resulting correctly-offset binary suitable for a GP2040-CE board
     """
-    if len(board_binary) < storage.USER_CONFIG_BINARY_LOCATION + storage.STORAGE_SIZE:
+    if len(board_binary) < USER_CONFIG_BINARY_LOCATION + STORAGE_SIZE:
         # this is functionally the same, since this doesn't sanity check the firmware
         return combine_firmware_and_config(board_binary, bytearray([]), config_binary)
     else:
         new_binary = bytearray(copy.copy(board_binary))
-        new_config = storage.pad_config_to_storage_size(config_binary)
+        new_config = pad_config_to_storage_size(config_binary)
-        location_end = storage.USER_CONFIG_BINARY_LOCATION + storage.STORAGE_SIZE
+        new_binary[USER_CONFIG_BINARY_LOCATION:(USER_CONFIG_BINARY_LOCATION + STORAGE_SIZE)] = new_config
-        new_binary[storage.USER_CONFIG_BINARY_LOCATION:location_end] = new_config
         return new_binary
 
 
@@ -246,18 +210,18 @@ def write_new_config_to_filename(config: Message, filename: str, inject: bool =
                 the whole file is replaced
     """
     if inject:
-        config_binary = storage.serialize_config_with_footer(config)
+        config_binary = serialize_config_with_footer(config)
         with open(filename, 'rb') as file:
             existing_binary = file.read()
         binary = replace_config_in_binary(bytearray(existing_binary), config_binary)
         with open(filename, 'wb') as file:
             file.write(binary)
     else:
-        binary = storage.serialize_config_with_footer(config)
+        binary = serialize_config_with_footer(config)
         with open(filename, 'wb') as file:
             if filename[-4:] == '.uf2':
-                file.write(storage.convert_binary_to_uf2(storage.pad_config_to_storage_size(binary),
+                file.write(convert_binary_to_uf2(pad_config_to_storage_size(binary),
-                                                         start=storage.USER_CONFIG_BINARY_LOCATION))
+                                                 start=USER_CONFIG_BINARY_LOCATION))
             else:
                 file.write(binary)
 
@@ -270,7 +234,7 @@ def write_new_config_to_usb(config: Message, endpoint_out: object, endpoint_in:
         endpoint_out: the USB endpoint to write to
         endpoint_in: the USB endpoint to read from
     """
-    serialized = storage.serialize_config_with_footer(config)
+    serialized = serialize_config_with_footer(config)
     # we don't write the whole area, just the minimum from the end of the storage section
     # nevertheless, the USB device needs writes to start at 256 byte boundaries
     logger.debug("serialized: %s", serialized)
@@ -281,8 +245,7 @@ def write_new_config_to_usb(config: Message, endpoint_out: object, endpoint_in:
     logger.debug("length: %s with %s bytes of padding", len(serialized), padding)
     binary = bytearray(b'\x00' * padding) + serialized
     logger.debug("binary for writing: %s", binary)
-    write(endpoint_out, endpoint_in, storage.USER_CONFIG_BOOTSEL_ADDRESS + (storage.STORAGE_SIZE - len(binary)),
+    write(endpoint_out, endpoint_in, USER_CONFIG_BOOTSEL_ADDRESS + (STORAGE_SIZE - len(binary)), bytes(binary))
-          bytes(binary))
 
 
 ############
@@ -335,40 +298,3 @@ def dump_gp2040ce():
     content, _, _ = get_gp2040ce_from_usb()
     with open(args.binary_filename, 'wb') as out_file:
         out_file.write(content)
-
-
-def summarize_gp2040ce():
-    """Provide information on a dump or USB device."""
-    parser = argparse.ArgumentParser(
-        description="Read a file or USB device to determine what GP2040-CE parts are present.",
-        parents=[core_parser],
-    )
-    input_group = parser.add_mutually_exclusive_group(required=True)
-    input_group.add_argument('--usb', action='store_true', help="inspect the RP2040 device over USB")
-    input_group.add_argument('--filename', help="input .bin or .uf2 file to inspect")
-
-    args, _ = parser.parse_known_args()
-    if args.usb:
-        content, endpoint, _ = get_gp2040ce_from_usb()
-        print(f"USB device {hex(endpoint.device.idVendor)}:{hex(endpoint.device.idProduct)}:\n")
-    else:
-        with open(args.filename, 'rb') as file_:
-            content = file_.read()
-        print(f"File {args.filename}:\n")
-
-    gp2040ce_version = find_version_string_in_binary(content)
-    try:
-        board_config = storage.get_config(storage.get_board_storage_section(bytes(content)))
-        board_config_version = board_config.boardVersion if board_config.boardVersion else "NOT SPECIFIED"
-    except storage.ConfigReadError:
-        board_config_version = "NONE"
-    try:
-        user_config = storage.get_config(storage.get_user_storage_section(bytes(content)))
-        user_config_version = user_config.boardVersion if user_config.boardVersion else "NOT FOUND"
-    except storage.ConfigReadError:
-        user_config_version = "NONE"
-
-    print("GP2040-CE Information")
-    print(f"  detected GP2040-CE version:     {gp2040ce_version}")
-    print(f"  detected board config version:  {board_config_version}")
-    print(f"  detected user config version:   {user_config_version}")

gp2040ce_bintools/gui.py

@@ -54,7 +54,9 @@ class EditScreen(ModalScreen):
             self.input_field = Input(value=repr(self.field_value), validators=[Number()], id='field-input')
         elif self.field_descriptor.type == descriptor.FieldDescriptor.TYPE_STRING:
             self.input_field = Input(value=self.field_value, id='field-input')
-
+        else:
+            # we don't handle whatever these are yet
+            self.input_field = Label(repr(self.field_value), id='field-input')
         yield Grid(
             Container(Label(self.field_descriptor.full_name, id='field-name'), id='field-name-container'),
             Container(self.input_field, id='input-field-container'),
@@ -89,16 +91,17 @@ class EditScreen(ModalScreen):
 
     def _save(self):
         """Save the field value to the retained config item."""
-        if self.field_descriptor.type in (descriptor.FieldDescriptor.TYPE_INT32,
+        if not isinstance(self.input_field, Label):
-                                          descriptor.FieldDescriptor.TYPE_INT64,
+            if self.field_descriptor.type in (descriptor.FieldDescriptor.TYPE_INT32,
-                                          descriptor.FieldDescriptor.TYPE_UINT32,
+                                              descriptor.FieldDescriptor.TYPE_INT64,
-                                          descriptor.FieldDescriptor.TYPE_UINT64):
+                                              descriptor.FieldDescriptor.TYPE_UINT32,
-            field_value = int(self.input_field.value)
+                                              descriptor.FieldDescriptor.TYPE_UINT64):
-        else:
+                field_value = int(self.input_field.value)
-            field_value = self.input_field.value
+            else:
-        setattr(self.parent_config, self.field_descriptor.name, field_value)
+                field_value = self.input_field.value
-        logger.debug("parent config post-change: %s", self.parent_config)
+            setattr(self.parent_config, self.field_descriptor.name, field_value)
-        self.node.set_label(pb_field_to_node_label(self.field_descriptor, field_value))
+            logger.debug("parent config post-change: %s", self.parent_config)
+            self.node.set_label(pb_field_to_node_label(self.field_descriptor, field_value))
 
 
 class MessageScreen(ModalScreen):

gp2040ce_bintools/storage.py

@@ -86,44 +86,6 @@ def convert_binary_to_uf2(binary: bytearray, start: int = 0) -> bytearray:
     return uf2
 
 
-def convert_uf2_to_binary(uf2: bytearray) -> bytearray:
-    """Convert a Microsoft's UF2 payload to a raw binary.
-
-    https://github.com/microsoft/uf2/tree/master#overview
-
-    Args:
-        uf2: bytearray content to convert from a UF2 payload
-    Returns:
-        the content in sequential binary format
-    """
-    if len(uf2) % 512 != 0:
-        raise ValueError(f"provided binary is length {len(uf2)}, which isn't fully divisible by 512!")
-
-    binary = bytearray()
-    old_uf2_addr = None
-
-    for index in range(0, len(uf2), 512):
-        chunk = uf2[index:index+512]
-        _, _, _, uf2_addr, bytes_, block_num, block_count, _ = struct.unpack('<LLLLLLLL', chunk[0:32])
-        content = chunk[32:508]
-        if block_num != index // 512:
-            raise ValueError(f"inconsistent block number in reading UF2, got {block_num}, expected {index // 512}!")
-        if block_count != len(uf2) // 512:
-            raise ValueError(f"inconsistent block count in reading UF2, got {block_count}, expected {len(uf2) // 512}!")
-
-        # the UF2 is indexed, which we could convert to binary with padding 0s, but we don't yet
-        if old_uf2_addr and (uf2_addr != old_uf2_addr + 256):
-            raise ValueError("segmented UF2 files are not yet supported!")
-
-        binary += content[0:bytes_]
-        old_uf2_addr = uf2_addr
-
-    # when this is all done we should have counted the expected number of blocks
-    if block_count != block_num + 1:
-        raise ValueError(f"not all expected blocks ({block_count}) were found, only got {block_num + 1}!")
-    return binary
-
-
 def get_config(content: bytes) -> Message:
     """Read the config from a GP2040-CE storage section.
 
@@ -216,10 +178,7 @@ def get_config_from_file(filename: str, whole_board: bool = False, allow_no_file
     """
     try:
         with open(filename, 'rb') as dump:
-            if filename[-4:] == '.uf2':
+            content = dump.read()
-                content = bytes(convert_uf2_to_binary(bytearray(dump.read())))
-            else:
-                content = dump.read()
     except FileNotFoundError:
         if not allow_no_file:
             raise
@@ -365,15 +324,10 @@ def dump_config():
         description="Read the configuration section from a USB device and save it to a binary file.",
         parents=[core_parser],
     )
-    parser.add_argument('--board-config', action='store_true', default=False,
-                        help="dump the board config rather than the user config")
     parser.add_argument('filename', help="file to save the GP2040-CE board's config section to --- if the "
                                          "suffix is .uf2, it is saved in UF2 format, else it is a raw binary")
     args, _ = parser.parse_known_args()
-    if args.board_config:
+    config, _, _ = get_user_config_from_usb()
-        config, _, _ = get_board_config_from_usb()
-    else:
-        config, _, _ = get_user_config_from_usb()
     binary_config = serialize_config_with_footer(config)
     with open(args.filename, 'wb') as out_file:
         if args.filename[-4:] == '.uf2':

pyproject.toml

@@ -38,8 +38,7 @@ concatenate = "gp2040ce_bintools.builder:concatenate"
 dump-config = "gp2040ce_bintools.storage:dump_config"
 dump-gp2040ce = "gp2040ce_bintools.builder:dump_gp2040ce"
 edit-config = "gp2040ce_bintools.gui:edit_config"
-summarize-gp2040ce = "gp2040ce_bintools.builder:summarize_gp2040ce"
+visualize-storage = "gp2040ce_bintools.storage:visualize"
-visualize-config = "gp2040ce_bintools.storage:visualize"
 
 [tool.flake8]
 enable-extensions = "G,M"

tests/test_builder.py

@@ -10,8 +10,11 @@ import unittest.mock as mock
 import pytest
 from decorator import decorator
 
-import gp2040ce_bintools.builder as builder
 from gp2040ce_bintools import get_config_pb2
+from gp2040ce_bintools.builder import (FirmwareLengthError, combine_firmware_and_config,
+                                       concatenate_firmware_and_storage_files, get_gp2040ce_from_usb,
+                                       pad_binary_up_to_board_config, pad_binary_up_to_user_config,
+                                       replace_config_in_binary, write_new_config_to_filename, write_new_config_to_usb)
 from gp2040ce_bintools.storage import (STORAGE_SIZE, get_board_storage_section, get_config, get_config_footer,
                                        get_user_storage_section, serialize_config_with_footer)
 
@@ -35,8 +38,8 @@ def test_concatenate_to_file(tmp_path):
     tmp_file = os.path.join(tmp_path, 'concat.bin')
     firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
     config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
-    builder.concatenate_firmware_and_storage_files(firmware_file, binary_user_config_filename=config_file,
+    concatenate_firmware_and_storage_files(firmware_file, binary_user_config_filename=config_file,
-                                                   combined_filename=tmp_file)
+                                           combined_filename=tmp_file)
     with open(tmp_file, 'rb') as file:
         content = file.read()
     assert len(content) == 2 * 1024 * 1024
@@ -47,8 +50,8 @@ def test_concatenate_board_config_to_file(tmp_path):
     tmp_file = os.path.join(tmp_path, 'concat.bin')
     firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
     config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
-    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+    concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
-                                                   combined_filename=tmp_file)
+                                           combined_filename=tmp_file)
     with open(tmp_file, 'rb') as file:
         content = file.read()
     assert len(content) == (2 * 1024 * 1024) - (16 * 1024)
@@ -59,8 +62,8 @@ def test_concatenate_both_configs_to_file(tmp_path):
     tmp_file = os.path.join(tmp_path, 'concat.bin')
     firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
     config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
-    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+    concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
-                                                   binary_user_config_filename=config_file, combined_filename=tmp_file)
+                                           binary_user_config_filename=config_file, combined_filename=tmp_file)
     with open(tmp_file, 'rb') as file:
         content = file.read()
     assert len(content) == 2 * 1024 * 1024
@@ -78,8 +81,8 @@ def test_concatenate_user_json_to_file(tmp_path):
     tmp_file = os.path.join(tmp_path, 'concat.bin')
     firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
     config_file = os.path.join(HERE, 'test-files', 'test-config.json')
-    builder.concatenate_firmware_and_storage_files(firmware_file, json_user_config_filename=config_file,
+    concatenate_firmware_and_storage_files(firmware_file, json_user_config_filename=config_file,
-                                                   combined_filename=tmp_file)
+                                           combined_filename=tmp_file)
     with open(tmp_file, 'rb') as file:
         content = file.read()
     assert len(content) == 2 * 1024 * 1024
@@ -90,7 +93,7 @@ def test_concatenate_to_file_incomplete_args_is_error(tmp_path):
     tmp_file = os.path.join(tmp_path, 'concat.bin')
     firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
     with pytest.raises(ValueError):
-        builder.concatenate_firmware_and_storage_files(firmware_file, combined_filename=tmp_file)
+        concatenate_firmware_and_storage_files(firmware_file, combined_filename=tmp_file)
 
 
 def test_concatenate_to_usb(tmp_path):
@@ -100,71 +103,34 @@ def test_concatenate_to_usb(tmp_path):
     end_out, end_in = mock.MagicMock(), mock.MagicMock()
     with mock.patch('gp2040ce_bintools.builder.get_bootsel_endpoints', return_value=(end_out, end_in)):
         with mock.patch('gp2040ce_bintools.builder.write') as mock_write:
-            builder.concatenate_firmware_and_storage_files(firmware_file, binary_user_config_filename=config_file,
+            concatenate_firmware_and_storage_files(firmware_file, binary_user_config_filename=config_file,
-                                                           usb=True)
+                                                   usb=True)
 
     assert mock_write.call_args.args[2] == 0x10000000
     assert len(mock_write.call_args.args[3]) == 2 * 1024 * 1024
 
 
-def test_concatenate_to_uf2(tmp_path, firmware_binary, config_binary):
-    """Test that we write a UF2 file as expected."""
-    tmp_file = os.path.join(tmp_path, 'concat.uf2')
-    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
-    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
-    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
-                                                   binary_user_config_filename=config_file,
-                                                   combined_filename=tmp_file)
-    with open(tmp_file, 'rb') as file:
-        content = file.read()
-    # size of the file should be 2x the binary version, and the binary is 2 MB
-    assert len(content) == 2 * 2 * 1024 * 1024
-
-
-def test_concatenate_to_uf2_board_only(tmp_path, firmware_binary, config_binary):
-    """Test that we write a UF2 file as expected."""
-    tmp_file = os.path.join(tmp_path, 'concat.uf2')
-    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
-    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
-    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
-                                                   combined_filename=tmp_file)
-    with open(tmp_file, 'rb') as file:
-        content = file.read()
-    # size of the file should be 2x the binary version (minus user config space), and the binary is 2 MB - 16KB
-    assert len(content) == 2 * (2 * 1024 * 1024 - 16384)
-
-
-def test_find_version_string(firmware_binary):
-    """Test that we can find a version string in a binary."""
-    assert builder.find_version_string_in_binary(firmware_binary) == 'v0.7.5'
-
-
-def test_dont_always_find_version_string(firmware_binary):
-    """Test that we can find a version string in a binary."""
-    assert builder.find_version_string_in_binary(b'\x00') == 'NONE'
-
-
 def test_padding_firmware(firmware_binary):
     """Test that firmware is padded to the expected size."""
-    padded = builder.pad_binary_up_to_user_config(firmware_binary)
+    padded = pad_binary_up_to_user_config(firmware_binary)
     assert len(padded) == 2080768
 
 
 def test_padding_firmware_can_truncate():
     """Test that firmware is padded to the expected size."""
-    padded = builder.pad_binary_up_to_user_config(bytearray(b'\x00' * 4 * 1024 * 1024), or_truncate=True)
+    padded = pad_binary_up_to_user_config(bytearray(b'\x00' * 4 * 1024 * 1024), or_truncate=True)
     assert len(padded) == 2080768
 
 
 def test_padding_firmware_to_board(firmware_binary):
     """Test that firmware is padded to the expected size."""
-    padded = builder.pad_binary_up_to_board_config(firmware_binary)
+    padded = pad_binary_up_to_board_config(firmware_binary)
     assert len(padded) == 2080768 - (16 * 1024)
 
 
 def test_firmware_plus_storage_section(firmware_binary, storage_dump):
     """Test that combining firmware and storage produces a valid combined binary."""
-    whole_board = builder.combine_firmware_and_config(firmware_binary, None, storage_dump)
+    whole_board = combine_firmware_and_config(firmware_binary, None, storage_dump)
     # if this is valid, we should be able to find the storage and footer again
     storage = get_user_storage_section(whole_board)
     footer_size, _, _ = get_config_footer(storage)
@@ -173,7 +139,7 @@ def test_firmware_plus_storage_section(firmware_binary, storage_dump):
 
 def test_firmware_plus_user_config_binary(firmware_binary, config_binary):
     """Test that combining firmware and user config produces a valid combined binary."""
-    whole_board = builder.combine_firmware_and_config(firmware_binary, None, config_binary)
+    whole_board = combine_firmware_and_config(firmware_binary, None, config_binary)
     # if this is valid, we should be able to find the storage and footer again
     storage = get_user_storage_section(whole_board)
     footer_size, _, _ = get_config_footer(storage)
@@ -182,8 +148,8 @@ def test_firmware_plus_user_config_binary(firmware_binary, config_binary):
 
 def test_chunky_firmware_plus_user_config_binary(config_binary):
     """Test that combining giant firmware and storage produces a valid combined binary."""
-    whole_board = builder.combine_firmware_and_config(bytearray(b'\x00' * 4 * 1024 * 1024), None, config_binary,
+    whole_board = combine_firmware_and_config(bytearray(b'\x00' * 4 * 1024 * 1024), None, config_binary,
-                                                      replace_extra=True)
+                                              replace_extra=True)
     # if this is valid, we should be able to find the storage and footer again
     storage = get_user_storage_section(whole_board)
     footer_size, _, _ = get_config_footer(storage)
@@ -192,7 +158,7 @@ def test_chunky_firmware_plus_user_config_binary(config_binary):
 
 def test_firmware_plus_board_config_binary(firmware_binary, config_binary):
     """Test that combining firmware and board config produces a valid combined binary."""
-    almost_whole_board = builder.combine_firmware_and_config(firmware_binary, config_binary, None)
+    almost_whole_board = combine_firmware_and_config(firmware_binary, config_binary, None)
     assert len(almost_whole_board) == (2 * 1024 * 1024) - (16 * 1024)
     # if this is valid, we should be able to find the storage and footer again
     storage = get_board_storage_section(almost_whole_board)
@@ -202,7 +168,7 @@ def test_firmware_plus_board_config_binary(firmware_binary, config_binary):
 
 def test_firmware_plus_board_and_user_config_binary(firmware_binary, config_binary):
     """Test that combining firmware and both board and user configs produces a valid combined binary."""
-    whole_board = builder.combine_firmware_and_config(firmware_binary, config_binary, config_binary)
+    whole_board = combine_firmware_and_config(firmware_binary, config_binary, config_binary)
     assert len(whole_board) == 2 * 1024 * 1024
     # if this is valid, we should be able to find the storage and footer again
     storage = get_board_storage_section(whole_board)
@@ -216,12 +182,12 @@ def test_firmware_plus_board_and_user_config_binary(firmware_binary, config_bina
 def test_combine_must_get_at_least_one_config(firmware_binary):
     """Test that we error if we are asked to combine with nothing to combine."""
     with pytest.raises(ValueError):
-        builder.combine_firmware_and_config(firmware_binary, None, None)
+        combine_firmware_and_config(firmware_binary, None, None)
 
 
 def test_replace_config_in_binary(config_binary):
     """Test that a config binary is placed in the storage location of a source binary to overwrite."""
-    whole_board = builder.replace_config_in_binary(bytearray(b'\x00' * 3 * 1024 * 1024), config_binary)
+    whole_board = replace_config_in_binary(bytearray(b'\x00' * 3 * 1024 * 1024), config_binary)
     assert len(whole_board) == 3 * 1024 * 1024
     # if this is valid, we should be able to find the storage and footer again
     storage = get_user_storage_section(whole_board)
@@ -231,7 +197,7 @@ def test_replace_config_in_binary(config_binary):
 
 def test_replace_config_in_binary_not_big_enough(config_binary):
     """Test that a config binary is placed in the storage location of a source binary to pad."""
-    whole_board = builder.replace_config_in_binary(bytearray(b'\x00' * 1 * 1024 * 1024), config_binary)
+    whole_board = replace_config_in_binary(bytearray(b'\x00' * 1 * 1024 * 1024), config_binary)
     assert len(whole_board) == 2 * 1024 * 1024
     # if this is valid, we should be able to find the storage and footer again
     storage = get_user_storage_section(whole_board)
@@ -241,8 +207,8 @@ def test_replace_config_in_binary_not_big_enough(config_binary):
 
 def test_padding_firmware_too_big(firmware_binary):
     """Test that firmware is padded to the expected size."""
-    with pytest.raises(builder.FirmwareLengthError):
+    with pytest.raises(FirmwareLengthError):
-        _ = builder.pad_binary_up_to_user_config(firmware_binary + firmware_binary + firmware_binary)
+        _ = pad_binary_up_to_user_config(firmware_binary + firmware_binary + firmware_binary)
 
 
 @with_pb2s
@@ -258,7 +224,7 @@ def test_write_new_config_to_whole_board(whole_board_dump, tmp_path):
     config = get_config(get_user_storage_section(board_dump))
     assert config.boardVersion == 'v0.7.5'
     config.boardVersion = 'v0.7.5-COOL'
-    builder.write_new_config_to_filename(config, tmp_file, inject=True)
+    write_new_config_to_filename(config, tmp_file, inject=True)
 
     # read new file
     with open(tmp_file, 'rb') as file:
@@ -278,7 +244,7 @@ def test_write_new_config_to_firmware(firmware_binary, tmp_path):
     config_pb2 = get_config_pb2()
     config = config_pb2.Config()
     config.boardVersion = 'v0.7.5-COOL'
-    builder.write_new_config_to_filename(config, tmp_file, inject=True)
+    write_new_config_to_filename(config, tmp_file, inject=True)
 
     # read new file
     with open(tmp_file, 'rb') as file:
@@ -295,7 +261,7 @@ def test_write_new_config_to_config_bin(firmware_binary, tmp_path):
     config_pb2 = get_config_pb2()
     config = config_pb2.Config()
     config.boardVersion = 'v0.7.5-COOL'
-    builder.write_new_config_to_filename(config, tmp_file)
+    write_new_config_to_filename(config, tmp_file)
 
     # read new file
     with open(tmp_file, 'rb') as file:
@@ -313,7 +279,7 @@ def test_write_new_config_to_config_uf2(firmware_binary, tmp_path):
     config_pb2 = get_config_pb2()
     config = config_pb2.Config()
     config.boardVersion = 'v0.7.5-COOL'
-    builder.write_new_config_to_filename(config, tmp_file)
+    write_new_config_to_filename(config, tmp_file)
 
     # read new file
     with open(tmp_file, 'rb') as file:
@@ -330,7 +296,7 @@ def test_write_new_config_to_usb(config_binary):
     serialized = serialize_config_with_footer(config)
     end_out, end_in = mock.MagicMock(), mock.MagicMock()
     with mock.patch('gp2040ce_bintools.builder.write') as mock_write:
-        builder.write_new_config_to_usb(config, end_out, end_in)
+        write_new_config_to_usb(config, end_out, end_in)
 
     # check that it got padded
     assert len(serialized) == 3321
@@ -349,7 +315,7 @@ def test_get_gp2040ce_from_usb():
     mock_in = mock.MagicMock()
     with mock.patch('gp2040ce_bintools.builder.get_bootsel_endpoints', return_value=(mock_out, mock_in)) as mock_get:
         with mock.patch('gp2040ce_bintools.builder.read') as mock_read:
-            config, _, _ = builder.get_gp2040ce_from_usb()
+            config, _, _ = get_gp2040ce_from_usb()
 
     mock_get.assert_called_once()
     mock_read.assert_called_with(mock_out, mock_in, 0x10000000, 2 * 1024 * 1024)

tests/test_commands.py

@@ -29,14 +29,14 @@ def with_pb2s(test, *args, **kwargs):
 
 def test_version_flag():
     """Test that tools report the version."""
-    result = run(['visualize-config', '-v'], capture_output=True, encoding='utf8')
+    result = run(['visualize-storage', '-v'], capture_output=True, encoding='utf8')
     assert __version__ in result.stdout
 
 
 def test_help_flag():
     """Test that tools report the usage information."""
-    result = run(['visualize-config', '-h'], capture_output=True, encoding='utf8')
+    result = run(['visualize-storage', '-h'], capture_output=True, encoding='utf8')
-    assert 'usage: visualize-config' in result.stdout
+    assert 'usage: visualize-storage' in result.stdout
     assert 'Read the configuration section from a dump of a GP2040-CE board' in result.stdout
 
 
@@ -64,16 +64,9 @@ def test_concatenate_invocation_json(tmpdir):
     assert out[2093382:2097152] == storage
 
 
-def test_summarize_invocation(tmpdir):
-    """Test that we can get some summary information."""
-    result = run(['summarize-gp2040ce', '--filename', 'tests/test-files/test-firmware.bin'],
-                 capture_output=True, encoding='utf8')
-    assert 'detected GP2040-CE version:     v0.7.5' in result.stdout
-
-
 def test_storage_dump_invocation():
     """Test that a normal invocation against a dump works."""
-    result = run(['visualize-config', '-P', 'tests/test-files/proto-files',
+    result = run(['visualize-storage', '-P', 'tests/test-files/proto-files',
                   '--filename', 'tests/test-files/test-storage-area.bin'],
                  capture_output=True, encoding='utf8')
     assert 'boardVersion: "v0.7.5"' in result.stdout
@@ -81,7 +74,7 @@ def test_storage_dump_invocation():
 
 def test_debug_storage_dump_invocation():
     """Test that a normal invocation against a dump works."""
-    result = run(['visualize-config', '-d', '-P', 'tests/test-files/proto-files',
+    result = run(['visualize-storage', '-d', '-P', 'tests/test-files/proto-files',
                   '--filename', 'tests/test-files/test-storage-area.bin'],
                  capture_output=True, encoding='utf8')
     assert 'boardVersion: "v0.7.5"' in result.stdout
@@ -90,7 +83,7 @@ def test_debug_storage_dump_invocation():
 
 def test_storage_dump_json_invocation():
     """Test that a normal invocation against a dump works."""
-    result = run(['visualize-config', '-P', 'tests/test-files/proto-files', '--json',
+    result = run(['visualize-storage', '-P', 'tests/test-files/proto-files', '--json',
                   '--filename', 'tests/test-files/test-storage-area.bin'],
                  capture_output=True, encoding='utf8')
     to_dict = json.loads(result.stdout)

tests/test_gui.py

@@ -125,41 +125,6 @@ async def test_simple_edit_via_input_field():
         assert pilot.app.config.displayOptions.deprecatedI2cSpeed == 5
 
 
-@pytest.mark.asyncio
-@with_pb2s
-async def test_cancel_simple_edit_via_input_field():
-    """Test that we can cancel out of saving an int via UI and see it reflected in the config."""
-    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
-    async with app.run_test() as pilot:
-        tree = pilot.app.query_one(Tree)
-        display_node = tree.root.children[5]
-        i2cspeed_node = display_node.children[4]
-        assert pilot.app.config.displayOptions.deprecatedI2cSpeed == 400000
-
-        tree.root.expand_all()
-        await pilot.wait_for_scheduled_animations()
-        tree.select_node(i2cspeed_node)
-        tree.action_select_cursor()
-        await pilot.wait_for_scheduled_animations()
-        await pilot.click('Input#field-input')
-        await pilot.wait_for_scheduled_animations()
-        await pilot.press('backspace', 'backspace', 'backspace', 'backspace', 'backspace', 'backspace', '5')
-        await pilot.wait_for_scheduled_animations()
-        await pilot.click('Button#cancel-button')
-        assert pilot.app.config.displayOptions.deprecatedI2cSpeed == 400000
-
-
-@pytest.mark.asyncio
-@with_pb2s
-async def test_about():
-    """Test that we can bring up the about box."""
-    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
-    async with app.run_test() as pilot:
-        await pilot.press('?')
-        await pilot.wait_for_scheduled_animations()
-        await pilot.click('Button#ok-button')
-
-
 @pytest.mark.asyncio
 @with_pb2s
 async def test_simple_edit_via_input_field_enum():

tests/test_storage.py

@@ -3,7 +3,6 @@
 SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
 SPDX-License-Identifier: GPL-3.0-or-later
 """
-import math
 import os
 import sys
 import unittest.mock as mock
@@ -140,15 +139,6 @@ def test_convert_binary_to_uf2(whole_board_with_board_config_dump):
     assert uf2[524:528] == bytearray(b'\x00\x01\x00\x10')   # address to write the second chunk
 
 
-def test_convert_unaligned_binary_to_uf2(firmware_binary):
-    """Do some sanity checks in the attempt to convert a binary to a UF2."""
-    uf2 = storage.convert_binary_to_uf2(firmware_binary)
-    assert len(uf2) == math.ceil(len(firmware_binary)/256) * 512    # 256 byte complete/partial chunks -> 512 b chunks
-    assert uf2[0:4] == b'\x55\x46\x32\x0a' == b'UF2\n'      # proper magic
-    assert uf2[8:12] == bytearray(b'\x00\x20\x00\x00')      # family ID set
-    assert uf2[524:528] == bytearray(b'\x00\x01\x00\x10')   # address to write the second chunk
-
-
 def test_convert_binary_to_uf2_with_offsets(whole_board_with_board_config_dump):
     """Do some sanity checks in the attempt to convert a binary to a UF2."""
     uf2 = storage.convert_binary_to_uf2(whole_board_with_board_config_dump, start=storage.USER_CONFIG_BINARY_LOCATION)
@@ -158,35 +148,6 @@ def test_convert_binary_to_uf2_with_offsets(whole_board_with_board_config_dump):
     assert uf2[524:528] == bytearray(b'\x00\xc1\x1f\x10')   # address to write the second chunk
 
 
-def test_convert_binary_to_uf2_to_binary(whole_board_with_board_config_dump):
-    """Do some sanity checks in the attempt to convert a binary to a UF2."""
-    uf2 = storage.convert_binary_to_uf2(whole_board_with_board_config_dump)
-    binary = storage.convert_uf2_to_binary(uf2)
-    assert len(binary) == 2097152
-    assert whole_board_with_board_config_dump == binary
-
-
-def test_malformed_uf2(whole_board_with_board_config_dump):
-    """Check that we expect a properly-formed UF2."""
-    uf2 = storage.convert_binary_to_uf2(whole_board_with_board_config_dump)
-
-    # truncated UF2 --- byte mismatch
-    with pytest.raises(ValueError):
-        storage.convert_uf2_to_binary(uf2[:-4])
-
-    # truncated uf2 --- counter is wrong
-    with pytest.raises(ValueError):
-        storage.convert_uf2_to_binary(uf2[512:])
-
-    # truncated uf2 --- total count is wrong
-    with pytest.raises(ValueError):
-        storage.convert_uf2_to_binary(uf2[:-512])
-
-    # malformed UF2 --- counter jumps in the middle, suggests total blocks is wrong
-    with pytest.raises(ValueError):
-        storage.convert_uf2_to_binary(uf2 + uf2)
-
-
 @with_pb2s
 def test_serialize_config_with_footer(storage_dump, config_binary):
     """Test that reserializing a read in config matches the original.